https://orcid.org/0000-0001-9926-8442
Figures
Abstract
Introduction
In community-based research projects, needs assessments are one of the first steps to identify community priorities. Access-related issues often pose significant barriers to participation in research and evaluation for rural and remote communities, particularly Indigenous communities, which also have a complex relationship with academia due to a history of exploitation. To bridge this gap, work with Indigenous communities requires consistent and meaningful engagement. The prominence of digital devices (i.e., smartphones) offers an unparalleled opportunity for ethical and equitable engagement between researchers and communities across jurisdictions, particularly in remote communities.
Methods
This paper presents a framework to guide needs assessments which embed digital platforms in partnership with Indigenous communities. Guided by this framework, a qualitative needs assessment was conducted with a subarctic Métis community in Saskatchewan, Canada. This project is governed by an Advisory Council comprised of Knowledge Keepers, Elders, and youth in the community. An environmental scan of relevant programs, three key informant interviews, and two focus groups (n = 4 in each) were conducted to systematically identify community priorities.
Results
Through discussions with the community, four priorities were identified: (1) the Coronavirus pandemic, (2) climate change impacts on the environment, (3) mental health and wellbeing, and (4) food security and sovereignty. Given the timing of the needs assessment, the community identified the Coronavirus pandemic as a key priority requiring digital initiatives.
Citation: Bhawra J, Buchan MC, Green B, Skinner K, Katapally TR (2022) A guiding framework for needs assessment evaluations to embed digital platforms in partnership with Indigenous communities. PLoS ONE 17(12): e0279282. https://doi.org/10.1371/journal.pone.0279282
Editor: Stephane Shepherd, Swinburne University of Technology, AUSTRALIA
Received: June 1, 2022; Accepted: December 2, 2022; Published: December 22, 2022
Copyright: © 2022 Bhawra et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are co-owned by the community and all data requests should be approved by the Citizen Scientist Advisory Council and the University of Regina Research Office. Citizen Scientist Advisory Council Contact: Mr. Duane Favel, Mayor of Ile-a-lacrosse, email: dfavel@sasktel.net; duane.favel.ile@gmail.com University of Regina Research Office contact: Ara Steininger, Research Compliance Officer; E-mail: ara.steininger@uregina.ca. Those interested can access the data in the same manner as the authors.
Funding: TRK received funding from the Canadian Institutes of Health Research (CIHR) and the Canada Research Chairs Program to conduct this research. The funding organization had no role to play in any part of the study implementation of manuscript generation.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Community engagement has been the cornerstone of participatory action research in a range of disciplines. Every community has a unique culture and identity, hence community members are the experts regarding their diverse histories, priorities, and growth [1–3]. As a result, the successful uptake, implementation, and longevity of community-based research initiatives largely depends on meaningful community engagement [4–9]. There is a considerable body of evidence establishing the need for ethical community-research partnerships which empower citizens and ensure relevant and sustainable solutions [1–3, 10]. For groups that have been marginalized or disadvantaged, community-engaged research that prioritizes citizens’ control in the research process can provide a platform to amplify citizens’ voices and ensure necessary representation in decision-making [11]. Such initiatives must be developed in alignment with a community’s cultural framework, expectations, and vision [12] to support continuous and meaningful engagement throughout the project. In particular, when partnering with Indigenous communities, a Two-Eyed Seeing approach can provide valuable perspective to combine the strengths of Indigenous and Western Knowledges, including culturally relevant methods, technologies, and tools [13–15].
Many communities have a complicated relationship with research as a result of colonialism, and the trauma of exploitation and discrimination has continued to limit the participation of some communities in academic partnerships [16]. Indigenous Peoples in Canada experience a disproportionate number of health, economic, and social inequalities compared to non-Indigenous Canadians [17]. Many of these health (e.g., elevated risk of chronic and communicable diseases) [18–21]), socioeconomic (e.g., elevated levels of unemployment and poverty) [19, 22–24], and social (e.g., racism and discrimination) [19, 22–24]) inequities can be traced back to the long-term impacts of assimilation, colonization, residential schools, and a lack of access to healthcare [19, 20, 22–24]. To bridge this gap, and more importantly, to work towards Truth and Reconciliation [25], work with Indigenous Peoples must be community-driven, and community-academia relationship building is essential before exploring co-conceptualization of initiatives [26].
One of the first steps in building a relationship is to learn more about community priorities by conducting a needs assessment [27, 28]. A needs assessment is a research and evaluation method for identifying areas for improvement or gaps in current policies, programs, and services [29]. When conducted in partnership with a specific community, needs assessments can identify priorities and be used to develop innovative solutions, while leveraging the existing knowledge and systems that communities have in place [30]. Needs assessments pave the path for understanding the value and applicability of research for community members, incorporating key perspectives, and building authentic partnerships with communities to support effective translation of research into practice.
For rural, remote, and northern communities within Canada, issues related to access (e.g., geographic location, transportation, methods of communication, etc.) pose significant barriers to participation in research and related initiatives [31]. Digital devices, and in particular, the extensive usage of smartphones [32] offers a new opportunity to ethically and equitably engage citizens [33]. Digital platforms (also referred to as digital tools) are applications and software programs accessible through digital devices. Digital platforms can be used for a variety of purposes, ranging from project management, to healthcare delivery or mass communication [34]. Digital infrastructure–the larger systems which support access and use of these digital platforms, including internet, satellites, cellular networks, and data storage centres [34]. The Coronavirus (COVID-19) pandemic has catalyzed the expansion of digital technology, infrastructure and the use of digital devices in delivering essential services (e.g., healthcare) and programs to communities [35, 36].
While digital platforms have been used in Indigenous communities for numerous initiatives, including environmental mapping initiatives (e.g., research and monitoring, land use planning, and wildlife and harvest studies) [37, 38] and telehealth [39], there has largely been isolated app development without a corresponding investment in digital infrastructure. This approach limits the sustainability of digital initiatives, and importantly does not acknowledge an Indigenous world view of holistic solutions [39].
Thus given the increasing prominence of digital devices [39, 40], it is critical to evaluate the conceptualization, implementation, and knowledge dissemination of digital platforms. To date, there is little guidance on how to evaluate digital platforms, particularly in partnership with rural and remote communities [41]. A review of recent literature on community-based needs assessments uncovered numerous resources for conducting evaluations of digital platforms, however, a key gap is the lack of practical guidance for conducting needs assessments in close collaboration with communities in ways that acknowledge existing needs, resources, supports and infrastructure that also incorporates the potential role of digital platforms in addressing community priorities.
This paper aims to provide researchers and evaluators with a framework (step-by-step guide) to conduct needs assessments for digital platforms in collaboration with Indigenous communities. To achieve this goal, a novel needs assessment framework was developed using a Two-Eyed Seeing approach [13–15] to enable the identification of community priorities, barriers and supports, as well as existing digital infrastructure to successfully implement digital solutions. To demonstrate the application of this framework, a community-engaged needs assessment conducted with a subarctic Indigenous community in Canada is described and discussed in detail.
Framework design and development
This project commenced with the design and development of a new framework to guide community-based needs assessments in the digital age.
Needs assessments
Needs assessments are a type of formative evaluation and are often considered a form of strategic or program planning, even more than they are considered a type of evaluation. Needs assessments can occur both before and during an evaluation or program implementation; however, needs assessments are most effective when they are conducted before a new initiative begins or before a decision is made about what to do (e.g., how to make program changes) [29]. Typically, a needs assessment includes: 1) collecting information about a community; 2) determining what needs are already being met; and 3) determining what needs are not being met and what resources are available to meet those needs [42].
Framework development
Based on existing literature, community consultation, and drawing expertise from our team of evaluation experts who have over a decade of experience working with Indigenous communities on a range of research and evaluation projects, a novel framework was developed to guide community-based needs assessments focused on the application of digital platforms.
This framework (see Fig 1) is driven by core questions necessary to identify community priorities that can be addressed by developing and implementing digital platforms. Through team discussion and community consultation, five key topic areas for the assessment of community needs were identified: i) current supports; ii) desired supports; iii) barriers; iv) community engagement; and v) digital access and connectivity. A series of general questions across the five needs assessment topic areas were developed. Thereafter, a set of sub-questions were embedded in each key topic area.
The Guiding Framework outlines an approach for conducting community needs assessments which can be adapted across communities and jurisdictions. This framework offers a flexible template that can be used iteratively and applied to various community-engaged needs assessments in a range of areas, including but not limited to community health and wellness projects. The questions assigned to each topic area can be used to guide needs assessments of any priority identified by community stakeholders as suitable for addressing with digital platforms.
Needs assessment methods
The Guiding Framework was implemented in collaboration with a subarctic Indigenous community in Canada, and was used to identify key community priorities, barriers, supports, and existing digital infrastructure which could inform the design and implementation of tailored digital platforms.
Design
Using an environmental scan of relevant documents and qualitative focus groups and interviews, a needs assessment was conducted with the Northern Village of Île-à-la-Crosse, Saskatchewan, Canada between February and May 2020.
This project is governed by a Citizen Scientist Advisory Council which included researchers, Knowledge Keepers, Elders, and youth from Île-à-la-Crosse. The study PI (TRK) and Co-Investigator (JB) developed a relationship with key decision-makers in Île-à-la-Crosse in 2020. Through their guidance and several community visits, the decision-makers introduced the research team to Elders, youth, and other community members to gain a better understanding of current priorities and needs in Île-à-la-Crosse. The research team developed relationships with these community members and invited them to join the Council to formally capture feedback and plan ongoing projects to promote health and wellbeing in the community. The Council represents the needs and interests of the community, and guides the project development, implementation, and evaluation. Council members were provided with Can $150 (US $119.30) as honoraria for each meeting to respect their time, knowledge, and contributions.
Written consent was obtained from all focus group participants and verbal consent was obtained from all key informants participating in interviews. This study received ethics clearance from the research ethics boards of the University of Regina and the University of Saskatchewan through a synchronized review protocol (REB# 2017–29).
Setting
Established in 1776, Île-à-la-Crosse is a northern subarctic community with road access in northwest Saskatchewan. Sakitawak, the Cree name for Île-à-la-Crosse, means “where the rivers meet,” hence the community was an historically important meeting point for the fur trade in the 1800s [43, 44] The community lies on a peninsula on the Churchill River, near the intersections with the Beaver River and Canoe River systems. Île-à-la-Crosse has a rich history dating back to the fur trade. Due to its strategic location, Montreal-based fur traders established the first trading point in Île-à-la-Crosse in 1776, making the community Saskatchewan’s oldest continually inhabited community next to Cumberland House [45]. In 1821, Île-à-la-Crosse became the headquarters for the Hudson’s Bay Company’s operations in the territory. In 1860, the first convent was established bringing Western culture, medical services, and education to the community.
Île-à-la-Crosse has a population of roughly 1,300 people [19]. Consistent with Indigenous populations across Canada, the average age of the community is 32.7 years, roughly 10 years younger than the Canadian non-Indigenous average [19]. Census data report that just under half (44%) of the community’s population is under the age of 25, 46.3% are aged 25–64, and 9.3% aged 65 and over [19]. Members of the community predominantly identify as Métis (77%), with some identifying as First Nations (18%), multiple Indigenous responses (1.2%), and non-Indigenous (2.7%) [19]. Many community members are employed in a traditional manner utilizing resources of the land (e.g., hunting, fishing, trapping), others in a less traditional manner (e.g., lumbering, tourism, wild rice harvesting), and some are employed through the hospital and schools. The community currently has one elementary school with approximately 200 students from preschool to Grade 6, and one high school serving Grades 7–12 with adult educational programming. Île-à-la-Crosse has a regional hospital with Emergency Services, which includes a health services centre with a total of 29 beds. Other infrastructure of the community includes a Royal Canadian Mounted Police (RCMP) station, a village office, volunteer fire brigade, and a catholic church [46].
Needs assessment approach
Île-à-la-Crosse shared their vision of integrating digital technology and infrastructure as part of its growth, thus the needs assessment was identified as an appropriate method to provide the formative information necessary to understand what the needs are, including who (i.e., players, partners), and what (i.e., information sources) would need to be involved, what opportunities exist to address the needs, and setting priorities for action with key community stakeholders [47]. As a starting point and rationale for this needs assessment, the community of Île-à-la-Crosse values the potential of technology for improving health communication, information reach, access to resources, and care, and was interested in identifying priorities to begin building digital infrastructure. Given the timing of the COVID-19 pandemic, being responsive to community health needs were key priorities that they wanted to start addressing using a digital platform. This needs assessment facilitated and enabled new conversations around key priorities and next steps.
The evaluation approach was culturally-responsive and included empowerment principles [48–50]. Empowerment evaluation intends to foster self-determination. The empowerment approach [50] involved community members–represented through the Citizen Scientist Advisory Council–engaging in co-production of the evaluation design and implementation by establishing key objectives for the evaluation, informing evaluation questions, building relevant and culturally responsive indicators, developing focus group guides, leading recruitment and data collection, and interpreting results [51]. In this way, the approach incorporated local community and Indigenous Knowledges as well as Western knowledge, in a similar approach to Two-Eyed Seeing [13–15]. Using these needs assessment evaluation results, the community will identify emerging needs and potential application issues, and work with the researchers to continue shaping project development and implementation.
Two-Eyed Seeing to embed digital platforms
Two-Eyed Seeing as described by Elder Albert Marshall [13, 14], refers to learning to see with the strengths of Indigenous and Western Knowledges. Our engagement and overall approach to working with the community of Île-à-la-Crosse takes a Two-Eyed Seeing lens, from co-conceptualization of solutions, which starts with understanding the needs of the community. All needs are a result of direct Indigenous Knowledge that was provided by the Advisory Council. Indigenous Knowledge is not limited to the knowledge of Elders and Traditional Knowledge Keepers; however, they play a critical role in guiding that knowledge through by providing historical, geographic, and cultural context. Moreover, the Knowledge Keepers can be key decision-makers in the community, and in our case, they were key informants who participated in this needs assessment. Every aspect of needs assessment was dependent on the Advisory Council and Key informants providing the Indigenous Knowledge that the research team needed to tailor digital solutions. As a result, Two-Eyed Seeing approach informed all aspects of the research process.
As we are working to develop, and bring digital platforms and technologies (i.e., Western methods) to address key community priorities, Indigenous Knowledge is central to the overall project. Indigenous Elders, decision-makers, and Advisory Council members are bringing both their historical and lived experience to inform project goals, key priority areas, target groups, and methods. Île-à-la-Crosse is a predominantly Metis community, which differs in culture from other Indigenous communities in Canada—First Nations and Inuit communities. Ceremony is not a key part of community functioning; thus, specific cultural ceremonies were not conducted upon advice of the Advisory Council. Instead, the knowledge of historical issues, challenges, and success stories in the community is considered Indigenous Knowledge for this needs assessment, and more importantly, this Indigenous Knowledge informed the focus areas and next steps for this project. Overall, the spirit of collaboration and co-creation which combined Western research methods/technology with Indigenous Knowledge and expertise is considered Two-Eyed Seeing in this project. This lens was taken at all phases, from the engagement stage to Advisory Council meetings, to planning and executing the needs assessment and next steps.
Data collection
In order to obtain an in-depth understanding of the key priorities and supports within the community of Île-à-la-Crosse, this needs assessment used a qualitative approach. An environmental scan was conducted in February 2020 of current school and community policies and programs. Published reports, meeting memos, community social media accounts, and the Île-à-la-Crosse website were reviewed for existing policies and programs. The Citizen Scientist Advisory Council identified appropriate data sources for the document review and corroborated which programs and initiatives were currently active in the community.
Qualitative data were collected from key decision-makers and other members within the community. A purposeful convenience sampling approach was employed to identify members of the community who could serve on the Council and participate in focus group discussions. Key decision makers and existing Council members recommended other community members who could join the focus group discussions to provide detailed and relevant information on community priorities, digital infrastructure, supports, and challenges. Two focus groups were conducted by members of the research team in Île-à-la-Crosse with the Council in May 2020. Focus group participants were asked to describe community priorities, supports, and barriers, as well as experience and comfort with digital platforms. Each focus group had four participants, were two-hours in length, and followed an unstructured approach. Three key informant interviews were conducted in Île-à-la-Crosse between February and April 2020. One-hour interviews were conducted one-on-one and followed a semi-structured interview format. The focus groups and key informant interviews were led by the study PI, TRK, and Co-Investigator, JB, who have extensive training and experience with qualitative research methods, particularly in partnership with Indigenous communities. Focus groups and key informant interviews were conducted virtually using Zoom [52]. The key informant interviews and focus groups were audio-recorded and transcribed. All data were aggregated, anonymized, and securely stored in a cloud server. Data are owned by the community. Both the Council and the research team have equal access to the data.
Data analysis
All documents identified through the environmental scan were reviewed for key themes. A list of existing school and community programs was compiled and organized by theme (i.e., education-focused, nutrition-focused, health-focused, etc.). Follow-up conversations with key informants verified the continued planning and provision of these programs.
Following the 6-step method by Braun and Clarke (2006), a thematic analysis was conducted to systematically identify key topic areas and patterns across discussions [53]. A shortlist of themes was created for the key informant interviews and focus groups, respectively. A manual open coding process was conducted by two reviewers who reached consensus on the final coding manual and themes. Separate analyses were conducted for key informant interviews and focus group discussions; however, findings were synthesized to identify key themes and sub-themes in key priorities for the community, community supports and barriers, as well as digital connectivity and infrastructure needs.
Needs assessment findings
The needs assessment guiding framework informed specific discussions of key issues in the community of Île-à-la-Crosse. Key informant interviews and focus group discussions commenced by asking about priorities–“what are the key areas of focus for the community?” In all conversations–including a document review of initiatives in Île-à-la-Crosse–health was highlighted as a current priority; hence, questions in the guiding framework were tailored to fit a needs assessment focused on community health. The following five overarching evaluation questions were used to guide the evaluation: i) What are the prominent health issues facing residents of Île-à-la-Crosse?; ii) What supports are currently available to help residents address prominent health issues in the community?; iii) What types of barriers do community members face to accessing services to manage their health?; iv) How is health-related information currently shared in the community?; and v) To what extent are health services and information currently managed digitally/electronically? The evaluation questions were kept broad to capture a range of perspectives. An evaluation matrix linking the proposed evaluation questions to their respective sub-questions, indicators, and data collection tools is outlined in Table 1.
Feedback on each needs assessment topic area is summarized in the sections below. Sample quotes supporting each of the key topic areas is provided in Table 2.
Key priorities
Four priorities were identified through the focus groups, key informant interviews, and document review (Fig 2). Given the timing of the discussion, the primary issue of concern was the COVID-19 pandemic. Many community members were worried about contracting the virus, and the risk it posed to Elders in the community. Of greater concern, however, was how COVID-19 exacerbated many existing health concerns including diabetes and hypertension in the community. For example, routine procedures were postponed and community members with other health conditions were not receiving routine healthcare during the height of the pandemic. The St. Joseph’s Hospital and Health Centre services Île-à-la-Crosse and bordering communities, hence maintaining capacity for COVID-19 patients was a priority. COVID-19 exposed existing barriers in the healthcare system which are described in greater detail in the barriers to community health section.
Another priority discussed by many community members was climate change and the environment. Community members noted that changes in wildlife patterns, land use, and early winter ice road thaw were areas of concern, particularly due to the impact these factors have on traditional food acquisition practices (i.e., hunting) and food access. For instance, the geographic location of Île-à-la-Crosse is surrounded by a lake, and the main highway which connects the community to the land has experienced increased flooding in the past few years.
In addition to posing immediate danger to community members, food security and sovereignty are also closely linked to road access. While the community produces some of its own food through the local fishery and greenhouses, Île-à-la-Crosse is still dependent on a food supply from the south (i.e., Saskatoon). During COVID-19, food access was further restricted due to limited transport and delivery of food products, which increased the risk of food insecurity for community members. Food insecurity was believed to be of bigger concern for Elders in the community compared to younger members. Younger community members expressed having the ability to source their own food in a variety of capacities (e.g., fishing in the lake), whereas Elders rely more heavily on community resources and support (e.g., grocery stores, friends, and family).
Community members also discussed issues surrounding mental health and wellbeing. This topic was of particular concern for youth and Elders in the community. Community members discussed the importance of identifying covert racism (vs. discrimination) that exists within health services that exacerbated mental health issues and care, as well as developing coping strategies, resilience, and supports to prevent mental health crises. Key informants emphasized the need to minimize the stigma around mental health and focus on holistic wellbeing as they work to develop strategies to improve community wellness.
Community health supports
Île-à-la-Crosse has been working on developing supports to improve community health through various initiatives. A document review identified a community-specific wellness model which has informed program development and planning over the past few years. The key components of the Île-à-la-Crosse wellness model are: i) healthy parenting; ii) healthy youth; iii) healthy communities; iv) Elders; v) healing towards wellness; and vi) food sovereignty. The Elders Lodge in the community provides support for holistic wellbeing by promoting intergenerational knowledge transmission, guidance to youth and community members, as well as land-based activities which improve bonding, cultural awareness, and mental and spiritual well-being among community members. The Elders Lodge hosts both drop-in and organized events.
Several initiatives have been developed to support food sovereignty in the community, including a greenhouse program where fruits and vegetables are grown and shared locally. This program is run in partnership with the school to increase food knowledge and skills among youth. In addition, after-school programs including traditional food education (i.e., cooking classes) and land-based activities (i.e., berry picking) led by Elders support the goals of the wellness model. The community is currently working on developing additional programs dedicated to improving mental wellness among adults, youth, and Elders.
Barriers to community health
When key informants were asked to identify barriers to community health, they described delays in access to timely health information. For example, daily COVID-19 tests conducted at the regional health centre in Île-à-la-Crosse were relayed to the provincial health authority; however, information about the total number of COVID-19 cases could take up to one week to be sent back to the community. This time lag restricted community decision-makers’ ability to enact timely policy (i.e., contact tracing) and rapidly respond to managing cases.
A second barrier that was raised by community members was a delay in access to timely healthcare. The Île-à-la-Crosse hospital is a regional health service centre serving the community as well as surrounding areas. Community members noted that the load often exceeded the capacity of the single hospital, and some patients and procedures were relocated to hospitals and clinics in the larger city of Saskatoon, Saskatchewan. This was reported to be challenging for many community members as it was associated with longer wait times, long commutes, and sometimes required time off work. Many of these challenges were exacerbated during the COVID-19 pandemic. As a result of the pandemic, many medical centres and hospitals postponed routine and elective medical procedures in an attempt to accommodate the overwhelming influx of patients who contracted COVID-19. In addition, community members were advised to avoid spending time in health centres to limit risk of exposure to the virus. These COVID-related changes further delayed access to timely healthcare for many community members of Île-à-la-Crosse.
Several community members reported experiencing institutional racism in healthcare and social service settings outside of Île-à-la-Crosse. This was particularly exacerbated during the COVID-19 movement restrictions, where community members faced significant difficulties in accessing services and care in larger urban centres, and experienced further discrimination due to the stigma of COVID-19-related rumours about communities in the north.
Lastly, community members discussed a lack of awareness about some health topics, including where and how to access reliable health information. Some community members attributed this lack of awareness to a general distrust in government health information due to a history of colonialism and exploitation in Canada, which likely contributed to increased misinformation about COVID-19 risk and spread.
Health communication
The primary modes of communication within Île-à-la-Crosse are radio and social media. These platforms were used throughout the pandemic to communicate health information about COVID-19 case counts and trends. Community members also reported obtaining health information from healthcare practitioners (i.e., for those already visiting a healthcare provider), Elders, and the internet. Key informants indicated an interest in improving digital infrastructure to enable sharing of timely and accurate health information with community members and minimize misinformation. Key informants also reported room for improvement in the community’s digital health infrastructure, particularly in improving timely communication with community members, and to inform decision-making in crisis situations.
Digital infrastructure and connectivity
Île-à-la-Crosse has its own cell tower which offers reliable access to cellular data. The community also has access to internet via the provincial internet provider–SaskTel, as well as a local internet provider—Île-à-la-Crosse Communications Society Inc. Key informants and community members confirmed that most individuals above 13 years of age have access to smartphones, and that these mobile devices are the primary mode of internet access. However, it was unclear whether everyone who owns smartphones also has consistent data plans or home internet connections. Key informants described the great potential of digital devices like smartphones to increase the speed and accuracy of information sharing. Discussions with both key informants and community members suggested the need for a community-specific app or platform which could provide timely health information that was tailored to the community’s needs.
Community members noted that expanding digital infrastructure had to be paired with efforts to improve digital literacy–particularly as it relates to data security, privacy, and online misinformation. A separate initiative was discussed which could work to improve digital literacy among youth and Elders, as this would improve both the uptake of digital health platforms, as well as their usefulness and application. Key informants discussed the importance of building digital infrastructure that would enable data sovereignty, self-governance, and determination. The key informants, who are also primary decision-makers in the community, described opportunities for ethical development of digital platforms that would ensure that data is owned by the community.
Discussion
Needs assessments are commonly the first step in understanding specific community needs, [27, 28]; however, few evaluation frameworks provide practical guidance on how to engage communities in needs assessments [41]. This paper provides a step-by-step guide for conducting needs assessments in collaboration with communities in the digital age. Using the series of questions outlined in the Guiding Framework, researchers and evaluators can gain an in-depth understanding of a community’s priorities, needs, existing capacity, and relevant solutions.
The Guiding Framework was critical to establishing a partnership with the community of Île-à-la-Crosse, as it enabled the research team to obtain detailed insight into their priorities–in this case, community health–as well as community capacity. Taking a Two-Eyed Seeing approach [15], conversations with the community highlighted strengths of Western digital technology and the diversity of Indigenous Knowledges for addressing priorities [13]. This approach was also important to establishing trust and respect for the variety of perspectives that could be used to address community priorities. The resulting partnership also enabled the conceptualization of tangible action items that were aligned with current and future priorities–a key factor in the sustainability and feasibility of community-based initiatives [4–8, 54].
Challenges and opportunities for using digital platforms for priorities identified by needs assessment
Many rural and remote communities face similar challenges and share common priorities with Île-à-la-Crosse. For example, resource and service access, including food and other essential supplies, healthcare, and internet connectivity are issues faced by many rural and remote communities across Canada [55–60]. Key informants and community members from our partner community corroborated these access issues, particularly in relation to public health. Given the potential for digital technology to bridge access gaps, it has become pertinent to invest in digital infrastructure and platform development.
Research has shown that in many rural and remote communities, smartphone ownership is not the limiting factor–it is internet inequity, which is defined as differential internet access based on wealth, location (urban, rural, or remote), gender, age, or ethnicity [61]. The United Nations has declared internet access a human right [10], which makes it imperative to develop digital infrastructure such as internet connectivity to improve digital accessibility. Île-à-la-Crosse has its own cell tower which offers reliable access to cellular data. The community of Île-à-la-Crosse also has access to consistent and dedicated internet service through a provincial internet provider and local internet provider. The needs assessment showed that the universality of smartphone ownership combined with good internet connectivity lays the foundation for the development of tailored, culturally appropriate digital health platforms in communities like Île-à-la-Crosse.
In particular, the needs assessment revealed that smartphone apps, which most citizens are well-versed with, can be used to provide local services and access to resources. For example, a locally developed app can connect the Mayor’s office with community members in real-time to provide updates on COVID-19 outbreaks. Apps also have the potential to connect communities to resources within and outside of the community [35, 57]. For example, advanced artificial intelligence algorithms can be used to anticipate community needs prior to urgent crises like COVID-19, environmental disasters, or food crises [35, 62–65]. To date, the issue has not been the lack of technology or ability to bridge this gap for rural and remote communities. Instead, larger systemic inequities have limited our ability to co-create local solutions for global problems by decentralizing technology that is widely available [35, 66], which highlights upstream inequities in developing digital platforms.
Recommendations for inclusive digital needs assessments
Given the widespread adoption of digital technology, digital platforms can provide rich data to identify and address community crises [2, 3, 35]. Importantly, co-created digital platforms can be used to share knowledge in real-time with community members and other stakeholders to enable remote engagement, which is especially important during crisis situations such as a pandemic [2, 3, 35]. As we implement creative digital platforms in varied programs or research projects, we must also integrate this digital perspective into the evaluation process. Research and evaluation literature has well established approaches to needs assessment evaluations [29, 42, 67]; however, in the 21st century, we need to account for the use and application of digital platforms in community-focused initiatives. To identify how and where digital platforms can play a role in addressing community priorities, we propose several recommendations for inclusive community-based needs assessments.
First, at the crux of all community-based needs assessments is relationships. A relationship built on respect, reciprocity, mutual understanding, and prioritizing the needs and vision of communities is essential for sustainable impact. The First Nations OCAP® principles [68] informed conversations between the research team and community about data ownership and control. These principles include ownership of knowledge and data, control over all aspects of research, access to information about one’s own community, and possession or control of data [68]. The OCAP® principles ensure First Nations and other Indigenous Peoples the right to their own information, and also reflect commitments to use and share information in a way that maximizes the benefit to a community, while minimizing harm. Some communities may choose to lead a project, or work closely in collaboration with experts for specific projects. Irrespective of the project dynamics, needs assessments rely on detailed information and context about a community for a project to succeed.
Second, it is important for researchers and evaluators to gain an understanding of the current digital infrastructure and connectivity in the community. The needs assessment framework (Fig 1) includes relevant questions for identifying data and WIFI access in a community, penetration of digital devices, and existing digital infrastructure. Even for community-based initiatives that are not focused on a digital platforms, digital technologies will inevitably be a part of the solution, a barrier, or both. Hence the digital landscape has become part of the context that we must capture and understand in a needs assessment to better design and develop programming, policies, and other initiatives.
Third, it is important to ask the question of where and how a digital tool or platform could help. Are there gaps that digital platforms can help address or fill? In rural and remote communities, in particular, digital platforms can provide access to real-time information and services not otherwise available. For example, Telehealth [69, 70] in the Canadian north offers citizens access to essential healthcare services, including video appointments with medical specialists. Prior to Telehealth, many residents would need to fly into bigger cities in the nearest province to access health care [55].
Lastly, an understanding of the broader context which affects a community’s ability to adopt digital platforms is critical to the success of digital initiatives. This includes, but is not limited to, capturing data on socioeconomic status and the accessibility of internet-connected digital devices. Digital platforms should help to bridge the divide in resource, service, and information access–not widen the gap. For some communities, this may require working on building digital infrastructure and obtaining dedicated funds to expand access prior to implementing digital initiatives. In addition, digital literacy cannot be taken for granted. Digital literacy refers to individuals’ ability to not only use digital devices, but according to Eshet-Alkalai [71], “includes a large variety of complex cognitive, motor, sociological, and emotional skills, which users need in order to function effectively in digital environments.” In its simplest form, digital literacy may include the ability to navigate digital platforms, download apps, and communicate electronically. Other more specific skills include ability to read and understand instructions, terms and services, as well as data privacy and security statements [72–74] As part of a needs assessment, identifying digital literacy within a community is an important step to safe, ethical, and relevant digital tool development.
Considering the challenges, immense potential, and learnings from applying the Guiding Framework, a tailored digital platform was conceptualized called Sakitawak Health.
Development of Sakitawak Health
Sakitawak Health is a culturally-responsive digital epidemiological platform to monitor, mitigate, and manage COVID-19 outbreaks. The needs assessment concluded that digital platforms can be used for emerging or other existing population health crises within Île-à-la-Crosse and potentially other Indigenous communities. Moreover, to co-create digital platforms, the Île-à-la-Crosse Citizen Scientist Advisory Council identified key features to embed in CO-Away, including free virtual care for citizens via a smartphone app at the frontend, and access to anonymized community data on the backend for decision-makers.
The app will provide three key precision medicine services that are specific to each citizen: 1) continuous risk assessment of COVID-19 infection; 2) evidence-based public health communication; and 3) citizen reporting of food availability, access to public services, and COVID-19 symptoms and test results. These culturally-responsive features have been co-created with Métis decision-makers in Île-à-la-Crosse based on imminent community needs and preferences. CO-Away will enable real-time data collection through continuous citizen engagement to inform municipal jurisdictional policies.
There are three guiding principles for developing Sakitawak Health: I) Citizen empowerment and data ownership: Active engagement is enabled through app features such as visualizing community risk. More importantly, the community owns the data to ensure data sovereignty; II) Privacy: Utilizing a cutting-edge methodology called federated machine learning, we will develop artificial intelligence algorithms that stores sensitive data such as participant location on mobile devices itself (i.e., sensitive data are not stored in external servers); III) Security and scalability: The backend server will be located in Cloud in Canada, which allows for horizontal and vertical scalability (i.e., the potential for developing multiple frontend apps and decision-making dashboards).
Recognizing the importance of data sovereignty and Indigenous self-governance
Data sovereignty and social justice are important aspects of community-based work, particularly for communities that have experienced discrimination or systemic inequities [2, 75]. Data sovereignty refers to meaningful control and ownership of one’s data [76]. For Indigenous communities in Canada, self-determination and self-governance are of paramount importance given the colonial history of oppression, trauma, and disenfranchisement [77], and data sovereignty and ownership of digital platforms can promote that independence. In conducting digital community-based needs assessments, the application of a Two-Eyed Seeing lens enables us to leverage strengths of both Indigenous and Western Ways of Knowing to help focus on key priorities and develop solutions.
The engagement and overall approach to working with the community of Île-à-la-Crosse applied a Two-Eyed Seeing lens. In the needs assessment with Île-à-la-Crosse, Two-Eyed Seeing involved incorporation of Métis Knowledge during team engagements, which ensured that any digital platforms developed would incorporate Indigenous Knowledge to promote data sovereignty. All priorities identified within this manuscript are a result of direct Indigenous Knowledge that was provided by the Council. Indigenous Knowledge is not limited to the knowledge of Elders and Traditional Knowledge Keepers; however, they play a critical role in guiding that knowledge through by providing historical, geographic, and cultural context. Discussions with Île-à-la-Crosse about data sovereignty centered around citizen ownership of data, community access, and ensuring data privacy and security. The ultimate goal of this approach to data sovereignty is to facilitate decreased dependence on external systems and use digital solutions for Indigenous self-determination and self-governance.
Next steps
The needs assessment represents the first phase of a larger evaluation strategy to develop and implement culturally appropriate digital platforms for community health. Phase 1 involved identifying core health priorities and desired supports in the community of Île-à-la-Crosse. Based on the needs assessment findings, Phase 2 of this project will involve the development of tailored digital health platforms and programming to support digital literacy. As part of Phase 2, digital literacy programs and tailored digital health platforms will be pilot tested and adapted prior to their implementation. In Phase 3, a process evaluation will be conducted to assess the reach, uptake, and use of digital health platforms and digital literacy programming. Integrated knowledge translation will be conducted during all phases to ensure continuous feedback, communication, and knowledge sharing with all relevant stakeholder groups.
Conclusions
Needs assessments can facilitate important conversations in community-based research and evaluation to learn about key priorities, challenges, and opportunities for growth. The Guiding Framework for Community-Based Needs Assessments to Embed Digital Platforms details a step-by-step approach to begin a conversation with communities to better understand their needs, and to tailor research and evaluation projects focused on embedding digital platforms. In Île-à-la-Crosse, the needs assessment framework has propelled the launch of a timely, community-engaged digital initiative to address key priorities, starting with COVID-19. Overall, tailored platforms can help bridge existing gaps in resource, program, and service access in Indigenous communities, irrespective of their location across the world.
Acknowledgments
The authors would like to acknowledge the contributions of community members of Île-à-la-Crosse. The Elders, youth, and key decision-makers who are part of the Île-à-la-Crosse Citizen Scientist Advisory Council have been invaluable in providing support, guidance, and cultural training to the research team. The authors also acknowledge the support of the Canadian Internet Registration Authority in advancing the uptake of digital health applications.
References
- 1. Wallerstein N, Duran B, Oetzel JG, Minkler M. Community-Based Participatory Research for Health: Advancing Social and Health Equity. 2017.
- 2.
Smith LT. Decolonizing Methodologies: Research and Indigenous Peoples. London; New York; Dunedin: Zed Books Ltd; University of Otago Press; Distributed in the USA exclusively by St Martin’s Press; 2021. https://doi.org/10.4324/9781315597843-10.
- 3. Datta R. Decolonizing both researcher and research and its effectiveness in Indigenous research. Res Ethics 2018;14:1–24. https://doi.org/10.1177/1747016117733296.
- 4. Carr-Hill R, Street AD. Economic Analysis of Cost-Effectiveness of Community Engagement to Improve Health 2008.
- 5. Mason AR, Carr Hill R, Myers LA, Street AD. Establishing the economics of engaging communities in health promotion: What is desirable, what is feasible? Crit Public Health 2008;18:285–97.
- 6. Popay J, Attree P, Hornby D, Milton B, Whitehead M, French B, et al. Community engagement to address the wider social determinants of health: A review of evidence on impact, experience & process 2007.
- 7.
Narayan D. Empowerment and Poverty Reduction: A Sourcebook. Washington, D.C.; The World Bank; 2002.
- 8.
Cairncross L, Morrell C, Drake J, Brownhill S. Tenants Managing: An Evaluation of Tenant Management Organisations in England. London: Office of the Deputy Prime Minister; 2002.
- 9.
Pratchett L, Durose C, Lowndes V, Smith G, Stoker G, Wales C. Empowering communities to influence local decision making: systematic review of the evidence. London: 2009.
- 10.
La Rue F. Report of the Special Rapporteur on the Promotion and Protection of the Right to Freedom of Opinion and Expression, Frank La Rue: addendum. 2011.
- 11. Cyril S, Smith BJ, Possamai-Inesedy A, Renzaho AMN. Exploring the role of community engagement in improving the health of disadvantaged populations: A systematic review. Glob Health Action 2015;8. pmid:26689460
- 12. Wehipeihana N, Davidson EJ, Mckegg K. What Does it Take to do Evaluation in Communities and Cultural Contexts Other Than Our Own? J Multidiscip Eval 2010;6:182–92.
- 13. Bartlett C, Marshall M, Marshall A. Two-Eyed Seeing and other lessons learned within a co-learning journey of bringing together indigenous and mainstream knowledges and ways of knowing. J Environ Stud Sci 2012;2:331–40. https://doi.org/10.1007/s13412-012-0086-8.
- 14. Iwama M, Marshall M, Marshall A, Bartlett C. Two-Eyed Seeing and the Language of Healing in Community-Based Research. Can J Nat Educ 2009;32:3–23.
- 15. Martin DH. Two-eyed seeing: a framework for Indigenous approaches to Indigenous health research. Can J Nurs Res 2012;44:20–42.
- 16. Sylvestre P, Castleden H, Martin D, McNally M. “Thank you very much… You can leave our community now.”: Geographies of responsibility, relational ethics, acts of refusal, and the conflicting requirements of academic localities in indigenous research. Acme 2018;17:750–79.
- 17. Frohlich KL, Ross N, Richmond C. Health disparities in Canada today: Some evidence and a theoretical framework. Health Policy (New York) 2006;79:132–43. pmid:16519957
- 18. Public Health Agency of Canada. Key Health Inequalities in Canada: A National Portrait. 2018.
- 19.
Statistics Canada. Île-à-la-Crosse, NV [Census subdivision], Saskatchewan and Saskatchewan [Province] (table). Ottawa: 2017. https://doi.org/Statistics Canada Catalogue no. 98-316-X2016001.
- 20.
Waldram JB, Herring A, Young TK. Aboriginal Health in Canada: Historical, Cultural, and Epidemiological Perspectives. 2nd editio. Toronto: University of Toronto Press; n.d.
- 21. Marrone S. Understanding barriers to health care: a review of disparities in health care services among indigenous populations. Int J Circumpolar Health 2007;66:188–98. pmid:17655060
- 22.
Central Urban Métis Federation Inc., Kinistin Saulteaux Nation, Saskatoon Health Region. Strengthening the Circle: Partnering for Improved Health for Aboriginal People. Saskatoon, SK: 2010.
- 23. King M, Smith A, Gracey M. Indigenous health part 2: the underlying causes of the health gap. Lancet 2009;374:76–85. pmid:19577696
- 24. Gracey M, King M. Indigenous health part 1: determinants and disease patterns. Lancet 2009;374:65–75. pmid:19577695
- 25. Marsh TN, Coholic D, Cote-Meek S, Najavits LM. Honouring the Truth, Reconciling for the Future Honouring the Truth, Reconciling for the Future Summary of the. vol. 12. 2015.
- 26. Levac L, Parizeau K, Varghese J, Morton M, Jackson E, Hawkins L. Towards a framework for building community-university resilience research agendas. Soc Sci 2018;7. https://doi.org/10.3390/socsci7120260.
- 27. Smith LU. Community engagement framework for community assessment and improvement planning. J Public Heal Manag Pract 2017;23:S22–8. pmid:28542060
- 28.
Pavlish CP, Pharris MD. Community-based collaborative action research : a nursing approach. Sudbury, MA: Jones & Bartlett Learning; 2012.
- 29. Watkins R, Meiers MW, Visser YL. Section 1 –Needs Assessment:Frequently Asked Questions. 2012. https://doi.org/10.1596/9780821388686_ch02.
- 30. Skinner K, Burnett K, Pratley E, Parker B, Dovick B. Case Study 2: Engaging Youth in a Needs Assessment for Programming and Evaluation, 2021, p. 97–100. https://doi.org/10.1007/978-3-030-34390-3_13.
- 31. Pelletier CA, Pousette A, Ward K, Fox G. Exploring the perspectives of community members as research partners in rural and remote areas. Res Involv Engagem 2020;6:1–10. https://doi.org/10.1186/s40900-020-0179-6.
- 32. O’Dea S. Number of smartphone users from 2016 to 2021 2021. https://www.statista.com/statistics/330695/number-of-smartphone-users-worldwide/.
- 33. Katapally TR. Global Digital Citizen Science Policy for Pandemics. J Med Internet Res 2020;22:e19357. https://doi.org/10.2196/preprints.19357.
- 34. Kenney M, Zysman J. The next phase in the digital revolution. Commun ACM 2018;61:54–63.
- 35. Katapally TR. A global digital citizen science policy to tackle pandemics like COVID-19. J Med Internet Res 2020;22:1–7. pmid:32408267
- 36. Whitelaw S, Mamas MA, Topol E, Van Spall HGC. Applications of digital technology in COVID-19 pandemic planning and response. Lancet Digit Heal 2020;2:e435–40. pmid:32835201
- 37. Gearheard S, Aipellee G, O’Keefe K. The Igliniit Project: Combining Inuit Knowledge and Geomatics Engineering to Develop a New Observation Tool for Hunters. SIKU Knowing Our Ice Doc. Inuit Sea Ice Knowl. Use, 2010, p. 181–202. https://doi.org/10.1007/978-90-481-8587-0.
- 38. Bennett TD, Lantz TC. Participatory photomapping: a method for documenting, contextualizing, and sharing indigenous observations of environmental conditions. Polar Geogr 2014;37:28–47. https://doi.org/10.1080/1088937X.2013.873089.
- 39. O’Donnell S, Beaton B, McMahon R, Hudson HE, Williams D, Whiteduck T. Digital technology adoption in remote and Northern Indigenous communities in Canada. Can Sociol Assoc 2016 Annu Conf 2016:1–44.
- 40. Katapally T, Kannan P, Kwabia E. Citizen Science and COVID : Identifying the Needs of Rural and Remote Education COVID-19 SERIES : FROM CRISIS TO RECOVERY. 2021.
- 41. Centers for Disease Control and Prevention (CDC). Community Needs Assessment: Participant Workbook. Particip Workb 2013:1–77.
- 42.
Stufflebeam DL, Shinkfield AJ. Evaluation theory, models, and applications. 1st ed. San Francisco, CA: Jossey-Bass; 2007.
- 43.
McLennan D. Sakitawak Development’s Facebook Page n.d.
- 44.
Cameco Corp. Patuanak—English River First Nation—Community Profiles—Community. Cameco North Saskatchewan 2020.
- 45.
McLennan D. The Encyclopedia of Saskatchewan: Île-à-la-Crosse n.d. https://esask.uregina.ca/entry/ile-a-la-crosse.jsp.
- 46. Favel D, Roy G. Northern Village of Ile a la Crosse. 2013.
- 47. Health Quality Ontario. Needs Assessment Resource Guide for Family Health Teams. J Contin Educ Nurs 2009;37:148–9. https://doi.org/10.3928/00220124-20060701-02.
- 48. Askew K, Beverly MG, Jay ML. Aligning collaborative and culturally responsive evaluation approaches. Eval Program Plann 2012;35:552–7. pmid:22284941
- 49. Chouinard JA, Cram F. Culturally Responsive Approaches to Evaluation: Empirical Implications for Theory and Practice 2020.
- 50. Fetterman DM. Empowerment Evaluation. Int Encycl Soc Behav Sci Second Ed 2015:577–83. https://doi.org/10.1016/B978-0-08-097086-8.10572-0.
- 51. Voegeli C. Evaluation Advisory Groups: Considerations for Design and Management. Can J Progr Eval 2021. https://doi.org/10.3138/cjpe.69949.
- 52.
Zoom Video Communications Inc. Security guide 2016.
- 53. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3:77–101. https://doi.org/10.1191/1478088706qp063oa.
- 54. Pratchett L, Lowndes V, Smith G, Stoker G, Wales C, Durose C. Empowering communities to influence local decision making. 2009.
- 55. Oosterveer TM, Young TK. Primary health care accessibility challenges in remote indigenous communities in Canada’s North. Int J Circumpolar Health 2015;74:1–7. pmid:26507717
- 56. Skinner K, Burnett K, Williams P, Martin D, Stothart C, LeBlanc J, et al. Challenges in assessing food environments in northern and remote communities in Canada. Can J Public Heal 2016;107:eS60–3. pmid:27281518
- 57. Goodridge D, Marciniuk D. Rural and remote care. Chron Respir Dis 2016;13:192–203. https://doi.org/10.1177/1479972316633414.
- 58. Forbes D, Edge D. Canadian home care policy and practice in rural and remote settings: Challenges and solutions. J Agromedicine 2009;14:119–24. pmid:19437267
- 59. McNally MB, Rathi D, Joseph K, Evaniew J, Adkisson A. Ongoing policy, regulatory, and competitive challenges facing Canada’s small internet service providers. J Inf Policy 2018;8:167–98. https://doi.org/10.5325/jinfopoli.8.2018.0167.
- 60. Haight M, Quan-Haase A, Corbett BA. Revisiting the digital divide in Canada: The impact of demographic factors on access to the internet, level of online activity, and social networking site usage. Inf Commun Soc 2014;17:503–19. https://doi.org/10.1080/1369118X.2014.891633.
- 61. Katapally TR. The SMART framework: Integration of citizen science, community-based participatory research, and systems science for population health science in the digital age. JMIR MHealth UHealth 2019;7:1–12. pmid:31471963
- 62. Gray DM, Joseph JJ, Olayiwola JN. Strategies for Digital Care of Vulnerable Patients in a COVID-19 World—Keeping in Touch. JAMA Heal Forum 2020;1:e200734. https://doi.org/10.1001/jamahealthforum.2020.0734.
- 63. Meechang K, Leelawat N, Tang J, Kodaka A, Chintanapakdee C. The acceptance of using information technology for disaster risk management: A systematic review. Eng J 2020;24:111–32. https://doi.org/10.4186/ej.2020.24.4.111.
- 64. Castro D, New J. The Promise of Artificial Intelligence. Tex Med 2016;115:32–5. https://doi.org/10.7551/mitpress/12385.001.0001.
- 65. Zhou L, Zhang C, Liu F, Qiu Z, He Y. Application of Deep Learning in Food: A Review. Compr Rev Food Sci Food Saf 2019;18:1793–811. pmid:33336958
- 66. Katapally TR. Are democratic nations ready to democratize technology?: Tarun Katapally for Inside Policy. Macdonald-Laurier Inst 2020. https://macdonaldlaurier.ca/democratic-nations-democratize-technology/.
- 67. Saunders RP, Evans MH, Joshi P. Developing a Process-Evaluation Plan for Assessing Health Promotion Program Implementation: A How-To Guide. Health Promot Pract 2005;6:134–47. pmid:15855283
- 68.
The First Nations Information Governance Centre. The First Nations Principles of OCAPTM Training Course 2021:1–6. https://fnigc.ca/ocap-training.
- 69.
Telehealth 2021. North Heal 2021. https://www.northernhealth.ca/services/digital-health/telehealth.
- 70. Sevean P, Dampier S, Spadoni M, Strickland S, Pilatzke S. Patients and families experiences with video telehealth in rural/remote communities in northern Canada. J Clin Nurs 2009;18:2573–9. pmid:19694885
- 71. Eshet-Alkalai Y. Digital Literacy: A Conceptual Framework for Survival Skills in the Digital era. J Educ Multimed Hypermedia 2004;13:93–106.
- 72. Tomczyk Ł. Skills in the area of digital safety as a key component of digital literacy among teachers. Educ Inf Technol 2020;25:471–86. https://doi.org/10.1007/s10639-019-09980-6.
- 73. Oh SS, Kim KA, Kim M, Oh J, Chu SH, Choi JY. Measurement of digital literacy among older adults: Systematic review. J Med Internet Res 2021;23:1–15. https://doi.org/10.2196/26145.
- 74. Nouri SS, Avila-Garcia P, Cemballi AG, Sarkar U, Aguilera A, Lyles CR. Assessing mobile phone digital literacy and engagement in user-centered design in a diverse, safety-net population: Mixed methods study. JMIR MHealth UHealth 2019;7:1–11. pmid:31469083
- 75.
Kukutai T, Taylor J, editors. Indigenous Data Sovereignty: Toward an Agenda. The Australian National University Press; 2016. https://doi.org/10.1109/IPDPSW.2014.214.
- 76. Hummel P, Braun M, Tretter M, Dabrock P. Data sovereignty: A review. Big Data Soc 2021;8. https://doi.org/10.1177/2053951720982012.
- 77. Allan B, Smylie J. First Peoples, Second Class Treatment. First Peoples, Second Cl Treat Role Racism Heal Well-Being Indig Peoples Canada 2015:1–65.