Skip to main content
Advertisement
  • Loading metrics

Global assessment of existing HIV and key population stigma indicators: A data mapping exercise to inform country-level stigma measurement

Abstract

Background

Stigma is an established barrier to the provision and uptake of HIV prevention, diagnostic, and treatment services. Despite consensus on the importance of addressing stigma, there are currently no country-level summary measures to characterize stigma and track progress in reducing stigma around the globe. This data mapping exercise aimed to assess the potential for existing data to be used to summarize and track stigma, including discrimination, related to HIV status, or key population membership at the country level.

Methods and findings

This study assessed existing indicators of stigma related to living with HIV or belonging to 1 of 4 key populations including gay men and other men who have sex with men, sex workers, people who use drugs, and transgender persons. UNAIDS Strategic Information Department led an initial drafting of possible domains, subdomains, and indicators, and a 3-week e-consultation was held to provide feedback. From the e-consultation, 44 indicators were proposed for HIV stigma; 14 for sexual minority stigma (including sexual behavior or orientation) related to men who have sex with men; 12 for sex work stigma; 10 for drug use stigma; and 17 for gender identity stigma related to transgender persons. We conducted a global data mapping exercise to identify and describe the availability and quality of stigma data across countries with the following sources: UNAIDS National Commitments and Policies Instrument (NCPI) database; Multiple Indicator Cluster Surveys (MICS); Demographic and Health Surveys (DHS); People Living with HIV Stigma Index surveys; HIV Key Populations Data Repository; Integrated Biological and Behavioral Surveys (IBBS); and network databases. Data extraction was conducted between August and November 2020. Indicators were evaluated based on the following: if an existing data source could be identified; the number of countries for which data were available for the indicator at present and in the future; variation in the indicator across countries; and considerations of data quality or accuracy. This mapping exercise resulted in the identification of 24 HIV stigma indicators and 10 key population indicators as having potential to be used at present in the creation of valid summary measures of stigma at the country level. These indicators may allow assessment of legal, societal, and behavioral manifestations of stigma across population groups and settings. Study limitations include potential selection bias due to available data sources to the research team and other biases due to the exploratory nature of this data mapping process.

Conclusions

Based on the current state of data available, several indicators have the potential to characterize the level and nature of stigma affecting people living with HIV and key populations across countries and across time. This exercise revealed challenges for an empirical process reliant on existing data to determine how to weight and best combine indicators into indices. However, results for this study can be combined with participatory processes to inform summary measure development and set data collection priorities going forward.

Author summary

Why was this study done?

  • Many people living with HIV and key populations, including gay men and other men who have sex with men, sex workers, people who use drugs, and transgender persons, are affected by stigma.
  • Stigma is a barrier to HIV prevention, diagnostic, and treatment services and also negatively impacts the health and quality of life of those affected.
  • Although reducing stigma is an important pillar of the HIV response, there are currently no established summary measures to characterize the overall level of stigma within a country and track progress in addressing stigma over time.
  • The main purpose of this study was to review existing stigma data globally and assess which stigma indicators have the potential to inform a summary measure of stigma at the country level.

What did the researchers do and find?

  • In this global mapping exercise, data for stigma indicators detailed in the following sources were reviewed and assessed: UNAIDS National Commitments and Policies Instrument (NCPI) database; Multiple Indicator Cluster Surveys (MICS); Demographic and Health Surveys (DHS); People Living with HIV Stigma Index surveys; HIV Key Populations Data Repository; Integrated Biological and Behavioral Surveys (IBBS); and other databases.
  • Through this exercise, 24 HIV stigma indicators and 10 key population indicators were determined to have potential for use in the creation of valid summary measures of stigma at the country level.

What do these findings mean?

  • The results of this study highlight challenges in creating summary measures for stigma experienced by key populations or related to HIV based on existing global data sources.
  • However, there is an opportunity to use participatory approaches alongside the existing data to create summary measures to describe and track stigma over time.

Introduction

Despite significant advancements in HIV prevention, early detection, and treatment, 37.7 million people are living with HIV around the world, and there were an estimated 1.5 million new infections in 2020 [1]. The UNAIDS global targets for 2025 focus on primary prevention of HIV as well as ensuring that 95% of people living with HIV become aware of their status; 95% of people diagnosed with HIV receive Antiretroviral Therapy (ART); and 95% of people living with HIV on ART achieve sustained viral suppression [2]. Collectively, the goal is to end new HIV infections by 2030. However, the stated goal for 2020 was to reduce new infections to 500,000, which was not achieved due in part to limited progress in reducing stigma affecting people at risk for and living with HIV [2]. Stigma has been identified as a social determinant of health and a key barrier to improving health outcomes among people living with HIV [38]. As such, stigma continues to present barriers to achieving the HIV prevention and treatment targets by interfering with the provision and uptake of prevention, diagnostic, and treatment services. This is particularly true for key populations (sex workers; gay, bisexual, and other men who have sex with men; people who use drugs; and transgender persons) who may experience stigma relating to actual or assumed HIV status in addition to experiencing intersecting stigma related to their actual or assumed behaviors or identities [2]. Stigma’s negative impact on the health and quality of life of people living with HIV and key populations is also well documented [3,914].

Stigma is a social process in which an individual or group is linked to a negative stereotype or misconception, often resulting in adverse experiences, loss of social status, and limited opportunities [15,16]. Stigma may occur at the individual, interpersonal, community, and structural levels and can be experienced as anticipated, perceived, internalized, or enacted [17]. Anticipated stigmas refer to the expectation of bias perpetrated by others [1720]. Perceived stigmas refer to felt stigma and the perception of bias as understood by a person living with a stigmatized identity [21]. Internalized, or self-stigma, is the adoption of negative feelings or devaluing of oneself on account of a stigmatized identity [17,22]. Enacted, or experienced stigma, is the perpetration of mistreatment or discriminatory acts by someone on the basis of a stigmatized identity [23]. Discrimination, part of the social process of stigma [15], includes any distinction, exclusion, or restriction made to human rights and fundamental freedoms, in the political, economic, social, cultural, civil, or any other field [24]. Discrimination can be institutionalized through existing laws, policies, or practices that negatively impact people living with HIV or key populations. Although the conceptualization of discrimination in relation to stigma varies, here, we consider enacted stigma to be inclusive of discrimination [17]. Drivers of HIV and key population stigmas can include individual-level factors such as lack of awareness or education (i.e., misinformation about HIV risk and transmission); however, societal level policies, cultural norms, and religious values can also act as drivers or facilitators of stigma [23,2527].

In recognition of the central role stigma has in impeding HIV epidemic transition, UNAIDS has established a vision to achieve 3 zeros by 2030: zero new HIV infections, zero AIDS-related deaths, and zero discrimination [2,9]. To support this vision, UNAIDS has also established the 10–10–10 goals that focus on removing societal and legal barriers to HIV services, including reductions in punitive laws and policies, experiences of stigma and discrimination, and experiences of gender inequality and violence by 2025 [28]. Stigma related to HIV and key population statuses have also been recognized as a key factor impeding progress toward the 2030 Agenda for Sustainable Development Goal (SDG 3.3) of achieving the end of the HIV pandemic [24,29]. Despite this consensus on the importance of addressing stigma, measurement of stigma experienced by people living with HIV and key populations continues to be a challenge for public health practitioners and policymakers [18,30]. Stigma measurement has unique challenges given that it is a latent construct, a social process that acts across multiple levels, and a phenomenon that has often been thought of as too complex to measure [3133]. However, progress has been made over the last decade in developing standardized measures to quantify both HIV stigma and key population stigma [18,3438]. Specifically, advancements have been made through the Global AIDS Monitoring indicators and WHO Strategic Information guidelines [39,40]. Despite the existence of many validated measures of different forms of stigma related to HIV and key population membership, there is currently no established methodology for bringing these together into more concise but validated summary measures at the country level. A summary measure to quantify progress toward achieving zero discrimination may allow for easy comparisons within and across countries and possibly improved accountability for progress.

The ability to characterize stigma across countries was recommended as a strategy to achieve the end of the HIV pandemic during the 2017 UNAIDS Science Panel meeting on ending the AIDS epidemic by 2030 [41]. In response, the UNAIDS Monitoring Technical Advisory Group (MTAG) created a task team in 2018 comprised of technical experts on HIV and key population stigma from civil society and academia from around the globe. This task team’s mission was to provide a set of consensus recommendations to UNAIDS on the establishment of a country-level summary measure of stigma faced by people living with HIV and key populations at high risk of HIV and the legal and policy environment for the protection of these individuals’ fundamental rights, including their ability to access health and HIV services.

The creation of country-level indices that characterize the degree of stigma existing within a country may facilitate understanding and interpreting the overall burden of stigma and its effects. For the purposes of this work, we consider an indicator to be a single item measuring an aspect of stigma, and an index is a composite of indicators to generate a summary or average value representing a broader stigma construct. Summary index measures may also be used by global- and national-level policymakers, advocates, and program implementers engaged in the AIDS response to track progress on eliminating HIV and key population stigma, potentially increasing the likelihood that this goal can and will be achieved. The question remains, however, whether sufficient data exist to inform country-level indices that characterize levels of stigma for people living with HIV and key populations. This study aims to identify and describe the availability and quality of stigma data across countries through a global data mapping exercise.

Methods

The methods used in this exercise, including preparatory activities for data mapping and the mapping of available data for country-level summary measures of HIV and key population stigma, are outlined in Fig 1. The protocol for this exercise is included as S1 Protocol.

thumbnail
Fig 1. Flowchart of methodological approach for data preparation and data mapping.

DHS, Demographic and Health Surveys; IBBS, Integrated Biological and Behavioral Surveys; ILGA, International Lesbian, Gay, Bisexual, Trans and Intersex Association; MICS, Multiple Indicator Cluster Surveys.

https://doi.org/10.1371/journal.pmed.1003914.g001

Task team and consultation process in preparation of data mapping

UNAIDS Strategic Information Department led an initial drafting of possible domains, subdomains, and indicators to be considered for inclusion in summary measures of stigma affecting people living with HIV and 4 key populations: men who have sex with men, sex workers, people who use drugs, and transgender people. In drafting indicators, established definitions, such as those created as a part of the Global AIDS Monitoring process [39], were used when available. The long-term goal of this process is to generate summary measures using existing indicators as developing new indicators would take significant time to finalized and generate data. Therefore, this initial consultation process only considered existing indicators that had been defined and validated or existing validated data collection tools.

The designated task team provided feedback on UNAIDS’ initial selection and helped guide a 3-week e-consultation that took place in August to September 2019. The consultation information was shared through UNAIDS mailing lists, social media, and partners. Interested participants were invited to register on the consultation page. Overall, 804 people registered, of whom 171 participated in online discussions and 160 in a short survey on their background. Individuals participated from different global regions, including individuals from HIV or key population communities, nongovernmental organizations, civil society or community-based or faith-based organizations, academia, government ministries, and the private sector or professional organizations. Descriptive characteristics of participants in e-consultation online survey are outlined in S1 Table. During the e-consultation, participants were invited to engage in online discussions on key questions structured around the draft domains, subdomains, and indicators, as well as how to bring these together.

Mapping available data for country-level summary measures of HIV and key population stigma

This data mapping exercise was an exploratory process. The criteria used to identify, include, and assess indicators were established a priori. However, final decisions were informed by the data obtained as part of a larger plan to understand stigma metrics.

Indicator identification and extraction.

As described, UNAIDS considered existing tracking systems and data sources in the initial identification and selection of indicators. However, a need to identify the level of data availability and quality for each proposed indicator remained in order to determine the feasibility of inclusion in measure development. In collaboration with UNAIDS, the research team conducted a data mapping exercise to determine the number of countries with sufficient and appropriate data that could be used to calculate each indicator over the period of 2000 to 2020. As a first step, the research team reviewed relevant datasets including the UNAIDS National Commitments and Policies Instrument (NCPI) database [42]; Multiple Indicator Cluster Surveys (MICS) [43]; Demographic and Health Surveys (DHS) [44]; People Living with HIV Stigma Index 1.0 surveys [45]; HIV Key Population Data Repository; a selection of Integrated Biological and Behavioral Surveys (IBBS) available to the research team [4,37]; and network databases. This entailed downloading relevant NCPI data from the UNAIDS databases and MICS data from the UNICEF webpage. DHS surveys were pulled from STATcompiler, and individual country DHS reports were reviewed as relevant. The research team worked directly with the Global Network of People Living with HIV (GNP+) to access and combine individual files of 34 country datasets of the People Living with HIV Stigma Index 1.0. GNP+ is currently leading a People Living with HIV Stigma Index 2.0, and, therefore, the survey for this study was also reviewed as were plans for its implementation. The IBBS studies considered in this exercise were implemented by the Key Populations Program at Johns Hopkins School of Public Health and therefore the research team had access to these data. IBBS data were obtained from individual studies conducted among men who have sex with men or sex workers in sub-Saharan Africa where possible based on the team’s preexisting access as well as review of files that summarized existing literature provided by UNAIDS. The HIV Key Populations Repository, which is a database developed from a comprehensive review of all available data for key populations, including burden and risk of HIV, prevalence, incidence, prevention indicators and treatment cascades, population size estimates, experienced violence, and engagement with healthcare systems was reviewed [46]. Additionally, databases from networks in including the International Lesbian, Gay, Bisexual, Trans and Intersex Association (ILGA), HIV Justice Network, Advancing HIV Justice were reviewed [47]. Other databases were reviewed to assess data availability for specific indicators collected outside of health-focused surveys as necessary. Data extraction was conducted between August and November 2020.

Indicator selection.

After data were extracted from these sources, further cleaning was conducted. This included reverse coding to ensure consistent direction of endorsement where necessary and the identification of outliers. With the extracted data, data aggregated to the country level was combined to create a country-level database. Descriptive statistics were calculated for all indicators with existing data, including, number, proportion, or percentage, as well as mean and SD when appropriate. Importantly, not all estimates in this dataset are nationally representative, given that data points are subject to individual study design and recruitment strategies.

Indicators were then selected based on a set of a priori criteria, outlined below, for their potential for creating a summary measure, also known as an index. Indicators for which no existing data source was identified for any country were removed from consideration. Indicators for which data were identified were further assessed based on (1) number of countries for which data were available for the indicator; (2) variation present across countries in the indicator; and (3) data quality or accuracy. Indicators for which data were available from less than 25 countries were removed. We examined variable distributions to both determine if any data transformations were necessary but also to assess whether sufficient variation in values existed to inform a composite index. Variation in the indicator was determined based on the proportion of affirmed responses. Indicators with less than 10% endorsement were considered to have low variation and therefore limited utility in the creation of an index. Indicators with low variation were removed from further consideration for use at present in empirical index creation processes. Data quality and accuracy of indicators were assessed to determine if there were fundamental concerns in the response options, data collection methods, or national-level representation.

Last, a correlation matrix of all indicators was generated to assess potential redundancy. While we expected correlation between indicators, a very high degree of correlation between 2 indicators was taken to be suggestive of redundancy that should be eliminated to create the most parsimonious and informative measure [48]. A pairwise correlation matrix was generated between all stigma indicators related to HIV or key population status, and a Pearson correlation coefficient (rho) of greater than 0.6 was used as indication of a potential redundancy, i.e., where limited additional information would be added to an index given the inclusion of other indicators. Indicators determined to be largely redundant based on high intercorrelation with other indicators were considered for removal or to be combined into a single measure.

Indicator assessment.

Indicators that met the criteria for inclusion were further assessed for potential use in the creation of indices at present based on (1) proportion of countries for which data were available; (2) level of missingness of data within countries; and (3) potential for calculation over time. The proportion of countries for which data were available for the indicator was determined to be “sufficient” if it was available in 50% or more of the 193 UN member states. Indicators that were available in less than 50% of countries were designated as “limited.” The cutoff of 50% was meant to demonstrate data availability for the majority of UN member states (countries). Within countries, those which had data available for greater than 40% of the identified indicators, these countries were labeled as having “sufficient” available data; where countries had data available for less than 40% of the indicators, they were labeled as having “limited” available data. An indicator was also assessed for potential use in assessing changes over time. Specifically, if an indicator had more than 1 time point of available data and had not fundamentally changed in its definition over time, it was considered as “potentially” useful for tracking changes over time. If there were changes in the indicator over time, it was determined to have “limited” utility, and, if data for the indicator was available for only 1 time point with no planned future data collection, it was categorized as “cannot” track changes over time.

Ethical approval

The Johns Hopkins School of Public Health Institutional Review Board (JHSPH-IRB) reviewed this data mapping exercise and secondary data analysis protocol and determined it to be nonhuman subjects research as only deidentified data were being used.

Results

The summary results of indicator identification, extraction, selection, and assessment for HIV stigma indicators and the key population stigma indicators are presented in Table 1.

thumbnail
Table 1. Summary of stigma indicator mapping identification, extraction, selection, and assessment.

https://doi.org/10.1371/journal.pmed.1003914.t001

E-consultation

The e-consultation process resulted in a revised list of indicators that covered 6 domains: (1) structural stigmas; (2) social norms and attitudes reported among the general community; (3) anticipated stigma; (4) experienced stigma; (5) internalized stigma, and (6) experiences of violence as reported by people living with HIV or key populations. The full list if proposed indicators are described in S2S6 Tables. The structural stigma domain was originally proposed by UNAIDS as a laws and policies domain. As a result of data identified in this mapping process, we expanded this domain to include access to justice and renamed the domain accordingly. Additionally, anticipated and experienced stigmas were originally included in a single domain, however, were separated into 2 separate domains to represent these distinct types of stigmas. Five summary stigma measures were proposed that would be able to describe (1) stigma affecting people living with HIV; (2) sexual minority stigma (including sexual behavior or orientation) related to men who have sex with men; (3) sex work stigma; (4) drug use stigma; and (5) gender identity stigma related to transgender persons. Across the 5 summary measure categories, the subdomains and the exact indicators included varied. In total, 44 indicators were proposed for HIV stigma, and for key populations, 14 were proposed for men who have sex with men; 12 for sex work; 10 for drug use; and 17 for transgender persons.

Data availability and quality

We were able to identify a data source and existing data for 39 of the 44 HIV stigma indicators selected in the e-consultation process (Table 1). Data for 4 of these indicators were available from the People Living with HIV Stigma Index 1.0 survey; however, the corresponding questions were slated for exclusion from the 2.0 version of the survey, and, thus, these indicators were excluded from further consideration for use in the summary measure. This left a total of 35 indicators for HIV stigma in consideration, of which another 10 were removed: 3 due to the limited number of countries for which data were available; 3 due to limited variability; and 4 due to inconsistency in data collection and/or fundamental concerns about data quality or access.

In terms of key populations, data sources with existing available data were only found for 7 of 14 indicators for sexual minority stigma related to men who have sex with men, 7 of 12 indicators for sex work stigma, 3 of 10 indicators for drug use stigma, and 6 of 17 indicators for gender identity stigma related transgender persons (Table 1). We recommended changes to the time frame of 2 indicators to be consistent with available potential data sources. Outside of HIV stigma, no data were identified for indicators within the domains of social norms and attitudes, violence, and internalized stigma. Additionally, no indicators within the domain of stigma and discrimination had sufficient data across countries or planned data collection for measurement over time for key populations. Therefore, these domains could not be considered for stigmas related to key populations after completing data mapping at this time. Across key populations, we were only able to retain 3 of 7 men who have sex with men indicators, 3 of 7 sex work stigma indicators, 2 of 3 drug use stigma indicators, and 5 of 6 gender identity stigma indicators due to limited data availability across countries.

Interrelationship of indicators

Several sets of indicators were combined due to how the data were reported. Among indicators for sexual minority stigma related to men who have sex with men, the indicators “Existence of constitutional protections of discrimination or other nondiscrimination provisions related to sexual orientation” and “Existence of laws or other provisions that prohibit discrimination in employment based on sexual orientation” were assessed via items that shared a “No” response option and are therefore collapsed into a single measure. Among indicators for sex work stigma, “Existence of constitutional protections of discrimination based on occupation or other nondiscrimination provisions specifying sex work” and “Existence of laws or policies recognizing sex work as work” are collapsed into a single measure as these items were also collected with a shared “No” response option. Similarly, 2 of the 5 indicators for gender identity stigma with data available from more than just a few countries had to be combined due to how the data were reported through the NCPI (i.e., the questionnaire was structured so that indication of presence or absence of laws or policies assessed in these indicators shared a “None of these policies” category). These indicators were collapsed into a single measure.

For HIV stigma, the indicator “Percentage of people living with HIV who have lost a source of income or job because of their HIV status in the past 12 months” had a Pearson correlation coefficient of 0.74 with “Percentage of people living with HIV who have been refused employment or a work opportunity because of their HIV status in the past 12 months,” suggesting that inclusion of both of these indicators may not be necessary due to potential redundancy. We recommend retaining the former and excluding the latter given the former has data available for one additional country. For future assessment, PLHIV Stigma Index 2.0 index combines these 2 indicators which resolves this issue going forward [45]. These same indicators were correlated with “Percentage of people living with HIV who experienced social exclusion in the last 12 months due to their HIV status” (rho = 0.82 and 0.84, respectively). “Feeling shame and guilt” and “Avoiding healthcare out of fear of discrimination” were also correlated at rho = 0.63. Although these indicators are correlated, they are not from the same domain, and, therefore, it is not recommended that they are collapsed or removed at this stage.

Among key population indicators, the sexual minority and gender identity “Discrimination and employment protection policies” indicators were correlated at a rho = 0.68; however, given these are different key populations (men who have sex with men; transgender persons), these indicators were retained at this stage. Indicators with a correlation of rho <0.60 were not considered for combination. Although some indicators showed correlation, no indicators were removed for redundancy at this stage.

Final potential indicators

This mapping exercise identified 24 potential HIV stigma indicators and 10 key population indicators with potential for use in characterizing stigma and creating valid stigma summary measures. The key population indicators include 2 for sexual orientation/behavior stigma related to men who have sex with men; 2 for sex work-related stigma; 2 for drug use stigma; and 4 for gender identity stigma related to transgender people. These are described in detail in Tables 26.

thumbnail
Table 3. Final indicators for sexual behavior/orientation stigma related to men who have sex with men.

https://doi.org/10.1371/journal.pmed.1003914.t003

thumbnail
Table 6. Final indicators for gender identity stigma related to transgender persons.

https://doi.org/10.1371/journal.pmed.1003914.t006

Availability across countries

For the 24 remaining indicators of HIV stigma, 11 (45.8%) were identified as having data available in more than half of the 193 UN member states (Table 2). Among the 10 key population indicators under consideration (Tables 36), data were available for all in more than 50% of UN member states.

Geographic representation

We found 61 of 193 UN member states with any available data (32%) to have data for a sufficient number of HIV stigma indicators (missing data for less than 40% of indicators). Considering all key population stigma indicators together, 119 of 193 countries (62%) had data on a sufficient number of indicators. Countries that meet the criteria of having sufficient data across the 24 HIV indicators are displayed in Fig 2 and for the 10 key population indicators in Fig 3.

thumbnail
Fig 2. Countries with sufficient available data on HIV stigma indicators.

Base layer of the map used are from ArcGIS, ESRI: https://arcg.is/8DHLK. *Data were available for more than 40% of indicators.

https://doi.org/10.1371/journal.pmed.1003914.g002

thumbnail
Fig 3.

Countries with sufficient available data for key population stigma indicators. (A) Sexual behavior/orientation stigma related to men who have sex with men. (B) Sex work stigma. (C) Gender identity stigma related to transgender persons. (D) Drug use stigma. *Data were available for more than 40% of indicators. Base layers of the maps used are from ArcGIS, ESRI: Fig 3A: https://arcg.is/bu8Cu0. Fig 3B: https://arcg.is/Pbzn00. Fig 3C: https://arcg.is/0G0PDG. Fig 3D: https://arcg.is/1HS1zT.

https://doi.org/10.1371/journal.pmed.1003914.g003

Forms of stigma

Summary of results from existing data for the HIV stigma indicators are presented in Table 2. Overall, 44.3% (SD = 20) of women and men 15 to 49 years old report discriminatory attitudes as measured by a composite of 2 questions. The degree of criminalization of HIV and key population associated behaviors varied: 61.7% (N = 92/191) of countries reported existence of laws criminalizing the transmission of, nondisclosure of, or exposure to HIV transmission (Table 2); 35.2% (68/193) of countries reported the existence of laws criminalizing consensual same-sex sexual acts (Table 3); 85.8% (121/141) of countries reported the existence of laws criminalizing sex work or with any punitive measures related to sex work (Table 4); 81.7% (98/120) of countries report the existence of laws criminalizing drug use and/or possession for personal use (Table 5); and 18.5% (25/135) of countries reported existence of laws criminalizing transgender people and/or cross-dressing (Table 6).

Tracking change over time

Of the 24 potential HIV stigma indicators, 19 were found to have potential for use in tracking change in stigma over time, which includes 11 HIV stigma indicators which had data available in more than half of UN member states. All the final 10 stigma indicators for key population stigma, including sexual minority stigma, sex work stigma, drug use stigma, and gender identity stigma related to transgender persons have the potential to be used to track change over time.

Discussion

In 2022, data exist for comprehensively characterizing the nature and level of different forms of stigma affecting people living with HIV and key populations. However, the data mapping exercise described here highlighted a lack of stigma indicators that have ongoing, consistent data collection for assessing change over time, and thus the ability to track progress in mitigating stigma at the country level. In particular, data for indicators outside of the structural stigma domain are much less consistently and rigorously collected and made accessible, particularly for stigma experienced by key populations. There are a few indicators at present that have existing data and are incorporated within specific frameworks for routine data collection mechanisms. Given geographic representation, repeated implementation, and quality, these indicators provide some ability for beginning to explore if and how complex experiences of stigma may be able to be summarized to elucidate the state of stigma within a country and assess change going forward. Despite the current limitations in data availability and quality, participatory approaches involving experts from communities, civil society, academia, and public health practices can be used to help inform the creation of summary measures (i.e., via use of participatory weighting) and to inform data collection priorities moving forward (i.e., via participatory ranking of indicator importance).

Unfortunately, several domains, subdomains, and indicators of stigma were not identified as having sufficient data to use in a summary measure at present despite their importance. No publicly available data sources for appropriately assessing indicators within the domains of social norms and attitudes, or internalized stigma for key populations were identified, and, therefore, these domains had to be removed from key population stigma summary measures at present. Given that social norms and attitudes have been well documented to play a large role in the experiences of stigma among key populations and to be associated with health outcomes, data collection for indicators within this domain should be prioritized in order to include this domain when characterizing stigma at a country level. Indicators within the violence, anticipated stigma, experienced stigma, and stigma and discrimination domains were not retained for consideration as a summary measure for key populations due to limited availability and quality. Therefore, the potential indicators for key population–related stigma only included those from the structural stigma domain. This highlights the importance of expanding data collection for these domains to improve both quality and availability of data that can be used to more comprehensively characterize stigma as it exists in different forms for key populations. Guidance on biobehavioral surveys have been recently developed and can be used by countries to improve consistent data collection of validated measures in the future [49].

Often, only data aggregated at the sample or even country level were able to be located for indicators, which presents a challenge in accounting for differences in sampling strategies when comparing across countries. For instance, individual-level indicators assessing attitudes or experiences of discrimination are often reported at country level, although data may not have been drawn from a nationally representative sample. Statistical methods can be used to better account for sampling approaches to address these concerns of representativeness; however, this is not possible without access to individual-level data, and neither is it possible to recombine or disaggregate samples (such as by age or gender). While there may be consistency in how individual-level surveys are collected, without systems for sharing deidentified data from such surveys, it is not possible to use these data to inform broader comparative analyses aimed at assessing country progress. This limits availability for indicator calculation. The creation of a broader data repository for individual-level deidentified data provides an opportunity to address both of these concerns and keep data for indicators easily accessible for monitoring and analysis. Countries could then better understand the data available and use weighting strategies appropriately to characterize stigma even in the context of limited data. When stigma is measured in surveys, especially when stigmas affecting key populations are assessed, validated measures or scales are often not used, and there is great variety in the measures used across studies [18]. Therefore, an essential step in improving usability of data shared in a repository is increasing the consistency and standardization of stigma measures across countries to facilitate improved comparability of secular trends and health outcomes of stigma.

Stigma mitigation can emerge from both interventions within and outside of the health sector [23]. Measuring secular trends with standardized measures across and within settings can provide insight into the potential impact of interventions aimed at addressing human rights barriers and complementary investments aiming to address stigma [30]. A quantitative measure of stigma, which links directly to diminished uptake of HIV prevention or treatment services, offers an additional indicator against which to measure progress in the AIDS response. HIV (mathematical) modelers have sought such data on barriers to the AIDS response. For example, currently, we do not know if the people who do not know their HIV status do not know their risk or are avoiding testing due to stigma and discrimination. Several indicators identified in this mapping exercise had been revised in recent years, possibly in an effort to improve reporting and measurement. However, the change in definitions over time limits the comparability of indicators over time and complicates the ability to observe change. Among indicators which do currently have sufficient data to compare time periods, most indicators only have data available for a few time points, and many indicators, particularly at the structural level, necessitate longer intervals and follow-up periods to demonstrate meaningful change. Many of the data obtained in this mapping exercise on laws and policies are not available going back more than a few years, although if continued to be prioritized for collection could be used to track changes in stigma into the future.

Summary measures will be meaningful if they provide an accurate characterization of the overall environment of stigma within countries. Combining the indicators and domains identified in this mapping exercise into fewer items as an index may allow for the creation of summary measures. In order for these summary measures to be concrete and valid, indicators will need to be appropriately aggregated and weighted [50]. Based on the findings of this mapping exercise, there are 2 concurrent approaches that would be appropriate for determining indicator weights. The first is empiric, using methods such as exploratory factor analysis, to assess the underlying factor structure of a given set of variables and the strength of the association between a variable and a latent construct when sufficient data exist [51]. As there are insufficient data for conducting this process for key population stigma currently, we also propose a second participatory strategy to determine relative importance and weight indicators. Analytic hierarchy process, a multicriteria decision analysis method that seeks to solve a problem through a hierarchical approach [52], is one example of such a participatory approach. A participatory approach will allow for experts from communities, civil society, academia, and public health practices to fill in gaps where data are currently insufficient. Therefore, despite the limitations in the current data available for indicators identified through this mapping exercise, empiric and participatory approaches can be used alongside the data that do exist to inform the creation of summary measures and set data collection priorities moving forward.

There is heterogeneity in the experiences and burden of stigma within and across populations [4,53]. An essential consideration, true in the creation of any summary measure for a complex construct, is that heterogeneity within countries will be masked in the process of distilling multiple indicators into one or even a few indices. This data mapping exercise demonstrated significantly more data characterizing HIV stigmas than those focused on sexual behavior or orientation, gender identity, sex work, or drug use—challenging our ability to describe heterogeneity within the key population subgroups, such as age, residence, employment, etc. Discussions have emerged in this process to consider the development of a single summary measure of stigma for all key populations in order to clearly guide targeted HIV and human rights policy, advocacy, and practice. Not only may this mask diverse experiences of stigma across these populations, but it may also mask substantial differences in the relative state of stigma and progress toward its elimination. For instance, we found vast differences in the level of criminalization related to different behaviors associated with different key populations. However, creating separate indices may artificially segment experiences that are intersectional for individuals with HIV and membership of a key population. Intersectional stigma is the potentially compounded effect of stigmas among individuals with multiple identities which may be devalued by some in society [54]. Unfortunately, the development of measures for intersectional stigma is still in their nascency and would not be represented in this index [27,55]. Additionally, the emerging field of microaggressions is an important area of ongoing research that aims to characterize and understand stigma processes, and, although there is not yet consensus on how this fits within the stigma framework, there is likely great overlap with individual-level experiences of stigmas. However, this is not represented in the current stigma indicators that were considered and whether and how to characterize such experiences at a broader socioecological level to understand stigma at a country level is an important area of future research. Given that summary measures may not be nuanced enough to appropriately represent intersectionality when combined, in balancing all considerations, we recommend the creation of separate indices for HIV and each key population stigmas with ongoing consideration of how to examine intersectional influences on health. The addition of narratives from advocates and people with lived experience will be essential to consider in conjunction with any summary measure toward this purpose.

The process for creating summary measures to accurately describe and track country-level stigma, while also accounting for heterogeneity and complexity is a challenge, yet not an insurmountable one. As described in conceptual frameworks depicting the processes of stigma, these relationships are complex and often do not occur in isolation [17]. In terms of understanding stigma, there is discussion about the appropriate point of measurement—whether it be the social determinants of stigmas (reflected, for example, in indicators such as acceptability of partner violence, demonstration of controlling behaviors, or having experienced intimate partner violence) or the experiences of stigmas as an outcome or endpoint (reflected, for example, in indicators such as having been socially excluded for living with HIV, having experienced HIV discrimination in healthcare, or having lost a source of income due to living with HIV). Continuing this discussion on the process and methods for stigma measurement at a country level will be an essential step for improving our ability to characterize stigma among and between populations and track country progress in addressing stigma. This should be done in conjunction with, rather than at the expense of, continuing work to measure and assess stigma at other socioecological levels, including the interpersonal and individual levels. Ultimately, measures of stigma at the country level can be used in combination with for instance, individual-level stigma measures, in comparative research to help elucidate the complex interaction between structural stigma and other stigma manifestations.

In addition to the challenges outlined in this discussion, there are several additional limitations to this data mapping exercise that should be considered. The objective of this study was to understand the stigma data that exists for calculating several stigma indicators that have potential use for informing a standardized stigma measure at the country level. As a data mapping exercise, this process was exploratory and therefore may be subject to bias. Although a priori criteria were established and are reported, final decisions on inclusion were based on the state of the data observed during this process. The research and authorship team includes a diverse group of stigma experts from across different academic institutions, multilateral agencies, community networks, and international organizations. This collaborative approach was taken to expand the access to existing stigma data and to minimize selection bias. Our research team used publicly available data, supplemented by data obtained through collaborators, and directly through the research team where possible. This study was therefore unable to assess data that were not publicly available or databases that were not yet available at the time of this data mapping exercise, but could have potentially been shared by institutions outside of this collaborative research team [56,57]. Although systematic reviews of the literature may have informed which data had ever been collected, it would not have yielded the raw data needed to conduct this exercise (i.e., the data would not necessarily be accessible).

Conclusions

Several indicators were determined to have the potential to appropriately characterize the level and nature of stigma affecting people living with HIV and key populations across countries and time based on the data available. Although few available indicators have sufficient data available to use in summary measures for key populations, empiric and participatory approaches for weighting can be used to fill gaps in the current data and inform more comprehensive summary measures. While many countries lack data for the indicators identified in this data mapping exercise and therefore would be underrepresented in any global characterizations, highlighting these gaps can support the direction of funding opportunities, government endorsement, and supportive technical assistance to certain countries/areas to improve the representativeness of measurement of stigma mitigation progress in the future. The creation of a global stigma data repository would also serve to improve the availability and use of stigma measures, and efforts for increased data collection of validated stigma metrics can improve our ability to characterize country-level stigma across various domains in the future. Using the indicators identified as having sufficient data at present for the creation of country-level indices for HIV stigma, sexual minority stigma related to men who have sex with men, sex work stigma, drug use stigma, and gender identity stigma related to transgender persons can better support tracking progress in stigma mitigation for people living with HIV and key populations.

Supporting information

S1 Protocol. Indices on stigma and discrimination related to HIV and key populations (Phase I).

https://doi.org/10.1371/journal.pmed.1003914.s001

(DOCX)

S1 Table. Characteristics of participants in e-consultation online survey.

https://doi.org/10.1371/journal.pmed.1003914.s002

(DOCX)

S2 Table. Proposed indicators for HIV stigma.

https://doi.org/10.1371/journal.pmed.1003914.s003

(DOCX)

S3 Table. Proposed indicators for sexual behavior/orientation stigma related to men who have sex with men.

https://doi.org/10.1371/journal.pmed.1003914.s004

(DOCX)

S4 Table. Proposed indicators for sex work stigma.

https://doi.org/10.1371/journal.pmed.1003914.s005

(DOCX)

S5 Table. Proposed indicators for drug use stigma.

https://doi.org/10.1371/journal.pmed.1003914.s006

(DOCX)

S6 Table. Proposed indicators for gender identity stigma related to transgender persons.

https://doi.org/10.1371/journal.pmed.1003914.s007

(DOCX)

Acknowledgments

We would like to thank all the people living with HIV and members of key populations who agreed to participate in studies across all of the countries where data are available. We would like to thank the community groups and those working to call attention to both measure and address stigmas affecting people living with HIV and key populations. We would like to thank the members of the Task Team developed by UNAIDS Monitoring Technical Advisory Group (MTAG).

The UNAIDS Task Team members not listed as authors in the manuscript were Daniel Reidpath, George Ayala, Erika Castellanos, Ruth Morgan Thomas, Nicholas Niwagaba, Julian Kerboghossian, Sonal Mehta, Joanne Csete, Nazneen Damji, Elena Kudravtseva, Luisa Cabal, Kene Esom, Mianko Ramaroson, and Meaghan Kall.

References

  1. 1. UNAIDS. UNAIDS epidemiological estimates. 2020.
  2. 2. Joint United Nations Programme on HIV/AIDS (UNAIDS). Seizing the Moment: Tackling entrenched inequalities to end epidemics. 2020.
  3. 3. Poteat T, Ackerman B, Diouf D, Ceesay N, Mothopeng T, Odette KZ, et al. HIV prevalence and behavioral and psychosocial factors among transgender women and cisgender men who have sex with men in 8 African countries: A cross-sectional analysis. PLoS Med. 2017;14(11):e1002422. pmid:29112689.
  4. 4. Lyons CE, Schwartz SR, Murray SM, Shannon K, Diouf D, Mothopeng T, et al. The role of sex work laws and stigmas in increasing HIV risks among sex workers. Nat Commun. 2020;11(1):773. Epub 2020/02/20. pmid:32071298.
  5. 5. Cloete A, Simbayi LC, Kalichman SC, Strebel A, Henda N. Stigma and discrimination experiences of HIV-positive men who have sex with men in Cape Town, South Africa. AIDS Care. 2008;20(9):1105–10. Epub 2008/07/09. pmid:18608067; PubMed Central PMCID: PMC3320098.
  6. 6. Kim HY, Grosso A, Ky-Zerbo O, Lougue M, Stahlman S, Samadoulougou C, et al. Stigma as a barrier to health care utilization among female sex workers and men who have sex with men in Burkina Faso. Ann Epidemiol. 2018;28(1):13–9. Epub 2018/02/10. pmid:29425532.
  7. 7. Steward WT, Bharat S, Ramakrishna J, Heylen E, Ekstrand ML. Stigma is associated with delays in seeking care among HIV-infected people in India. J Int Assoc Provid AIDS Care. 2013;12(2):103–9. pmid:22282878
  8. 8. Yam EA, Pulerwitz J, Almonte D, García F, Del Valle A, Colom A, et al. Stigma among key populations living with HIV in the Dominican Republic: experiences of people of Haitian descent, MSM, and female sex workers. AIDS. 2020;34 Suppl 1:S43–s52. Epub 2020/09/04. pmid:32881793.
  9. 9. UNAIDS. End Inequalities. End AIDS. Global AIDS Strategy 2021–2026. Geneva, Switzerland: UNAIDS Coordinating Board; 2021.
  10. 10. Shannon K, Crago AL, Baral SD, Bekker LG, Kerrigan D, Decker MR, et al. The global response and unmet actions for HIV and sex workers. Lancet. 2018;392(10148):698–710. Epub 2018/07/25. pmid:30037733.
  11. 11. Stringer KL, Marotta P, Baker E, Turan B, Kempf MC, Drentea P, et al. Substance Use Stigma and Antiretroviral Therapy Adherence Among a Drug-Using Population Living with HIV. AIDS Patient Care STDS. 2019;33(6):282–93. Epub 2019/06/06. pmid:31166784; PubMed Central PMCID: PMC6588110.
  12. 12. Platt L, Grenfell P, Meiksin R, Elmes J, Sherman SG, Sanders T, et al. Associations between sex work laws and sex workers’ health: A systematic review and meta-analysis of quantitative and qualitative studies. PLoS Med. 2018;15(12):e1002680. Epub 2018/12/12. pmid:30532209.
  13. 13. Latkin C, Srikrishnan AK, Yang C, Johnson S, Solomon SS, Kumar S, et al. The relationship between drug use stigma and HIV injection risk behaviors among injection drug users in Chennai, India. Drug Alcohol Depend. 2010;110(3):221–7. Epub 2010/05/14. pmid:20462707; PubMed Central PMCID: PMC3169329.
  14. 14. Katz IT, Ryu AE, Onuegbu AG, Psaros C, Weiser SD, Bangsberg DR, et al. Impact of HIV-related stigma on treatment adherence: systematic review and meta-synthesis. J Int AIDS Soc. 2013;16(3 Suppl 2):18640. Epub 2013/11/28. pmid:24242258; PubMed Central PMCID: PMC3833107.
  15. 15. Link BG, Phelan JC. Conceptualizing Stigma. Annu Rev Sociol. 2001;27(1):363–85.
  16. 16. Goffman E. Stigma: notes on the management of spoiled identity. Englewood Cliffs, NJ: Prentice Hall; 1963.
  17. 17. Stangl AL, Earnshaw VA, Logie CH, van Brakel W, L CS, Barré I, et al. The Health Stigma and Discrimination Framework: a global, crosscutting framework to inform research, intervention development, and policy on health-related stigmas. BMC Med. 2019;17(1):31. Epub 2019/02/16. pmid:30764826; PubMed Central PMCID: PMC6376797.
  18. 18. Fitzgerald-Husek A, Van Wert MJ, Ewing WF, Grosso AL, Holland CE, Katterl R, et al. Measuring stigma affecting sex workers (SW) and men who have sex with men (MSM): A systematic review. PLoS ONE. 2017;12(11):e0188393. pmid:29190642.
  19. 19. Quinn DM, Chaudoir SR. Living with a concealable stigmatized identity: the impact of anticipated stigma, centrality, salience, and cultural stigma on psychological distress and health. J Pers Soc Psychol. 2009;97(4):634–51. Epub 2009/09/30. pmid:19785483; PubMed Central PMCID: PMC4511710.
  20. 20. Herek GM. Confronting sexual stigma and prejudice: Theory and practice. J Soc Issues. 2007;63(4):905–25. 2007-18628-015.
  21. 21. Zelaya CE, Sivaram S, Johnson SC, Srikrishnan AK, Suniti S, Celentano DD. Measurement of self, experienced, and perceived HIV/AIDS stigma using parallel scales in Chennai, India. AIDS Care. 2012;24(7):846–55. Epub 2012/01/26. pmid:22272891.
  22. 22. Rao D, Desmond M, Andrasik M, Rasberry T, Lambert N, Cohn SE, et al. Feasibility, acceptability, and preliminary efficacy of the unity workshop: an internalized stigma reduction intervention for African American women living with HIV. AIDS Patient Care STDS. 2012;26(10):614–20. Epub 2012/09/19. pmid:22984780; PubMed Central PMCID: PMC3462391.
  23. 23. Stangl AL, Lloyd JK, Brady LM, Holland CE, Baral S. A systematic review of interventions to reduce HIV-related stigma and discrimination from 2002 to 2013: how far have we come? J Int AIDS Soc. 2013;16(3 Suppl 2):18734. Epub 2013/11/28. pmid:24242268; PubMed Central PMCID: PMC3833106.
  24. 24. General comment no. 20, Non-discrimination in economic, social and cultural rights (art. 2, para. 2, of the International Covenant on Economic, Social and Cultural Rights). Geneva: United Nations, 2009 2 July 2009. Report No.: E/C.12/GC/20.
  25. 25. Nyatsanza T, Wood L. Problematizing official narratives of HIV and AIDS education in Scotland and Zimbabwe. SAHARA J. 2017;14(1):185–92. Epub 2017/11/11. pmid:29125014; PubMed Central PMCID: PMC5700494.
  26. 26. Perez-Brumer AG, Reisner SL, McLean SA, Silva-Santisteban A, Huerta L, Mayer KH, et al. Leveraging social capital: multilevel stigma, associated HIV vulnerabilities, and social resilience strategies among transgender women in Lima, Peru. J Int AIDS Soc. 2017;20(1):21462. Epub 2017/04/01. pmid:28362064; PubMed Central PMCID: PMC5467605.
  27. 27. Turan JM, Elafros MA, Logie CH, Banik S, Turan B, Crockett KB, et al. Challenges and opportunities in examining and addressing intersectional stigma and health. BMC Med. 2019;17(1):7. Epub 2019/02/16. pmid:30764816; PubMed Central PMCID: PMC6376691.
  28. 28. UNAIDS. Prevailing Against Pandemics by Putting People at the Centre. 2020.
  29. 29. The Sustainable Development Goals Report 2020. New York, United States: United Nations.
  30. 30. Stangl AL, Singh D, Windle M, Sievwright K, Footer K, Iovita A, et al. A systematic review of selected human rights programs to improve HIV-related outcomes from 2003 to 2015: what do we know? BMC Infect Dis. 2019;19(1):209. Epub 2019/03/06. pmid:30832599; PubMed Central PMCID: PMC6399958.
  31. 31. Lyons C, Stahlman S, Holland C, Ketende S, Van Lith L, Kochelani D, et al. Stigma and outness about sexual behaviors among cisgender men who have sex with men and transgender women in Eswatini: a latent class analysis. BMC Infect Dis. 2019;19(1):211. Epub 2019/03/06. pmid:30832602.
  32. 32. Ha TH, Liu H, Li J, Nield J, Lu Z. Psychometric assessment of scales measuring HIV public stigma, drug-use public stigma and fear of HIV infection among young adolescents and their parents. AIDS Care. 2012;24(1):39–45. Epub 2011/07/16. pmid:21756072; PubMed Central PMCID: PMC3206131.
  33. 33. Hatzenbuehler ML, Phelan JC, Link BG. Stigma as a fundamental cause of population health inequalities. Am J Public Health. 2013;103(5):813–21. Epub 2013/03/16. pmid:23488505; PubMed Central PMCID: PMC3682466.
  34. 34. Hargreaves JR, Stangl A, Bond V, Hoddinott G, Krishnaratne S, Mathema H, et al. HIV-related stigma and universal testing and treatment for HIV prevention and care: design of an implementation science evaluation nested in the HPTN 071 (PopART) cluster-randomized trial in Zambia and South Africa. Health Policy Plan. 2016;31(10):1342–54. Epub 2016/11/02. pmid:27375126; PubMed Central PMCID: PMC6702767.
  35. 35. Stahlman S, Hargreaves JR, Sprague L, Stangl AL, Baral SD. Measuring Sexual Behavior Stigma to Inform Effective HIV Prevention and Treatment Programs for Key Populations. JMIR Public Health Surveill. 2017;3(2):e23. Epub 2017/04/28. pmid:28446420; PubMed Central PMCID: PMC5425775.
  36. 36. Earnshaw VA, Chaudoir SR. From conceptualizing to measuring HIV stigma: a review of HIV stigma mechanism measures. AIDS Behav. 2009;13(6):1160–77. Epub 2009/07/29. pmid:19636699; PubMed Central PMCID: PMC4511707.
  37. 37. Augustinavicius JL, Baral SD, Murray SM, Jackman K, Xue QL, Sanchez TH, et al. Characterizing Cross-Culturally Relevant Metrics of Stigma Among Men Who Have Sex With Men Across 8 Sub-Saharan African Countries and the United States. Am J Epidemiol. 2020;189(7):690–7. Epub 2020/01/17. pmid:31942619.
  38. 38. Lamontagne E, d’Elbée M, Ross MW, Carroll A, Plessis AD, Loures L. A socioecological measurement of homophobia for all countries and its public health impact. Eur J Pub Health. 2018;28(5):967–72. Epub 2018/03/08. pmid:29514190.
  39. 39. Global AIDS Monitoring 2021: Indicators and questons for monitoring progress on the 2021 Political Declaration on HIV and AIDS. Geneva: Joint United Nations Programme on HIV/AIDS; 2021.
  40. 40. Consolidated HIV strategic information guidelines: driving impact through programme monitoring and management. Geneva: World Health Organization; 2020.
  41. 41. Making the end of AIDS real: Consensus building around the what we mean by epidemic control: A meeting convened by the UNAIDS Science Panel. Geneva: UNAIDS; 2017.
  42. 42. UNAIDS. NCPI Database: National Commitments and Policy Instrument AIDSinfo: UNAIDS. Available from: https://onlinedb.unaids.org/ncpi/libraries/aspx/Home.aspx.
  43. 43. UNICEF. Statistics and Monitoring: Multiple Indicator Cluster Survey UNICEF [cited 2021 30 March]. Available from: https://www.unicef.org/statistics/index_24302.html.
  44. 44. The DHS Program. The Demographic and Health Surveys 2021 [cited 2021 30 March]. Available from: https://dhsprogram.com/data/available-datasets.cfm.
  45. 45. The People Living with HIV Stigma Index [cited 2021 30 March 2021]. Available from: https://www.stigmaindex.org.
  46. 46. Rao A, Schwartz S, Sabin K, Wheeler T, Zhao J, Hargreaves J, et al. HIV-related data among key populations to inform evidence-based responses: protocol of a systematic review. Syst Rev. 2018;7(1):220. Epub 2018/12/05. pmid:30509317; PubMed Central PMCID: PMC6278072.
  47. 47. ILGA. Sexual Orientation Laws 2020. Available from: https://ilga.org/maps-sexual-orientation-laws.
  48. 48. Mazziotta M PA. Methods for Constructing Composite Indices: One for All or All for One? Riv Ital Econ Demogr E Stat. 2013;LXVII(2).
  49. 49. Global HIV Strategic Information Working Group. Biobehavioural Survey Guidelines For Key Populations At Risk For HIV. Switzerland World Health. Organization. 2017.
  50. 50. Becker W, Saisana M, Paruolo P, Vandecasteele I. Weights and importance in composite indicators: Closing the gap. Ecol Indic. 2017;80:12–22. Epub 2017/09/05. pmid:28867964; PubMed Central PMCID: PMC5473177.
  51. 51. DeVellis RF. Scale Development: Theory and Applications. 2nd ed. Newbury Park: Sage; 2003.
  52. 52. Greco SIA, Tasiou M, Torrisi G. On the Methodological Framework of Composite Indices: A Review of the Issues of Weighting, Aggregation, and Robustness. Soc Indic Res. 2019;141(1):61–94.
  53. 53. Papworth E, Ceesay N, An L, Thiam-Niangoin M, Ky-Zerbo O, Holland C, et al. Epidemiology of HIV among female sex workers, their clients, men who have sex with men and people who inject drugs in West and Central Africa. J Int AIDS Soc. 2013;16 Suppl 3(4Suppl 3):18751. Epub 2014/01/01. 18751. pmid:24321113; PubMed Central PMCID: PMC3852130.
  54. 54. Bowleg L. The problem with the phrase women and minorities: intersectionality-an important theoretical framework for public health. Am J Public Health. 2012;102(7):1267–73. Epub 2012/05/19. pmid:22594719; PubMed Central PMCID: PMC3477987.
  55. 55. Logie CH, Perez-Brumer A, Mothopeng T, Latif M, Ranotsi A, Baral SD. Conceptualizing LGBT Stigma and Associated HIV Vulnerabilities Among LGBT Persons in Lesotho. AIDS Behav. 2020;24(12):3462–72. Epub 2020/05/13. pmid:32394231; PubMed Central PMCID: PMC7222929.
  56. 56. HIV Policy Lab. Available from: https://www.hivpolicylab.org.
  57. 57. Rule of Law Index. Washington, DC: World Justice Project; 2021.