http://orcid.org/0000-0002-9906-5300
Figures
Abstract
Introduction
Improving social inclusion opportunities for population health has been identified as a priority area for international policy. There is a need to comprehensively examine and evaluate the quality of psychometric properties of measures of social inclusion that are used to guide social policy and outcomes.
Objective
To conduct a systematic review of the literature on all current measures of social inclusion for any population group, to evaluate the quality of the psychometric properties of identified measures, and to evaluate if they capture the construct of social inclusion.
Methods
A systematic search was performed using five electronic databases: CINAHL, PsycINFO, Embase, ERIC and Pubmed and grey literature were sourced to identify measures of social inclusion. The psychometric properties of the social inclusion measures were evaluated against the COSMIN taxonomy of measurement properties using pre-set psychometric criteria.
Results
Of the 109 measures identified, twenty-five measures, involving twenty-five studies and one manual met the inclusion criteria. The overall quality of the reviewed measures was variable, with the Social and Community Opportunities Profile-Short, Social Connectedness Scale and the Social Inclusion Scale demonstrating the strongest evidence for sound psychometric quality. The most common domain included in the measures was connectedness (21), followed by participation (19); the domain of citizenship was covered by the least number of measures (10). No single instrument measured all aspects within the three domains of social inclusion. Of the measures with sound psychometric evidence, the Social and Community Opportunities Profile-Short captured the construct of social inclusion best.
Conclusions
The overall quality of the psychometric properties demonstrate that the current suite of available instruments for the measurement of social inclusion are promising but need further refinement. There is a need for a universal working definition of social inclusion as an overarching construct for ongoing research in the area of the psychometric properties of social inclusion instruments.
Citation: Cordier R, Milbourn B, Martin R, Buchanan A, Chung D, Speyer R (2017) A systematic review evaluating the psychometric properties of measures of social inclusion. PLoS ONE 12(6): e0179109. https://doi.org/10.1371/journal.pone.0179109
Editor: Andre M. N. Renzaho, Western Sydney University, AUSTRALIA
Received: September 7, 2016; Accepted: May 10, 2017; Published: June 9, 2017
Copyright: © 2017 Cordier et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The concepts of social inclusion and exclusion focus on health, social, cultural and income inequalities and imbalances [1]. The term social inclusion is used in social policy and practice documents to highlight the importance of engagement and participation in society as a means of improving quality of life and reducing social isolation [2]. This is because communities that actively include and support individuals and groups to participate in valued social, economic and cultural activities are likely to be healthier than those where people face insecurity, exclusion and deprivation [3]. In order to further develop evidence about the ways in which forms of social inclusion can impact on the wellbeing of individuals, families and communities, it is important to accurately measure and report on what constitutes social inclusion. This paper sets out to consider how the concept of social inclusion has been deployed in policy and practice, how the construct has been operationalised as measures, and identifies the quality of the psychometric properties underpinning the evidence base. This will enable policy makers and practitioners to take a more evidence based approach to evaluating social inclusion initiatives in the future.
Social inclusion: History and definition of the concept
The term social inclusion has been used variously in international social policy and academia; indicating an underpinning policy and practice intent. There is debate about what defines social inclusion, largely due to differences in theoretical and political perspectives. It has also been used interchangeably at times with concepts such as social and cultural capital. Due to this lack of consensus on definition and conceptualisation, the operationalisation and measurement of social inclusion has not been straightforward.
In order to examine the emergence of social inclusion, an inspection of the theories, policies and practices which underpin both inclusion and exclusion is required. While social exclusion and inclusion are often framed as binary opposites, some would suggest the concepts are relative and intertwined and it is not possible to understand or measure social inclusion without reference to social exclusion [4]. However, some argue the two concepts represent entirely different experiences as exclusion suggests “social problems” and inclusion indicates “social membership” [5].
The construct of social exclusion emerged from René Lenoir’s [5] ideas in the 1970s. Lenoir, a French Secretary of State, highlighted incidents of exclusion resulting from poverty, disability, substance misuse, incarceration and mental health problems [5]. Durkheimian [6] ideas about social cohesion and order underpinned the original conceptualisation of social exclusion. Identifying and addressing social exclusion then became a key social policy focus across Europe in the 1980s, Britain in the 1990s and Australia in the 2000s. The policy shift to the language of social exclusion replaced discourses of poverty and disadvantage underpinning arguments that individual welfare payments failed to address the root causes of social exclusion [7]. While the link between social exclusion and poverty has been both emphasised and minimised, some argue that poverty, unemployment and social exclusion are “related, but should not be equated” [8] (p. v), suggesting that social exclusion is a complex set of intersecting variables and experiences which cannot be attributed to one event or factor. Further, Atkinson argues that social exclusion hinges on three key concepts: a) the ‘relativity’ of spatial, temporal and cultural contexts; b) the enactment of ‘agency’ (for example he argues that some ‘choose’ unemployment); and c) the ‘dynamics’ which highlight intergenerational patterns of exclusion [8] (p. 13–14). Social exclusion and inclusion frameworks also consider citizenship and participation in paid labour [9]; which signifies a change from the original focus of exclusion of particular groups to an emphasis on economic participation [4].
Social exclusion can exist across multiple domains and often disrupts activities such as citizenship, participation, social relationships and connections, health, employment, and housing [5, 8]. Aligned with these ideas, Levitas [6] identified three discursive trends within the social exclusion literature which cover impoverishment and exclusion in the economic, social and cultural spheres; the emphasis on paid work as a form of social integration, and a focus on the specific characteristics of excluded individuals. While it is possible to trace various discursive practices in the definitions of social exclusion, it is argued that “what lies at the heart of all processes of social exclusion, is [sic] a sense of social isolation and segregation from the formal structures and institutions of the economy, society and the state” [10]. The notion of community openness to welcome or create a space for those defined as excluded was largely overlooked [5].
Social inclusion as policy and practice
During the 1990s, the concept of social inclusion/exclusion entered state policy discourse providing a ‘catch all’ means to incorporate diverse forms of disadvantage and inequalities. This resulted in social policies across Europe, Britain, and Australia that ambitiously sought to counter the effects of social exclusion and bolster social inclusion. For example, when New Labour was elected in Britain in 1997, social inclusion was its key social policy platform. It was intended to represent a major shift away from the traditional British welfare state with which Labour had historically been associated. The then Prime Minister, Tony Blair, was personally committed to the approach and was highly influenced by the writings of prominent academic Antony Giddens who argued the aspirations for the modern welfare state had not come to fruition and there was a need for a ‘Third Way’ that did not solely rely on national level policy to reduce inequalities in Britain [11].
While the theory and conceptualisation of social exclusion and inclusion has been broad and somewhat fragmented, policy has largely been concerned with reducing unemployment and decreasing homelessness as key pathways to social inclusion. This indicates that policy has been oriented towards economic participation as the primary method for individuals to attain social inclusion. Social inclusion policies have largely promoted individual responsibility for change. However, policy implementation generally paid little attention to differences in material conditions amongst citizens who were unemployed and homeless, such as their cultural background, health, social and cultural capital. Similarly, policy has not deeply engaged with the dynamics, relativity or agency that authors like Atkinson [8] argue is associated with social exclusion. Consequently, measurement of social inclusion within the policy context has focused on the attainment or retention of employment and changes to homelessness status amongst specific populations. However, these measures alone are not adequate indicators of the levels and extent of social inclusion. In short, social inclusion has largely represented an aspirational goal, due to its relatively narrow policy operationalisation.
In contrast, social inclusion in professional practice (for example Community Development, Social Work, Public Health, and Occupational Therapy) has tended to focus on inclusion and participation of marginalised groups in social and community settings, as well as through forms of employment (open through to supported). However, it is rare to see social inclusion measured as a service outcome or indicator of success. As a practice principle underpinning human services, social inclusion has been associated with notions of citizenship and human rights. The assumption being, social connectedness and belonging are key to quality of life [12]. In practice, the social inclusion of service users has incorporated five interdependent aspects of human experience: the growth of relationships, choice and control, experiencing socially valued roles, sharing ordinary places, and contributions [13, 14]. The five aspects are not easily measured and while relative to individual service user’s contexts, enactment of agency and the dynamics associated with their specific lives [8], do not always take into account structural factors. The lack of consensus and clarity about social inclusion in practice has led to an absence of established methods of routinely measuring of the phenomenon to determine service delivery outcomes [15].
Despite the lack of consensus in definition and conceptualisation, the historical and contemporary literature highlights three overarching domains: 1) participation, 2) connectedness and a sense of belonging, and 3) citizenship and rights. For the purpose of this review, participation includes attendance and involvement [16] in social and community spaces and activities; engagement in the labour market and dynamics associated with exclusion, including intergenerational factors which may not be possible to overcome in order to participate. Similarly, participation cannot be accepted on face value. For example, employment as a form of participation is not always inclusion; particularly in marginal employment [8]. Connectedness relates to the actual and potential participation in social and community based activities, organisations, networks and relationships [17]. The relative exclusion of some groups is important to note, as is the potential for individuals to experience spatial connection, yet not have a sense of belonging [8, 18]. Citizenship is operationally defined as the “the link between the state and the individual that implies membership of some form of community…” [19] (p. 104) and is concerned with “a) the content of social rights and obligations; b) with the form or type of such obligations and rights; c) with the social forces that produce such practices; and finally d) with the various social arrangements whereby such benefits are distributed to different sectors of a society.” [20] (p. 3). Additionally, notions of agency, responsibility and the impact of others’ decisions is incorporated in our definition of citizenship [1, 8].
Useful measurements of social inclusion therefore need to include key measures of these three domains and their various components as discussed above. Only one study to date has attempted conduct a review of social inclusion measures [21]; however, the review was not systematic, was limited in scope, and did not use a standardised method to evaluate the quality of the psychometric properties. This systematic review will evaluate the measurements of social inclusion so that the evidence base for understanding the impact and effects of forms of social inclusion on individuals and communities will be better understood in the future and can be adapted and tested with a wider range of diverse groups.
Study aim
This study aimed to identify all current measures of social inclusion for any population group, to evaluate the psychometric properties of these measures, and the extent to which the measures comprehensively evaluate the domains of social inclusion. The COSMIN taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes was used to appraise the psychometric properties of the instruments reviewed [22]. COSMIN provides a consensus on terminology surrounding psychometric properties and a checklist for evaluating the methodological quality of studies reporting on validity, reliability and responsiveness [23]. This study focused on assessing the validity and reliability of all reviewed measures. Evaluation of responsiveness would require a review of studies that have utilised the identified measures as an outcome assessment and would have increased the size of this review significantly. As such, an evaluation of the responsiveness of the reviewed instruments would warrant its own systematic review. Therefore responsiveness as a psychometric property was excluded from this study. It is expected that this systematic review will assist in informing choice when selecting an instrument for the measurement of social inclusion.
Methods
The PRISMA statement guided the methodology and reporting of this systematic review. The PRISMA statement contains a 27 item checklist of elements considered essential for ensuring transparency in performing and reporting of systematic reviews [24]. A completed PRISMA checklist as it pertains to the current review is available (see S1 Table).
Eligibility criteria
Published research articles or manuals assessing the psychometric properties of instruments designed to measure social inclusion in any population were considered for review. Studies selected for review did not have to adhere to a predetermined definition of social inclusion as it is not a narrowly agreed on concept. Rather, the following three domains of social inclusion from the literature were used to guide our review: 1) participation (i.e., economic, social and spiritual), 2) connectedness and a sense of belonging (i.e., having a sense of connectedness to family, friends, neighbours, broader community), and 3) citizenship and rights (i.e., political and general community engagement, demonstrating altruism, and having access to community services). To be selected for review, the overall construct evaluated by an instrument needed to reflect these domains in either children or adults. If social inclusion was evaluated by a single subscale and was not the overall construct assessed by an instrument, the instrument was excluded from the review. Only manuals or published articles written in English in the 20 years prior to the search were eligible for review. Instruments were eligible for review if their psychometric properties were published or updated in the last 20 years (i.e. after 1994), to only capture the psychometric quality of contemporary measures of social inclusion. Conference abstracts, other reviews, case reports, student dissertations and editorials were also excluded.
Information sources
A systematic literature search was conducted using five electronic databases: CINAHL, Embase, ERIC, PsycINFO, and PubMed. Database searches were conducted between 28/07/2015 and the 08/08/2015. Search strategies used both free text words and subject headings, and comprised all journal articles up to August 2015. The database searches were conducted by one author (R.S.) due to her extensive experience in conducting systematic reviews. The databases were accessed from the libraries of Curtin University and James Cook University. The search strategy used for each database is reported in Table 1.
Grey literature was searched using Google Scholar. To be comprehensive, we also searched the websites of three major publishers of assessments in social sciences (Pearson, ACER and Western Psychological Services) to identify potential assessments not identified in earlier search strategies. A web search was also conducted using Google to identify any instruments available via alternative suppliers.
Study selection
A scoring procedure was developed to screen abstracts for inclusion. Abstracts were scored by four independent raters on a three-point scale to determine if: a) the study involved a measure of social inclusion, b) the measure assessed social inclusion or related terms (e.g., social participation, social capital, social responsibility, community inclusion, social justice, social acceptance, social reinforcement, community participation), and c) the study reported on psychometric data of the measure. All abstracts were examined by two reviewers to determine the inter-rater reliability: Weighted Kappa = 0.750 (95% CI: 0.714–0.786). Abstracts that did not meet any of the criteria were immediately excluded from this study. Abstracts that met two or three of the criteria were each screened again by two raters to ensure only studies that met all three eligibility criteria were selected for full text extraction.
Three raters screened the extracted full texts to ensure instruments met the eligibility criteria. Measures were excluded if social inclusion was not the overall construct of the assessment (i.e., if only measured by one subscale) or if the assessment quantified social exclusion. Raters reviewed all full texts together to ensure 100% consensus on reviewed instruments.
Data collection process and data extraction
Data to be extracted from the reviewed studies and manuals were guided by the Cochrane Handbook for Systematic Reviews section 7.3a [25], and the Systematic Reviews Centre for Reviews and Dissemination [26]. Comprehensive data collection forms were developed, and data were captured for the following parameters: study purpose, study population, age of the population, instrument purpose, measure type, number of subscales/forms, number of items, response option types, and domains of social inclusion measured. The COSMIN [23] was also used to capture data and to assess the methodological quality of the studies reviewed.
Methodological quality.
The first phase of the review evaluated the methodological quality of the selected studies. This was performed using the COSMIN taxonomy of measurement properties and definitions for health-related patient-reported outcomes [22]. The COSMIN checklist [23] is a standardised tool for assessing the methodological quality of studies on measurement properties. It evaluates nine domains: internal consistency, reliability (relative measures: including test-retest reliability, inter-rater reliability and intra-rater reliability), measurement error (absolute measures), content validity (including face validity), structural validity, hypotheses testing, cross-cultural validity, and criterion validity. A definition of each psychometric property, as guided by the COSMIN statement, is provided in Table 2. Responsiveness was outside the scope of this review, and criterion validity was not evaluated due to the absence of a ‘gold standard’ measure of social inclusion. Cross-cultural validity was not evaluated as the instruments reviewed were developed and published in English, and interpretability is not considered to be a psychometric property under the COSMIN framework and was therefore not described in this review. The domains of the COSMIN checklist contain 5 to 18 items rated on a four-point scale (poor, fair, excellent, good). The items rate the quality of study design and the robustness of statistical analyses conducted in studies of reliability, validity and responsiveness.
To allocate an overall methodological quality score to each study an alternative system to that which was proposed by the authors of COSMIN was utilised. Terwee, Mokkink [27] suggest taking the lowest rating of any item in a checklist domain as the final quality rating for that domain. It has been noted that subtle differences in the methodological quality between studies are difficult to detect via this method of scoring [28], so a revised scoring procedure was used in this study is as follows. Outcomes are presented as a percentage calculated using the following formula to ensure scores are not unfairly weighted by items that only provide ratings options at the extreme ends of the ordinal scale (i.e., “excellent” and “good”; “fair” and “poor”).
The overall percentage calculated is then categorised as either Poor (0–25.0%), Fair (25.1%-50.0), Good (50.1%-75.0%), or Excellent (75.1%-100.0%). To ensure consistency in the ratings, the sixth author trained four independent research assistants to complete the COSMIN checklist. A random selection of 72% of psychometric property domians were rated by at least two raters. If a discrepancy in COSMIN ratings occurred between raters, articles were given the highest rating percentage if both ratings fell within the same category (i.e. poor, fair, good or excellent). Raters met until 100% consensus was reached when ratings differed in category (ICC = 0.888 [95% CI: 0.795–0.940]).
Quality of psychometric properties.
Phase two assessed the quality of the psychometric properties measured in each study. The results of each study were evaluated using criteria set out by Terwee, Bot [29] and Schellingerhout, Verhagen [30] and Table 3 provides a summary of these criteria. Studies that received a “poor” methodological quality rating in phase one were excluded from further analysis and received a score of NE (not evaluated) in phase two. The raters from phase one also completed phase two, and a random selection of 72% of psychometric properties were evaluated by at least two raters. Raters met until 100% consensus was reached if psychometric quality ratings differed.
Overall quality of psychometric properties.
During the third and final phase, each measurement property for all instruments was given an overall quality score using criteria set out by Schellingerhout, Verhagen [30]. These criteria combine the scores of study quality obtained in phase one with the psychometric quality ratings measured in phase two, thereby creating an overall quality rating. A description of this process is provided in Table 4. Two of the raters from phases one and two gave an overall quality score to each instrument and conferred over discrepancies until 100% consensus was reached.
Data items, risk of bias and synthesis of results
Data items for each instrument were obtained. When an item was not reported, an ‘NR’ was recorded. Risk of bias was assessed at an individual study level during the rating of the COSMIN checklist in phase one. Studies obtaining high ratings during phase one are at low risk of bias, and studies with low ratings are at high risk of bias. Further risk for bias was assessed during phase two, as psychometric domains only received a “positive” or “negative” result if clear and appropriate methodology was reported. Any studies with unclear methodological reporting received an “indeterminate” rating as poor methodology left results open to bias. As the ratings from phase one and two were combined to create an overall rating for each psychometric property of each instrument, the risk of bias is subsumed into the final results. The results were synthesised and grouped as follows: 1) development and validation of the instrument, 2) the psychometric properties of the instruments, and 3) the instrument characteristics.
Results
Systematic literature search
A total of 8,541 abstracts were retrieved from five databases with the following breakdown: CINAHL = 954, ERIC = 2,090, Embase = 1,680, PsycINFO = 1,639, PubMed = 2,178. Fig 1 presents the flow diagram of the revision process according to PRISMA [24]. Reference lists of the reviewed articles were examined for further publications meeting the eligibility criteria. The grey literature search identified an additional 85 records. A total of 1,442 duplicates across the five databases were removed, leaving a total of 7,099 studies to screen for inclusion in this review. Following abstract screening, 127 full-text articles reporting on 108 different instruments were further assessed for eligibility. Of these 108 measures, 84 were excluded for the following reasons: 1 was published before 1994, 3 did not report psychometric data, 6 were published in dissertations, 23 were developed or published in languages other than English, and 51 did not measure the domains of social inclusion adopted for this review. Table 5 lists the 84 excluded instruments and reasons for their exclusion. One manual was located through additional searches. Thus, the psychometric properties were obtained for a total of 25 social inclusion measures which were accessed using 25 articles and 1 manual.
Measures of social inclusion
A summary of the studies on the development and validation of the 25 social inclusion measures reviewed is reported in Table 6. One measure was developed using an adolescent sample (12–17 years), with all others using an adult population alone. Of the 25 measures, 10 measures were developed and validated using a sample of adults with severe mental illnesses. Six were developed and validated with community samples; with 3 of these measures using a sample from rural communities or low-income neighbourhoods. Only one measure was developed and validated with both mentally healthy adults and adults with severe mental illnesses (Social and Community Opportunities Profile [SCOPE] [113]). Two measures used samples of adults without a tertiary education, 1 measure used a sample of adults with an intellectual disability and their carers, and 1 measure sampled caregivers of children with chronic illnesses.
Table 7 describes the characteristics of the reviewed measures. Of the 25 measures, 19 were published within the last 10 years (since 2005). Regarding the measure type, all used self-report with the exception of the Guernsey Community Participation and Leisure Assessment (GCPLA) which used carer-report if the respondent was unable to answer for themselves [119]. Ten measures collected responses via interviews; seven of which were conducted face-to-face, two of which were conducted over the telephone, and one which was administered via both modalities. Fourteen were self-report questionnaires, 3 of which were administered online, and 11 of which were administered via paper and pen (see Table 7). Response options varied greatly between measures; 16 reported the use of Likert-type scales, and 5 reported differing response types per item. Five of the measures using Likert-type scales reported using a 6-point scale, 7 reported using a 5-point scale, and 3 reported using a 4-point scale. Measures requiring differing response types utilised a combination of ordinal and nominal scales. The Sense of Community Index (SCI) reported the use of a dichotomous (true or false) rating system for its scale [124]. The Inclusion Web utilises a visual “web” in which respondents list people or places under various response categories [136]. Response options for the Activity and Participation Questionnaire (APQ) were not reported [114].
The domains of social inclusion measured by each instrument are summarised in Table 8. The sub-domains were categorised following a thematic synthesis by two members of the research team of the scales and subscales used by the reviewed measures and, where available, based on the definitions or descriptions of the scales and/or subscales provided in the reviewed studies. Based on the thematic analysis the following sub-domains were identified and subsumed under the most relevant domain: 1) participation (i.e., economic, social and spiritual), 2) connectedness and a sense of belonging (i.e., having a sense of connectedness to family, friends, neighbours, broader community), and 3) citizenship (i.e., political and general community engagement, demonstrating altruism, and having access to community services). Aspects of participation were measured by 19 instruments, 21 instruments evaluated aspects of connectedness and a sense of belonging, and aspects of citizenship were measured by 14 instruments. Ten measures included aspects of all three overarching domains of social inclusion, but no single instrument measured all sub-domains of participation, connectedness and a sense of belonging, and citizenship.
Psychometric properties
The methodological quality ratings of the studies reviewed are summarised in Table 9. Table 10 summarises the quality of the psychometric properties of the 25 measures based on the quality criteria described by Terwee, Bot [29] and Schellingerhout, Verhagen [30] (see Table 3). Table 11 provides an overall psychometric quality rating for each psychometric property using the criteria of Schellingerhout, Verhagen [30]. A description of the criteria used to rate psychometric quality is provided in the notes section for Table 10. As described by Schellingerhout, Verhagen [30], the overall level of psychometric quality (Table 11) is derived by integrating the ratings of 1) the methodological quality of the studies using the COSMIN checklist (Table 9); and 2) the quality criteria for the psychometric properties of assessments (Table 10).
Discussion
The purpose of this systematic review was to identify and evaluate the extent to which contemporary measures of social inclusion evaluate the construct in any population group, and the quality of their psychometric properties. The measurement of social inclusion is important to policy makers in health and social services as it can bring together a combination of economic, social, geographical and individual factors; the combination of which are increasingly being understood to influence health and social outcomes of populations. The systematic review of social inclusion measures provides a comprehensive summary of the quality of the psychometric properties of these measures.
Findings on psychometric properties
The systematic review identified a total of 25 measures published across 25 papers and 1 manual. For 24 measures, only single studies were identified reporting on one or more of the psychometric properties within the scope of this review. Only the SIS had two psychometric studies. Most studies only addressed a few of the six measurement properties evaluated within this review (average 3; range 1–5). Furthermore, when determining the overall quality score per psychometric property per measure, 45% of the overall ratings was classified as indeterminate. Consequently, the reporting of psychometric properties of social inclusion measures within the literature paints an incomplete picture. The lack of psychometric data in the literature is worrying. Whilst missing data do not necessarily indicate poor psychometric quality, without this knowledge clinicians and researchers are selecting measures based on incomplete psychometric evidence. Missing data on reliability, validity and responsiveness of measures, have an impact on the generalisability and interpretation of results.
Evaluation of the reliability (internal consistency, test-retest, interrater or measurement error) was conducted on a majority of reviewed measures (20 of 26). Internal consistency was the most frequently reported psychometric domain and was evaluated with strong methodological quality producing Cronbach’s alphas in the acceptable range in six instruments. In addition, the SCOPE-Short produced strong evidence for test-retest reliability. Issues with methodological quality were usually the reason for “indeterminate” results in the final overall quality scores for internal consistency, reliability and measurement error. In evaluations of internal consistency, most studies failed to collect an adequate sample size for the number of items in the instrument of focus. This may have been because of the specific population groups for which the instruments were validated. Problems with the recruitment of hard to reach populations (e.g., adults with mental illnesses, rural communities, those from low socioeconomic areas) may have reduced the study sample sizes, and these instruments require further validation with larger sample sizes so that conclusions about their psychometric properties can be drawn. Notably, when an adequate sample size was collected, internal consistency results were usually positive. Further methodological problems were evident in most evaluations of test-retest reliability, with researchers opting to report Pearson’s or Spearman’s correlations rather than Kappa or ICCs.
All but one instrument underwent an evaluation of at least one aspect of validity (i.e., content validity, structural validity and/or hypotheses testing). Results for all instruments were mixed, with many finding positive results in one aspect of validity and negative or indeterminate results in another. Inadequate reporting led to “indeterminate” results for the overall quality assessment of structural and content validity. Specifically, descriptions of measurement aims, target populations, concepts measured and means of item selection estimates were unclear or absent from studies reporting on content validity. Additionally, estimates of variance were not reported in some studies of structural validity. However, when adequate reporting was detected, overall quality scores for content validity and structural validity were usually positive. Most results for hypotheses testing were deemed indeterminate or negative. Indeterminate results were due to inadequate sample sizes and when studies utilised adequate sample sizes, ratings were often negative due to weak (r < 0.5) and/or statistically insignificant correlations. Criterion validity could not be assessed due to the absence of a “gold standard” measure for social inclusion, and cross-cultural validity was outside the scope of this review.
When considering those measures that showed no negative psychometric evidence (13 measures), the Social Connectedness Scale and the SCOPE Short seem to be the most promising measures. For the Social Connectedness Scale, strong positive psychometric evidence was found on three properties (internal consistency, content validity and structural validity) and indeterminate evidence on a fourth property (reliability). For the SCOPE-Short, strong positive evidence was found for reliability, moderate positive evidence for content validity and hypotheses testing, and indeterminate ratings for internal consistency and structural validity. Next, the SIS showed strong and moderate positive evidence on two properties: hypotheses testing and internal consistency, respectively. Data on reliability scored indeterminate. The other ten measures without negative evidence ratings, showed either positive evidence on single psychometric properties (5 measures) or indeterminate ratings (4 measures) only, resulting in very incomplete psychometric overviews for these measures. Four measures showed only negative psychometric evidence (BSCS, CPDM, SCI and SinQUE) in addition to indeterminate ratings. Finally, eight measures showed a combination of positive and negative evidence for at least two psychometric properties. When considering the overall psychometric quality scores for all 25 measures, many data proved missing or indeterminate and indicated an urgent need for further research to determine the psychometric properties of these measures. Further, the use in policy evaluation and clinical practice of measures having poor psychometric properties should not be supported.
Overall, the results demonstrate that the current suite of available instruments for the measurement of social inclusion is promising, but requires further refinement. There is a need for researchers to utilise more robust methodology when evaluating psychometrics, particularly in relation to the collection of adequately sized samples and the selection of statistical tests. While no instrument received a “poor” rating for methodological quality, flaws in methodology reduced the ability to draw conclusions about results in many studies. There is also a need for more complete reporting of instrument purpose, concepts assessed, target populations, and selection of items. Without this knowledge, there is a risk of clinicians, researchers and policy developers making inappropriate instrument selections.
The findings of the review also support the need for further consideration of instrument design when attempting to measure social inclusion. All identified instruments were self-report measures. While there are a number of advantages to using self-report measures, a disadvantage of self-report methodology is the potential for inaccurate reporting by the respondent [138]. Similarly, some of the measures used Likert scales in combination with dichotomous and nominal scales. Deciding on a scale and response format to use is not simple and requires attention to the meaning of the terms and words as well as the context [139]. Other design considerations emerged related to the fact that all but one of the identified measures (SCCS) were developed and validated with adults only. Moreover, the most frequently sampled population for the development and validation of the social inclusion measures was adults diagnosed with mental health problems. Further validation of instruments for the general populations, as well as populations at risk of social exclusion would allow researchers and policy makers to evaluate the impact of social policies and specific interventions for population subgroups as well as the broader population.
Social inclusion theory and measurement
The systematic review utilised social inclusion theory to inform a deductive thematic analysis of the findings [140]. The three domains of social inclusion (i.e., participation, connectedness and a sense of belonging, and citizenship) were used to analyse the reviewed instruments in relation to how comprehensively they assess the construct of social inclusion. The domain of participation includes the sub-domains of economic, social and spiritual participation. The domain of connectedness included four sub-domains: family, friends, neighbours, and broader community. Finally, the domain of citizenship comprised of four sub-domains: political, altruism, community engagement and access to community services. No single measure captured the complexities of social inclusion represented by these domains, and as such we have identified gaps in measuring social inclusion from a theoretical perspective.
Participation as a domain of social inclusion has previously been identified as an important predictor of social inclusion [141]. Often, vulnerable populations are left marginalised and at risk of reduced opportunities to participate in society [118]. The three sub-domains of economic, social and spiritual participation were identified as being consistent with how individuals contribute to and participate in their community.
The sub-domain of economic participation included employment, self-employment enterprise development, education and training [142]. Seven out of the 25 identified measures included the sub-domain of economic participation, but definitions of work and paid employment varied between measures. Some measures focused primarily on paid employment as a gateway to participation in society, however employment is not a sole guarantor of social inclusion [143]. As such, measures of social inclusion require broader consideration and examination of the concept of work, employment and education. From this perspective, facilitating participation to enhance social inclusion requires more than enabling people to enter paid employment. A broad perspective of a person’s means of contributing, participating and belonging to society is required [99].
Social roles are thought to be a nuanced aspect of participation and more than simple engagement in daily activities [42]. At a societal level, The World Health Organization interprets social participation within a number of different forms, including empowering communities to retain ultimate control over the key decisions that affect their wellbeing [144]. At the level of the individual, social participation includes participation in formal community organisations, informal community networks and activities, volunteer work, and care of family (including children and elderly) [142]. Kawachi, Kennedy [145] indicate low social participation may be a pathway associated with deprivation and poor health, and reduced social support and anchorage are often negatively associated with poor mental health outcomes [146]. Seventeen out of the 25 identified measures included the sub-domain social participation.
The remaining sub-domain of participation, spirituality, relates to participation in groups and activities with others who have similar beliefs and a common way of worship. A number of studies have noted that amongst other benefits, education and awareness around religious diversity and spirituality has an important role in advancing social inclusion [147]. Yet, only two of the 25 identified measures incorporated questions related to sub-domain of spirituality, making it an under-recognised aspect in the measurement of social inclusion.
The domain of connectedness and a sense of belonging relates to relationships within societal groups and associated feelings of emotional attachment [17]. Connectedness identifies social norms within a group (i.e., family or friendships) that may provide strong motivation to remain connected [148, 149]. Belongingness, a fundamental human need [150], adds an emotional aspect to the domain, as it is possible to be connected but not emotionally attach. As such, to facilitate true social inclusion a person needs to be both connected and have a sense of belonging. Becoming involved in community groups or organisations is one way of increasing a sense of connection and belongingness in a complex and fragmented society [151], however complete interpersonal integration means having a diversity in social networks (e.g., family, friends, neighbours, community groups) to provide care and companionship and moral support [113]. This systematic review identified nine measures that included the sub-domain of family, 12 that included the sub-domain of friends, and nine included the sub-domain of neighbours. With twenty-one out of the 25 identified measures having included the sub-domain of broader community connectedness and sense of belonging, it was the most common sub-domain captured within the measures of social inclusion.
In contrast to the connectedness domain, the domain of citizenship considers social inclusion as more than just participation and belonging within family, friendship and other social networks. Citizenship implies membership in a community with associated rights and obligations, and the ‘extent’ of citizenship is determined by the rules and norms of inclusion and exclusion that a society develops to define the boundaries of membership [152]. As per the sub-domains of citizenship adopted for this review, an individual can exercise citizenship through community engagement, community service access, political activism, and acts of altruism.
Social inclusion requires opportunities for community engagement, which in turn creates opportunities to reduce health inequities and increase positive mental and physical health outcomes [153–156]. The sub-domain of access to community services was also included as a sub-domain in this review, because accessing services is very different from engaging in the community. Community engagement was the most frequently measured sub-domain of citizenship, found in 11 of the 25 measures reviewed. Accessing community services was again an under-evaluated concept, appearing in four of the 25 measures.
The World Summit for Social Development [157] considers an inclusive society as one in which every individual has an active role in meeting their own rights and responsibilities. This highlights the importance of political action within the construct of social inclusion. Political action provides an avenue for individuals to influence their rights and responsibilities, and this is realised by accessing a sense of trust gained from reciprocal contribution to a network [158]. The notion of being able to “have a say” bestows a sense of empowerment upon the individual, and to be included in society there must be opportunities to have a political voice and take political action [159]. Altruism has also been shown to influence behaviours toward an inclusive community [160], and Cobigo, Ouellette-Kuntz [141] propose that the definition and value of social capital must also include altruism. Four of the 25 measures reviewed included the sub-domain of political action and three included the sub-domain of altruism, highlighting these as under-evaluated domains within the construct of social inclusion.
When we integrate the findings from the psychometric qualities of the identified measures with how well the measures cover the construct of social inclusion from a theoretical perspective, the SCOPE Short [113] has shown itself to be the most promising measure of social inclusion (covering 7 out of 11 sub-domains), followed by the SIS [134] (covering 5 out of 11 subdomains). While the quality of the psychometric properties of the Social Connectedness Scale [129] shows promise, it is narrow in its measurement of the construct of social inclusion. Overall the findings highlight the need for more research to fully capture the complex construct of social inclusion and to validate the measures using sound psychometric methodologies.
Conclusion
This systematic review reported evidence of the quality of psychometric properties of the 25 instruments used to measure social inclusion with any population. The COSMIN taxonomy, [22] was used to rate the reliability and validity information reported about the instruments. No single measure of social inclusion was found to demonstrate a consistent level of psychometric evidence across the six psychometric properties appraised. The research findings indicate there is then a need for a “gold standard” measure of social inclusion that utilises a more vigorous methodological design, including using adequate sample sizes and appropriate statistical analyses. Furthermore, the breadth of the definition of social inclusion highlights the necessity for having an expansive measure to fully capture all the nuances of the highly complex construct. None of the identified measures completely capture all aspects associated with social inclusion across the domains of participation, connectedness and a sense of belonging, and citizenship. The SCOPE-Short was the measure with the best evidence of sound psychometric properties and covering the breadth of the construct of social inclusion. In conclusion, a broad-based measure of social inclusion can offer policy makers with the opportunity to develop an evidence base that can be used to underpin the development of health and social policies and evaluate their impact following implementation.
Acknowledgments
We would like to thank research assistants, Belinda Cuomo, Tomomi McAuliffe, Xian Wei Liew, Lauren Parsons, Rebekah Totino, Jaya Saraswati, Daisy de Groot, Hannah Overheu, Donna-Lee Speedie, Michelle Blogna and Emma Adams who assisted with abstract screening and instrument ratings.
Author Contributions
- Conceptualization: RC BM AB DC RM RS.
- Data curation: RC RS.
- Formal analysis: RC RS.
- Methodology: RC BM AB DC RM RS.
- Validation: RC RS.
- Visualization: RC BM RS.
- Writing – original draft: RC BM AB DC RM RS.
- Writing – review & editing: RC BM AB DC RM RS.
References
- 1.
Atkinson AB, Marlier E. Analysing and Measuring Social Inclusion in a Global Context. New York: United Nations, 2010.
- 2. Oxoby R. Understanding social inclusion, social cohesion, and social capital. International Journal of Social Economics. 2009;36(12):1133–52.
- 3.
World Health Organization. The Solid Facts: Social Determinants of Health. Copenhagen, Denmark: 2003.
- 4. Peters MA, Besley TAC. Social exclusion/inclusion: Foucault's analytics of exclusion, the political ecology of social inclusion and the legitimation of inclusive education. Open Review of Educational Research. 2014;1(1):99–115.
- 5. Silver H. Understanding social inclusion and its meaning for Australia. Australian Journal of Social Issues. 2010;45(2):183.
- 6.
Levitas R. The concept and measurement of social inclusion. In: Pantazis C, Gordon D, Levitas R, editors. Poverty and Social Exclusion in Britain. Bristol, UK: The Policy Press; 2006. p. 123–60.
- 7. Lister R. From equality to social inclusion: New Labour and the welfare state. Social Policy and Society. 1998;18(2):215–25.
- 8.
Atkinson AB, Hills J. Social exclusion, poverty and unemployment. London, UK: CASEpaper; 1998.
- 9. Levitas R. The concept of social exclusion and the new Durkheimian hegemony. Critical Social Policy. 1996;16(46):5–20.
- 10. Somerville P. Explanations of social exclusion: where does housing fit in? Housing studies. 1998;13(6):761–80.
- 11.
Giddens A. The Third Way: The Renewal of Social Democracy. Bristol: Polity Press; 1998.
- 12. Haslam C, Cruwys T, Haslam SA, Jetten J. Social connectedness and health. Encyclopaedia of Geropsychology. 2015:46–1.
- 13.
O'Brien J. What's Worth Working For?: Leadership for Better Quality Human Services: Center on Human Policy, Syracuse University; 1989.
- 14. Van Asselt D, Buchanan A, Peterson S. Enablers and barriers of social inclusion for young adults with intellectual disability: A multidimensional view. Journal of Intellectual and Developmental Disability. 2015;40(1):37–48.
- 15. Bigby C. Social inclusion and people with intellectual disability and challenging behaviour: A systematic review. Journal of Intellectual and Developmental Disability. 2012;37(4):360–74. pmid:23002899
- 16. Imms C, Adair B, Keen D, Ullenhag A, Rosenbaum P. Participation: a systematic review of language, definitions, and constructs used in intervention research with children with disabilities. Dev Med Child Neurol. 2016;58:29–38. pmid:26411643
- 17.
Taket A, Crisp BR, Nevill A, Lamaro G, Graham M, Barter-Godfrey S. Theorising social exclusion: Routledge; 2009.
- 18.
Standsfeld SA. Social support and social cohesion. In: Marmot M, Wilkinson RG, editors. Social Determinants of Health. 2nd ed. Oxford: Oxford University Press; 2006.
- 19.
Lewis G. Forming nation, framing welfare. New York, NY: Routledge in association with the Open University 1998.
- 20.
Turner BS. Citizenship and Social theory. London: Sage Publications Ltd; 1993.
- 21. Coombs T, Nicholas A, Pirkis J. A review of social inclusion measures. Aust N Z J Psychiatry. 2013;47(10):906–19. pmid:23737598
- 22. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63(7):737–45. pmid:20494804
- 23. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19(4):539–49. pmid:20169472
- 24. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9. pmid:19622511
- 25.
Higgins JP, Green S. Cochrane handbook for systematic reviews of interventions: Wiley Online Library; 2008.
- 26.
Centre for Reviews Dissemination. Systematic reviews: CRD's guidance for undertaking reviews in health care. Layerthorpe, York: CRD, University of York; 2009.
- 27. Terwee CB, Mokkink LB, Knol DL, Ostelo RW, Bouter LM, de Vet HC. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012;21(4):651–7. pmid:21732199
- 28. Speyer R, Cordier R, Kertscher B, Heijnen BJ. Psychometric properties of questionnaires on functional health status in oropharyngeal dysphagia: A systematic literature review. BioMed Research International. 2014;2014. pmid:24877095
- 29. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34–42. pmid:17161752
- 30. Schellingerhout JM, Verhagen AP, Heymans MW, Koes BW, de Vet HC, Terwee CB. Measurement properties of disease-specific questionnaires in patients with neck pain: A systematic review. Qual Life Res. 2012;21(4):659–70. pmid:21735306
- 31. McMillan DW, Chavis DM. Sense of community: A definition and theory. J Community Psychol. 1986;14(6–23).
- 32.
Filia K. Social inclusion and mental illness [PhD Thesis]. Melbourne, Victoria: The University of Melbourne; 2014.
- 33.
Leader AE. The predictive validity of a social capital index in routine breast and cervical cancer screening [PhD Thesis]: George Washington University; 2008.
- 34.
Neeleman L. Factor analysis of the Youth-outcome Questionnaire: Evidence of construct validity [PhD Thesis]: Brigham Young University; 2001.
- 35.
Story WT. Social capital and health in the developing world: Meaning, mechanisms, and measurement [PhD Thesis]: Univerity of Michigan; 2013.
- 36.
Schibler JJ. The social capital of decision-making groups [PhD Thesis]: University of Rhode Island; 2010.
- 37.
Lott DL. Perceptions of college readiness and social capital of GED completers in entry-level college courses [PhD Thesis]: University of New Orleans; 2012.
- 38. Amagai M, Suzuki M, Shibata F, Tsai J. Development of an instrument to measure self-efficacy for social participation of people with mental illness. Arch Psychiatr Nurs. 2012;26(3):240–8. pmid:22633586.
- 39. Arslan E, Şahbaz Ü. A study to develop a scale for determining the social acceptance levels of special-needs students participating in inclusion practices. Educational Research and Reviews. 2012;7(29):651–62.
- 40. Kono A, Tadaka E, Kanaya Y, Dai Y, Itoi W, Imamatsu Y. Development of a community commitment scale with cross-sectional survey validation for preventing social isolation in older Japanese people. BMC Public Health. 2012;12:903. pmid:23095335
- 41. Koster M, Minnaert AEMG, Nakken H, Jan Pijl S, van Houten EJ. Assessing social participation of students with special needs in inclusive education: Validation of the Social Participation Questionnaire. J Psychoeduc Assess. 2010;29(3):199–213.
- 42. Mars GMJ, Kempen GIJM, Post MW, Proot IM, Mesters I, van Eijk JTM. The Maastricht Social Participation Profile: Development and clinimetric properties in older adults with chronic physical ilness. Qual Life Res. 2009;18(9):1207–18. pmid:19768655
- 43. Sosna U. Empirical measurement of social isolation in relation to mental disorders of the elderly. Acta Psychiatr Scand. 1980;62(S285):220–9.
- 44. van Bergen AP, Hoff SJ, van Ameijden EJ, van Hemert AM. Measuring social exclusion in routine public health surveys: Construction of a multidimensional instrument. PLoS One. 2014;9(5):e98680. pmid:24878842
- 45. van Beuningen J, Schmeets H. Developing a Social Capital Index for the Netherlands. Social Indiactors Research. 2012;113(3):859–86.
- 46. Jansen WS, Otten S, Van der Zee KI, Jans L. Inclusion: Conceptualisation and measurement. Eur J Soc Psychol. 2014;44:370–85.
- 47. Burnette D, Myagmarjav S. Translation and validation of the 18-item Lubben Social Network Scale with older adults in Mongolia. Int Psychogeriatr. 2013;25(9):1493–502. pmid:23790039
- 48. Story WT, Taleb F, Ahasan SM, Ali NA. Validating the measurement of social capital in Bangladesh: A cognitive approach. Qual Health Res. 2015;25(6):806–19. pmid:25857652
- 49. Koutra K, Orfanos P, Roumeliotaki T, Kritsotakis G, Kokkevi A, Philalithis A. Psychometric validation of the Youth Social Capital Scale in Greece. Research on Social Work Practice. 2011;22(3):333–43.
- 50. Kritsotakis G, Koutis AD, Alegakis AK, Philalithis AE. Development of the Social Capital Questionnaire in Greece. Res Nurs Health. 2008;31(3):217–25. pmid:18213683
- 51. Lee O, Kim Y, Kim OJ. Relations of perception of responsibility to intrinsic motivation and physical activity among Korean middle school students. Percept Mot Skills. 2012;115(3):944–52. pmid:23409606
- 52. Suttiwong J, Vongsirinavart M, Vachalathiti R, Chaiyawat P. Impact on Participation and Autonomy Questionnaire: Psychometric properties of the Thai version. Journal of Physical Therapy Science. 2013;25(7):769–74. pmid:24259849
- 53. Yari A, Nadrian H, Rashidian H, Nedjat S, Esmaeilnasab N, Doroudi R, et al. Psychometric properties of the Persian version of Social Capital Questionnaire in Iran. Med J Islam Repub Iran. 2013;28(17):1–12.
- 54. Safipour J, Tessma MK, Higginbottom G, Emami A. Measuring social alienation in adolescence: Translation and validation of the Jessor and Jessor Social Alienation Scale. Scand J Psychol. 2010;51(6):517–24. pmid:20338013
- 55. Herrero J, Gracia E. Measuring perceived community support: Factorial structure, longitudinal invariance, and predictive validity of the PCSQ (Perceived Community Support Questionnaire). J Community Psychol. 2007;35(2):197–217.
- 56. Kelders R, van Brakel W, Beise K, Irwanto . Testing and validating a simplified scale to measure social participation of people with disabilities in Indonesia. Disabil Rehabil. 2012;34(8):638–46. pmid:21992486
- 57. Wang P, Chen X, Gong J, Jacques-Tiura AJ. Reliability and validity of the Personal Social Capital Scale 16 and Personal Social Capital Scale 8: Two short instruments for survey studies. Social Indicators Research. 2013;119(2):1133–48.
- 58. Enfield RP, Nathaniel KC. Social capital: Its constructs and survey development. New Directions in Youth Development. 2013;2013(138):15–30. pmid:23878083
- 59.
Magne S, McTiernan A. Open hearts, open minds: A social inclusion self-assessment handbook; for staff working with the public and those who support them. Exeter, UK: Exeter Community Initiatives; 2004.
- 60.
Public Health England. Everybody Active, Every Day: An evidence-based approach to physical activity London, UK2014. Available from: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/353384/Everybody_Active__Every_Day_evidence_based_approach_CONSULTATION_VERSION.pdf.
- 61. Keyes CLM. Social well-being. Soc Psychol Q. 1998;61(2):121–37.
- 62. Brutt AL, Schulz H, Andreas S. Psychometric properties of an instrument to measure activities and participation according to the ICF concept in patients with mental disorders. Disabil Rehabil. 2015;37(3):259–67. pmid:24833419
- 63. Cheung C, Kam PK. Bonding and bridging social capital development by social workers. Journal of Social Service Research. 2010;36(5):402–13.
- 64. Andrews DW, Francis S. Development of Perceived Adolescent Relationship Scale. Percept Mot Skills. 1989;69(1):305–6. pmid:2780193
- 65. Bode RK, Heinemann AW, Butt Z, Stallings J, Taylor C, Rowe M, et al. Development and validation of participation and positive psychologic function measures for stroke survivors. Arch Phys Med Rehabil. 2010;91(9):1347–56. pmid:20801251
- 66. Chaplin FS. Social participation and social intelligence. Am Sociol Rev. 1939;4(2):157–66.
- 67. Donohue MV. Social Profile: Assessment of validity and reliability with preschool children. The Canadian Journal of Occupational Therapy. 2005;72(3):164–75. pmid:15988963
- 68. Cornwell EY, Waite LJ. Measureing social isolation among older adults using multiple indicators from the NSHAP study. Journal of Gerontology: Social Sciences. 2009;64B(S1):i38–i46. pmid:19508982
- 69. Dishion TJ, Kim H, Stormshak EA, O'Neill M. A brief measure of peer affiliation and social acceptance (PASA): Validity in an ethnically diverse sample of early adolescents. J Clin Child Adolesc Psychol. 2014;43(4):601–12. pmid:24611623
- 70. Furtado SRC, Sampaio RF, Vaz DV, Pinho BAS, Nascimento IO, Mancini MC. Brazilian version of the instrument of environmental assessment Craig Hospital Inventory of Environmental Factors (CHIEF): translation, cross-cultural adaptation and reliability. Brazilian Journal of Physical Therapy. 2014;18(3):259–67. pmid:25003279
- 71. Han C-W, Yajima Y, Lee E-J, Nakajima K, Meguro M, Kohzuki M. Validity and utility of the Craig Hospital Inventroy of Environmental Factors for Korean community-dwelling elderly with or without stroke. Tohoku J Exp Med. 2005;206:41–9. pmid:15802874
- 72. Rigby S, Thornton E, Tedman S, Burchardt F, Young C, Dougan C. Quality of life assessment in MND: development of a Social Withdrawal Scale. J Neurol Sci. 1999;169(1):26–34.
- 73. Gilman R, Carter-Sowell A, Dewall CN, Adams RE, Carboni I. Validation of the ostracism experience scale for adolescents. Psychol Assess. 2013;25(2):319–30. pmid:23205625.
- 74. Hawthorne G. Measuring social isolation in older adults: Development and initial validation of the friendship scale. Social Indicators Research. 2006;77:521–48.
- 75. Hayes RP, Vogtle LK, Allaire J, Jones AK, Blair AE. Development and preliminary validation of a measure of social functioning for adolescents with physical disabilities. Journal of Rehabilitation and Outcomes Measurement. 1999;3(3):34–41.
- 76. Hitzig SL, Romero Escobar EM, Noreau L, Craven BC. Validation of the Reintegration to Normal Living Index for community-dwelling persons with chronic spinal cord injury. Arch Phys Med Rehabil. 2012;93(1):108–14. pmid:22200389
- 77. Khetani M, Marley J, Baker M, Albrecht E, Bedell G, Coster W, et al. Validity of the Participation and Environment Measure for Children and Youth (PEM-CY) for Health Impact Assessment (HIA) in sustainable development projects. Disability and Health Journal. 2014;7(2):226–35. pmid:24680052
- 78. Khetani MA. Validation of environmental content in the Young Children's Participation and Environment Measure. Arch Phys Med Rehabil. 2015;96(2):317–22. pmid:25486608
- 79. Khetani MA, Graham JE, Davies PL, Law MC, Simeonsson RJ. Psychometric properties of the Young Children's Participation and Environment Measure. Arch Phys Med Rehabil. 2015;96(2):307–16. pmid:25449189
- 80. Noreau L, Fougeyrollas P, Vincent C. the LIFE-H: Assessment of quality of social participation. Technology and Disability. 2002;14:113–8.
- 81. Gagnon C, Mathieu J, Noreau L. Measurement of participation in myotonic dystrophy: Reliability of the LIFE-H. Neuromuscul Disord. 2006;16(4):262–8. pmid:16545567
- 82. Lacourse E, Villeneuve M, Claes M. Theoretical structure of adolescent alienation: A multigroup confirmatory factor analysis. Adolescence. 2003;38(152):639–50. pmid:15053491
- 83. Lee CJ, Kim D. A comparative analysis of the validity of US state- and county-level social capital measures and their associations with population health. Social Indicators Research. 2013;111(1):307–26. pmid:25574069
- 84. Saeki S, Okazaki T, Hachisuka K. Concurrent validity of the Community Integration Questionnaire in patients with traumatic brain injury in Japan. J Rehabil Med. 2006;38(5):333–5. pmid:16931465
- 85. Lequerica AH, Chiaravalloti ND, Sander AM, Pappadis MR, Arango-Lasprilla JC, Hart T, et al. The Community Integration Questionnaire: Factor structure across racial/ethnic groups in persons with traumatic brain injury. J Head Trauma Rehabil. 2013;28(6):E14–E22. pmid:23249771
- 86. Levy O, Peiperl M, Bouquet C. Transnational social capital: A conceptualization and research instrument. International Journal of Cross Cultural Management. 2013;13(3):319–38.
- 87. Lorber N. The Ohio Social Acceptance Scale. Educational Research. 1970;12(3):240–3.
- 88. Lubben J, Blozik E, Gillmann G, Iliffe S, von Rentelen Kruse W, Beck J, et al. Performance of an abbreviated version of the Lubben Social Network Scale among three Eurpean community-dwelling older adult populations. The Gerontologist. 2006;46(4):503–13. pmid:16921004
- 89. El Hassan K. Validation of the Harter Pictorial Scale of Perceive Competence and Social Acceptance with Lebanese children. Soc Behav Pers. 1999;27(4):339–54.
- 90. Maselko J, Hughes C, Cheney R. Religious social capital: Its measurement and utility in the study of the social determinants of health. Soc Sci Med. 2011;73(5):759–67. pmid:21802182
- 91. Morais DB, Ogden AC. Initial development and validation of the Global Citizenship Scale. Journal of Studies in International Education. 2010;15(5):445–66.
- 92. Moreno-Jimenez MP, Rios Rodriguez ML, Martin MV. Construction and validation of the Community and Socio-political Participation Scale (SCAP). Span J Psychol. 2013;16(E42):1–8. pmid:23866238
- 93. Ostir GV, Granger CV, Black T, Roberts P, Burgos L, Martinkewiz P, et al. Preliminary results for the PAR-PRO: A measure of home and community participation. Arch Phys Med Rehabil. 2006;87(8):1043–51. pmid:16876548
- 94. Paiva PC, de Paiva HN, de Oliveira Filho PM, Lamounier JA, Ferreira e Ferreira E, Ferreira RC, et al. Development and validation of a social capital questionnaire for adolescent students (SCQ-AS). PLoS One. 2014;9(8):e103785. pmid:25093409
- 95. Perry M, Williams RL, Wallerstein N, Waitzkin H. Social capital and health care experiences among low-income individuals. Am J Public Health. 2008;98(2):330–6. pmid:18172158
- 96. Post MW, de Witte LP, Reichrath E, Verdonschot MM, Wijlhuizen GJ, Perenboom RJ. Development and validation of IMPACT-S, an ICF-based questionnaire to measure activities and participation. J Rehabil Med. 2008;40(8):620–7. pmid:19020695
- 97. Primack BA, Colditz JB, Cohen E, Switzer GE, Robinson GF, Seltzer DL, et al. Measurement of social capital among clinical research trainees. Clin Transl Sci. 2014;7(1):33–7. pmid:24118964
- 98. Sachson AD, Rappoport L, Sinnett ER. The Activity Record: A measure if social isolation-involvement. Psychol Rep. 1970;26:413–4. pmid:4395028
- 99. Salzer MS, Brusilovskiy E, Prvu-Bettger J, Kottsieper P. Measuring community participation of adults with psychiatric disabilities: Reliability of two modes of data collection. Rehabil Psychol. 2014;59(2):211–9. pmid:24611922
- 100. Hill RM, Rey Y, Marin CE, Sharp C, Green KL, Pettit JW. Evaluating the Interpersonal Needs Questionnaire: Comparison of the reliability, factor structure, and predictive validity across five versions. Suicide Life Threat Behav. 2015;45(3):302–14. pmid:25308815
- 101. Vafaei A, Pickett W, Alvarado BE. Neighbourhood environment factors and the occurrence of injuries in Canadian adolescents: A validation study and exploration of structural confounding. BMJ Open. 2014;4(7):e004919. pmid:24993755
- 102. Washington LA, Wilson S, Engel JM, Jensen MP. Development and preliminary evaluation of a pediatric measure of community integration: The Pediatric Community Participation Questionnaire (PCPQ). Rehabil Psychol. 2007;52(2):241–5.
- 103. Van Der Gaag M, Snijders TA. The Resource Generator: Social capital quantification with concrete items. Social Networks. 2005;27(1):1–29.
- 104. Webber MP, Huxley PJ. Measuring access to social capital: The validity and reliability of the Resource Generator-UK and its association with common mental disorder. Soc Sci Med. 2007;65(3):481–92. pmid:17462805
- 105. Zeldin S, Krauss SE, Collura J, Lucchesi M, Sulaiman AH. Conceptualizing and measuring youth-adult partnership in community programs: A cross national study. Am J Community Psychol. 2014;54(3–4):337–47. pmid:25216734
- 106. Wickham S, Shevlin M, Bentall RP. Development and validation of a measure of perceived relative deprivation in childhood. Pers Individ Dif. 2013;55(4):399–405.
- 107.
Zodikoff D. Development of a scale of social attitudes of fourth, fifth and sixth grade pupils 1967. Available from: http://files.eric.ed.gov/fulltext/ED041293.pdf.
- 108. Chang FH, Coster WJ. Conceptualizing the construct of participation in adults with disabilities. Arch Phys Med Rehabil. 2014;95(9):1791–8. pmid:24879964
- 109. Whiteneck GG, Dijkers MP, Heinemann AW, Bogner JA, Bushnik T, Cicerone KD, et al. Development of the Participation Assessment with Recombined Tools–Objective for use after traumatic brain injury. Arch Phys Med Rehabil. 2011;92(4):542–51. pmid:21367393
- 110. Narayan D, Cassidy MF. A dimensional approach to measuring social capital: Development and validation of a social capital inventory. Current Sociology. 2001;49(2):59–102.
- 111. Sherbourne CD, Stewart AL. The MOS social support survey. Soc Sci Med. 1991;32(6):705–14. pmid:2035047
- 112.
Parham LD, Ecker C. Sensory Processing Measure—Preschool (SPM-P). Torrance, CA: WPS; 2010.
- 113. Huxley PJ, Evans S, Madge S, Webber MP, Burchardt T, McDaid D, et al. Development of a social inclusion index to capture subjective and objective life domains (Phase II): Psychometric development study. Health Technol Assess. 2012;16(1). pmid:22260923
- 114. Stewart G, Sara G, Harris M, Waghorn G, Hall A, Sivarajasingam S, et al. A brief measure of vocational activity and community participation: Development and reliability of the Activity and Participation Questionnaire. Aust N Z J Psychiatry. 2010;44(3):258–66. pmid:20050719
- 115. Berry HL, Rodgers B, Dear KB. Preliminary development and validation of an Australian community participation questionnaire: types of participation and associations with distress in a coastal community. Soc Sci Med. 2007;64(8):1719–37. pmid:17241727.
- 116. Brisson D, Usher CL. Conceptualizing and measuring bonding social capital in low-income neighborhoods. Journal of Social Service Research. 2007;34(1):1–11.
- 117. Peterson NA, Speer PW, McMillan DW. Validation of A brief sense of community scale: Confirmation of the principal theory of sense of community. J Community Psychol. 2008;36(1):61–73.
- 118. Chang FH, Coster WJ, Salzer MS, Brusilovskiy E, Ni P, Jette AM. A multidimensional measure of participation for adults with serious mental illnesses. Disabil Rehabil. 2015;38(7):695–703. pmid:26079634
- 119. Baker PA. Measurement of community participation and use of leisure by service users with intellectual disabilities: The Guernsey Community Participation and Leisure Assessment. Journal of Applied Research in Intellectual Disabilities. 2000;13(3):169–85.
- 120. Williams D. On and off the 'net: Scales for social capital in an online era. Journal of Computer-Mediated Communication. 2006;11(2):593–628.
- 121. Marino-Francis F, Worrall-Davies A. Development and validation of a social inclusion questionnaire to evaluate the impact of attending a modernised mental health day service. Mental Health Review Journal. 2010;15(1):37–48.
- 122. Archuleta AJ, Miller CR. Validity evidence for the translated version of the Personal Social Capital Scale among people of Mexican descent. Journal of the Society for Social Work and Research. 2011;2(2):39–53.
- 123. Jason LA, Stevens EB, Ram D. Development of a three-factor psychological sense of community scale. J Community Psychol. 2015;43(8):973–85. pmid:27667867
- 124. Stevens EB, Jason LA, Ferrari JR. Measurement performace of the Sense of Community Index in substance abuse recovery communal housing. The Australian Community Psychologist. 2011;23(2):135–47. pmid:25083166
- 125. Magson NR, Craven RG, Bodkin-Andrews GH. Measuring social capital: The development of the Social Capital and Cohesion Scale and the association between social capital and mental health. Australian Journal of Educational & Developmental Psychology. 2014;14:202–16.
- 126. Onyx J, Bullen P. Measuring social capital in five communities. J Appl Behav Sci. 2000;36(1):23–42.
- 127. O'Brien MS, Burdsal CA, Molgaard CA. Further development of an Australian-based measure of social capital in a US sample. Soc Sci Med. 2004;59(6):1207–17. pmid:15210092
- 128. Looman WS. Development and testing of the Social Capital Scale for families of children with special health care needs. Res Nurs Health. 2006;29(4):325–36. pmid:16847911
- 129. Lee RM, Robbins SB. Measuring belongingness: The Social Connectedness and Social Assurance Scales. Journal of Counselling Psychology. 1995;42(2):232–41.
- 130. Lee RM, Draper M, Lee S. Social connectedness, dysfunctional interpersonal behaviours, psychological distress: Testing a mediator model. Journal of Counselling Psychology. 2001;48(3):310–8.
- 131. de Greef M, Segers M, Verté D. Development of the SIT, an instrument to evaluate the transfer effects of adult education programs for social inclusion. Studies in Educational Evaluation. 2010;36(1–2):42–61.
- 132. Mezey G, White S, Thachil A, Berg R, Kallumparam S, Nasiruddin O, et al. Development and preliminary validation of a measure of social inclusion for use in people with mental health problems: the SInQUE. Int J Soc Psychiatry. 2013;59(5):501–7. pmid:22511274
- 133. Secker J, Hacking S, Kent L, Shenton J, Spandler H. Development of a measure of social inclusion for arts and mental health project participants. Journal of Mental Health. 2009;18(1):65–72.
- 134. Wilson C, Secker J. Validation of the Social Inclusion Scale with students. Social Inclusion. 2015;3(4):52–62.
- 135. Densley K, Davidson S, Gunn JM. Evaluation of the Social Participation Questionnaire in adult patients with depressive symptoms using Rasch analysis. Qual Life Res. 2013;22(8):1987–97. pmid:23341174
- 136. Hacking S, Bates P. The Inclusion Web: A tool for person-centred planning and service evaluation. Mental Health Review Journal. 2008;12(2):4–15.
- 137. Lloyd C, Waghorn G, Best M, Gemmell S. Reliability of a composite measure of social inclusion for people with psychiatric disabilities. Aust Occup Ther J. 2008;55(1):47–56. pmid:20887433
- 138.
Paulus D, Vazire S. The self report method. In: Robins R, Fraley RC, editors. Handbook of Research Methods in Personality Psychology. New York, NY: Guilford Publications; 2007. p. 630–78.
- 139. Carifio L, Perla R. Resolving the 50 year debate around using and misusing Likert scales. Med Educ. 2008;42:1150–2. pmid:19120943
- 140. Long-Sutehall T, Sque M, Addington-Hall J. Secondary analysis of qualitative data: a valuable method for exploring sensitive issues with an elusive population? J Res Nurs. 2010.
- 141. Cobigo V, Ouellette-Kuntz H, Lysaght R, Martin L. Shifting our conceptualization of social inclusion. Stigma research and action. 2012;2(2).
- 142.
McClure P. Participation support for a more equitable society: Final report of the reference group on welfare reform. Retrieved November 20, 2008. 2000.
- 143. Wright N, Stickley T. Concepts of social inclusion, exclusion and mental health: A review of the international literature. J Psychiatr Ment Health Nurs. 2013;20(1):71–81. pmid:22369652
- 144.
World Health Organization. Social determinants of health 2008. Available from: http://www.who.int/social_determinants/thecommission/countrywork/within/socialparticipation/en/.
- 145. Kawachi I, Kennedy BP, Glass R. Social capital and self-rated health: a contextual analysis. Am J Public Health. 1999;89(8):1187–93. pmid:10432904
- 146. Araya R, Dunstan F, Playle R, Thomas H, Palmer S, Lewis G. Perceptions of social capital and the built environment and mental health. Soc Sci Med. 2006;62(12):3072–83. pmid:16439045
- 147. Halafoff A. Education about diverse religions and worldviews, social inclusion and countering extremism: Lessons for the Australian curriculum. Journal of Intercultural Studies. 2015;36(3):362–79.
- 148. Crisp BR. Belonging, connectedness and social exclusion. Journal of Social Inclusion. 2010;1(2):123–32.
- 149.
Evans M, Kelley J. Religion, Morality and Public Policy in International Perspective, 1984–2002: Federation Press; 2004.
- 150. Baumeister RF, Leary MR. The need to belong: desire for interpersonal attachments as a fundamental human motivation. Psychol Bull. 1995;117(3):497. pmid:7777651
- 151.
Lynch G. The new spirituality: An introduction to progressive belief in the twenty-first century: IB Tauris; 2007.
- 152. Isin EF. Citizenship in flux: The figure of the activist citizen. Subjectivity. 2009;29(1):367–88.
- 153. Boardman J. Work, employment and psychiatric disability. Advances in Psychiatric Treatment. 2003;9(5):327–34.
- 154.
Waddell G, Burton AK. Is work good for your health and well-being?: The Stationery Office; 2006.
- 155. Whiteford H, Cullen M, Baingana F. Social Capital and Mental Health. Promoting Mental Health. 2005.
- 156. Milton B, Attree P, French B, Povall S, Whitehead M, Popay J. The impact of community engagement on health and social outcomes: a systematic review. Community Development Journal. 2011:bsr043.
- 157.
United Nations. Report of the World Summit for Social Development New York, NY: United Nations, 1995 Contract No.: 96.IV.8.
- 158.
Bullen P, Onyx J. Measuring social capital in five communities in NSW: A practitioner's guide. Coogee, New South Wales: Management Alternatives Pty Limited; 1998.
- 159. Thill C. Listening for policy change: how the voices of disabled people shaped Australia’s National Disability Insurance Scheme. Disability & Society. 2015;30(1):15–28.
- 160. Ma WW, Chan A. Knowledge sharing and social media: Altruism, perceived online attachment motivation, and perceived online relationship commitment. Comput Human Behav. 2014;39:51–8.