A major concern when studying HIV risk behaviours and practices in men who have sex with men (MSM) is the issue of stigma related to both homosexuality and HIV infection. Research has demonstrated homosexuality related stigma and its consequences.1 2 A number of studies conducted in different countries has documented that stigma is negatively associated with social support, self-esteem and disclosure of HIV positive status.3 4 Although HIV continues to spread among MSM in China and stigma against homosexuality has been documented,5 few empirical studies on HIV and homosexuality related stigma have been conducted in this hidden population. One of the obstacles in conducting such studies in China is the lack of reliable and valid stigma measurement scales. An extensive literature search yielded one qualitative study and one quantitative study on stigma against homosexuality.6 7 In the quantitative study, scales measuring homosexuality related stigma were assessed among Chinese MSM. Reliability of the reported scales of perceived stigma and enacted stigma was fairly low with a Cronbach’s alpha of 0.45 and 0.69, respectively. Based on the authors’ findings, further research on refining perceived and other stigma constructs for Chinese MSM is required.7

Stigma has been described as a quality that “significantly discredits” an individual in the eyes of others.8 It is generally conceptualised as “an attribute used to separate affected individuals from the normalised social order”.9 The separation implies a process of devaluation and discrimination against the stigmatised group. Stigma is socially constructed and reinforced by social inequality.10 Chinese collectivist culture may nourish stigma11 as it focuses on a social pattern that is tightly woven between people who consider themselves to be closely linked with one another.12 Members of collectivist cultures subordinate individual interests to that of the group or collective.13 14 As a result, the Chinese pay more attention than other cultures to how they feel they would be evaluated or viewed by others. It has been reported that Chinese societies display an unusually high degree of stigma and that stigma associated with mental illness is more severe than in the West.11

According to the theories of stigmatisation,15 16 stigma has at least two facets: public stigma and self stigma. Public stigma is the attitudes or reaction that the general population has to people who have a particular undesirable attribute, such as homosexuality or HIV infection. Self stigma, however, refers to the fear of societal attitudes and potential discrimination perceived by people who have such an undesirable attribute.16 17 Individuals who are homosexual in a society that widely endorses stigmatising ideas internalise these ideas and believe that they should be blamed and stigmatised and/or that they should conceal their homosexuality in order to avoid public stigma. Based on these theories, two scales were designed to measure homosexuality related stigma: one measuring public homosexual stigma perceived by MSM (public homosexual stigma) and the other measuring internalised homosexual stigma (self homosexual stigma). A final scale was designed to measure MSM’s stigmatising attitudes or reaction to people with HIV and their family members (public HIV stigma). This study examines the reliability of these scales and assesses findings from these scales in the context of the Chinese collectivist culture.

METHODS

Study site and subjects

We conducted a cross-sectional study of social network factors associated with HIV-related risks among MSM in Shenzhen between November and December 2007. Shenzhen, the first special economic region in China, is located along the southern coast of China bordering Hong Kong and Guangzhou. The prevalence of HIV infection is between 2 and 3% and between 9 and 19% for syphilis.18^–20 It is estimated that there are 60 000 MSM in the city of Shenzhen.21 A man was eligible for this study if he (1) was between 18–45 years old, (2) reported having engaged in anal intercourse with one or more men in the past year and (3) had lived in Shenzhen for more than 3 months at the time of the interview.

Respondent-driven sampling and interview

Respondent-driven sampling (RDS) was used to recruit MSM.22 A diverse group of 12 seeds heterogeneous in age (18–30 and 31–45 years old), MSM congregation venue (sauna, bar and public park) and engagement in commercial sex were selected and each were given three uniquely coded coupons to refer up to three peers. The planned sample size of 350 eligible MSM was obtained after four to five waves of RDS recruitment.

Eligible MSM recruited by the 12 seeds at wave 1 and new recruits at subsequent waves received a face-to-face anonymous interview in private interview rooms. All interviewers received training in interviewing techniques, developing rapport, ensuring confidentiality and answering questions raised by subjects.

Measures

Items measuring public homosexual stigma and self homosexual stigma were adapted from a set of scales created for a study designed to examine the association of racial and homosexual stigma with risk behaviours among Latino MSM.23 Bruce reported a 10-item scale of perceived homosexual stigma (public stigma) and a 10-item scale of internalised homosexual stigma (self stigma). In addition, we adapted three items from Preston et al’s scales.3 Items measuring HIV public stigma were adapted from our previous studies.24 25

Measurement items were initially drafted in English and then translated into Chinese by three research members who were fluent in both languages. The Chinese version of these items was distributed to research team members who reviewed and modified wording to make it appropriate for the Chinese context. Next, a plot study and a single focus-group discussion were conducted among 10 MSM, in which MSM first received an individual face-to-face interview and then participated in the group discussion. Research members and MSM reviewed and discussed each questionnaire item and possible answers, focusing on whether items were appropriate in their language and whether the MSM felt they could easily understand the questions and the possible answers. Items were refined and finalised based on the results from the focus group.

Data analysis

Exploratory factor analysis (EFA) was first performed to identify the latent factor structures underlying the survey data.26 27 The criteria used to determine the number of “meaningful” factors to retain were the eigenvalue-one criterion, the scree test and the proportion of variance extracted. For the first method, also known as the Kaiser criterion, factors with an eigenvalue >1 were retained and interpreted. For the scree test, the eigenvalues associated with each factor were plotted and a “break” was determined between the factors with relatively large eigenvalues and those with small eigenvalues. The factors that appeared before the break were assumed to be meaningful and were retained for rotation; those factors appearing after the break were assumed to be unimportant and were dropped. The third criterion was the cumulative proportion of variance extracted by successive factors. The factor extraction process was terminated when a threshold for maximum variance extracted (for example, 90%) had been achieved.

After the initial factors were extracted, a varimax rotation was performed to determine what was measured by each of the retained factors. In this study, loading values >0.35 were considered “meaningful loadings”. That is, an item was said to load on a given factor if the factor loading was ⩾0.35 for that factor and was <0.35 for all others. The EFA process was done using “proc factor” in SAS (version 9.1).

Confirmatory factor analysis (CFA) was used to assess the extent to which the hypothesised organisation of a set of identified factors fit the data. CFA is frequently used when the researchers have some knowledge about the underlying structure of the construct under investigation. It is also used to test the utility of the underlying dimensions of a construct identified through EFA.26 Goodness of fit in CFA was assessed with the χ2/degree ratio (<2), the comparative fit index (CFI⩾0.9), the root mean square error of approximation (RMSEA⩽0.06) and the standardised root mean square residual (SRMR⩽0.06). Mplus (version 5) was used to perform this analysis.

Scale reliability was assessed with Cronbach’s coefficient alpha. In order to examine the stability of scale reliability, the total sample was randomly split in half. Cronbach’s coefficient alpha of each scale was estimated for the first half sample, the second half sample and for the sample as a whole.

RESULTS

In total, 351 eligible subjects were recruited and interviewed. The mean age was 27 years old (SD = 6, range 18–44 years old). Of the subjects, 65% received a high-school or above education and about 4% received only a primary school education or no education. Thirty-nine per cent of MSM worked in entertainment venues, such as bars, saunas, night clubs or dance halls. More then two-thirds (78%) were single.

Using the three criteria for factor extraction and guided by the theoretical rationale, three factors were found and retained in the exploratory factor analyses. Using the eigenvalue-one criterion, the first three factors were retained as their eigenvalues were >1 (table 1). The remaining factors had an eigenvalue <1, with the eigenvalue of the fourth factor equal to 0.46 and for the fifth 0.43. With the scree test, the eigenvalues associated with the first 10 (of 25) factors were plotted (fig 1). There was a relatively large break between factors 1 and 2, 2 and 3, and another large break between 3 and 4. These results suggested that factors 1, 2 and 3 be retained as only these factors appear before the last big break.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1 The scree test: the first 10 factors.

The last criterion in deciding the number of factors was the proportion of variance in the data set. Table 1 lists the variance extracted by the first three factors. The cumulative variance explained by these three factors was 98.5%, with only 1.5% explained by other factors. This analysis supported the inclusion of the first three factors.

Table 2 presents questionnaire items clustered in each of the three retained factors and their corresponding loadings. An item was considered to load on a factor if the given factor loading was ⩾0.35 for that factor and was <0.35 for all others. A total of 10 items loaded on the first factor, which, according to the theories of stigmatisation, was labelled public homosexual stigma. Seven items loaded on the second factor, which was public HIV stigma. The remaining eight items loaded on the third factor, which measures self homosexual stigma.

To further validate the findings from the exploratory factor analysis, confirmatory factor analysis was performed to assess the three latent constructs measured by the questionnaire items presented in table 2. This CFA indicated acceptable fit to the data (the χ2/degree ratio = 1.84, CFI = 0.91, RMSEA = 0.05 and SRMR = 0.05).

Scale reliability was assessed using the Cronbach’s coefficient alpha. There were no substantial differences on alpha estimated from the first half sample, the second half sample and the whole sample, which indicated good stability. The Cronbach’s alpha was 0.85, 0.78 and 0.79 for public homosexual stigma, self homosexual stigma and public HIV stigma, respectively, for the full sample. These three scales were statistically significantly correlated (table 3).

DISCUSSION

The Chinese version of HIV and homosexuality related stigma scales designed in this study demonstrated robust evidence for reliability. Derived from the theories of stigmatisation,8 15 16 these scales assess a broad range of stigmatising perceptions, beliefs and attitudes, which include prejudice, devaluation, avoidance, delegitimisation and discrimination directed at homosexual people or people living with HIV/AIDS. The three measures are brief and suitable for use in developing countries where less time-consuming measurement scales are needed to measure the level of stigma. These scales provide data to inform, design and evaluate stigma interventions.

The comprehensive factor analyses demonstrated sound psychometric properties of these scales, indicating their appropriateness and usefulness. The selection of the three latent factors was supported by both statistical properties and stigma theory. Because there were no items with cross-loadings onto multiple latent factors, the distinctness of the constructs that the scales measure appears to be supported. The extent to which the hypothesised factoring structure of a set of measured factors fit the data was examined in confirmatory factor analysis: a theoretically driven approach that allows formal statistical testing of the scale structure. The fit indices in CFA indicate adequate support for the hypothesised three-factor structure of associations. Assessing scale reliability with Cronbach’s coefficient alpha is an important step in conducting questionnaire research. For these data, scale reliability was examined repeatedly in split samples and in the whole sample. For all samples, the Cronbach’s alpha for each of the three factors was fairly high (0.77–0.87). As indicated in psychometric theory,26 if items load on each retained factor, and if the factor pattern displays simple structures (no cross-loading items), chances are good that the resulting scales will demonstrate adequate internal consistency. Although the validity of the scales could not be directly assessed, the distinctive items used to measure the three different homosexuality and HIV related stigmas were embedded in the theories of stigmatisation. The significant correlations among the three scales indicate good construct validity.

Several limitations need to be noted. First, as the study was conducted in one city, the results may not be generalisable to other areas in China. Validation and refinement may be needed in a large scale study. In addition, because seeds were recruited in venues where there were high levels of sexual activity, the sample could be biased towards MSM in high sexual risk situations. Further studies are needed to sample MSM in stable relationships and to evaluate the representativeness of samples. Second, enacted homosexual stigma (that MSM actually experienced) was not designed in this study. Neilands and colleagues have reported a scale of enacted homosexual stigma that exhibited satisfactory internal reliability (0.69) and was associated with HIV sexual risk behaviour among Chinese MSM.7 Third, although these scales were retained on conceptual grounds, their validity should be further investigated in additional empirical studies.

Despite these limitations, the Chinese version of HIV and homosexuality related stigma scales designed and assessed in this study can be a valuable resource in the assessment of stigma and its consequences among MSM in China and countries with similar settings.