Identification of effective HIV control strategies requires accurate measurement of sexual behaviour. This is the case even for strategies that, for example, seek to reduce HIV transmission through the application of vaccines or by reducing the co-factor effects of other sexually transmitted infections (STIs) and for strategies that seek to provide treatment for people living with HIV/AIDS. In each case, the intervention will be counterproductive unless it supports efforts to promote and maintain safer behaviour. Reliable and valid data on sexual behaviour and, most especially, on sexual behaviour change are urgently needed¹ but difficult to obtain.^2,3

Major difficulties include selective participation, poor recall,⁴ and incorrect response due to desire to conceal socially proscribed activities (social desirability bias).⁵ When valid data on sexual behaviour change are required, period effects can pose additional challenges.⁶ For example, sexual behaviour in men in Zimbabwe is associated with length of time since last pay day.⁷ Respondents interviewed at different times of the month in successive rounds of a survey may appear to have changed their behaviour when this has not, in fact, been the case. Many of these difficulties are compounded in the case of behaviours implicated in HIV transmission. Indeed, the very fact that an aspect of behaviour is thought to be “risky” in the context of an HIV epidemic immediately makes it more problematic to measure. As populations become more aware of the risks associated with particular behaviours, those that continue to practise them may become less likely to admit to so doing.

A number of methods are being developed to address these difficulties.^8,9,10,11,12 In our work on HIV interventions among relatively well educated—by African standards—but predominantly rural populations in Zimbabwe, we have sought to develop an appropriate low technology and inexpensive method for collecting data on behaviours associated with HIV risk. In an earlier paper, we described this method in detail and reported results from a large scale comparison of data collected using this informal confidential voting interview (ICVI) method with equivalent data collected using a more conventional face to face interview (FTFI) approach.¹³ We found that reports of multiple sexual partners over varying time periods appeared to be more complete when the ICVI method was applied. This was particularly so among the groups expected to be most reluctant to report multiple partnerships—for example, married women and people living in traditional subsistence farming areas. However, these results may have been due, at least in part, to difficulties in ensuring complete understanding of questions by individual respondents when the ICVI method is applied. For questions on number of sexual partnerships, most responses are either 0 or 1 and relatively few responses are 2, 3, or more. Thus, the proportion of the responses recorded as 2 or more that are misrecorded responses from people who have actually had 0 or 1 partnerships may be much greater than the proportion of responses recorded as 0 or 1 that are misrecorded responses from those who had 2 or more partnerships. If so, this could give the false impression that the ICVI method is yielding more complete reporting of behaviours that are relatively rare.

In the current paper, we report findings from a second round of the same population based survey. We compare the relation between behaviours reported using the ICVI and FTFI methods for the two rounds of the survey and examine factors that may contribute to observed changes in the effect of ICVI. Factors considered include temporal changes in the degree of response error, factors affecting statistical power to detect differences between methods such as reductions in the prevalence of risk behaviours and sample size, and changes in social desirability bias. In addition, we present substantive findings on patterns of association between aspects of sexual behaviour and HIV risk. By examining the plausibility or otherwise of observed associations, we can derive further indirect evidence for the reliability of the data obtained using the ICVI method.

METHODS

Informal confidential voting interviews

The ICVI approach constitutes a blend of FTFI and self-completion methods. The first part of the interview is conducted using an informal variant of the FTFI method and is used to establish motivation, to build a strong rapport between respondent and enumerator, and to sensitise the respondent to the non-prejudicial viewpoint of the enumerator and the study. Questions gradually progress from relatively straightforward matters to more private topics. Time is allowed for some informal discussion of the subjects raised. Information on the most sensitive personal subjects is sought in the second half of the interview using a secret voting procedure based on Pocket Chart Voting, a participatory rapid assessment technique used in community development projects.¹⁴

In the current study, the secret voting procedure was applied using lightweight portable wooden “voting” boxes. The boxes were divided into compartments, each of which had a narrow slot in the top labelled with a different colour. In each interview, a set of short strips of card coloured to match the slots in the voting box compartments was used as voting tokens. The tokens were all subdivided into sections, each prelabelled with a number. The colours of the voting strips and the numbers marked on the sections were cross referenced to questions in the questionnaire.

In each interview, time was set aside for the enumerator to provide a full explanation of the reasons for using the voting box and of the ways in which the design and associated procedure reduce possible embarrassment and ensure confidentiality. The enumerator then wrote the respondent’s survey identification number on the back of the voting strips before handing them one by one to the respondent in a predetermined sequence. The enumerator read out the corresponding questions from the questionnaires, one at a time, and the respondent indicated his or her answers in the appropriate sections on the voting strips. The boxes had lids that respondents used as screens to conceal what they were writing from the enumerators and were locked with keys held by the supervisors. Respondents could seek clarification on the meanings of questions from their enumerators but were discouraged from discussing the substance of their responses. After completing each strip, a respondent would place it in the box through the slot labelled with the same colour and mix it with those completed by earlier participants. Thus, the information provided was kept secret from the enumerators. The field supervisors—who did not know the identities of individual respondents—subsequently matched the voting responses to the appropriate questionnaires (using the survey identification numbers) and questions (using the colours of the voting strips and the question numbers marked on the sections of each strip).

Study design

The application of the ICVI method reported upon here was embedded in the first two rounds of a longitudinal population based HIV/AIDS survey in 12 rural locations in Manicaland, Zimbabwe (Research Council of Zimbabwe (reference 02187); Applied and Qualitative Research Ethics Committee, Oxford, UK (reference N97.039)). The study sites comprised four roadside trading centres, four commercial farming estates, and four subsistence farming areas. The first round was conducted in a phased manner between July 1998 and January 2000 and the second round was completed three years later in each site. At each survey round, men and women aged 15–54 years were treated as eligible for the current study.

Illiterate respondents and a sample of literate respondents were interviewed using the FTFI method. In round one, two enumerators—one male and one female—were instructed to use the FTFI method in each study site so that any difference in reporting of socially proscribed behaviours could be measured. These enumerators used the FTFI method in all interviews in the households they visited, so that the selection process was random, and were rotated as the survey team moved from site to site to minimise interviewer effects. All other respondents were interviewed using the ICVI method. In round two, respondents who had been interviewed in the first round were interviewed again using the same method to ensure consistency. Residents who had previously been too young to be enrolled, or who had moved to the study areas since round one, were invited to participate in round two. All such new respondents were interviewed using the ICVI method unless they were not sufficiently literate or declined to do so.

The ICVI method was evaluated by comparing the responses given by survey participants in the ICVI and FTFI groups to a short series of questions on sexual experience—questions on numbers of partners within various different time frames (lifetime, last 12 months, last month, and current) and details of partnerships in the last month (sex acts, condom use, and characteristics of partners). Questions were constructed so as to elicit simple yes/no/don’t know or numerical responses and thereby to minimise non-participation and response errors. Where respondents had writing difficulties, simple picture codes were used to illustrate standardised symbols for yes (tick), no (cross), and so on.

In the questionnaires, the sexual behaviour questions followed a lengthy section on sociodemographic characteristics and were asked in order of increasing sensitivity. The survey introduction and the early part of the interview process were used to sensitise respondents in ways intended to encourage them to be more open about the later questions on sexual behaviour. The questions on age at first sex (and thereby on experience of sexual debut) and time since last sexual intercourse were asked using the FTFI method, even in interviews where the ICVI method was used to obtain the data on partnership details. In round two, additional questions on numbers of lifetime regular and non-regular sexual partnerships were included in the section where the alternative methods were applied.

In each round, interviews (ICVI and FTFI) were conducted by social science graduates, most of whom were qualified social workers skilled at establishing relationships of trust with clients. Enumerators familiar with the local language and customs were selected and matched for sex with respondents. They were trained to make culturally appropriate introductions and to explain the importance of the research objectives, the possibility of incorrect decisions being taken if unreliable data were provided, and the measures being taken to maintain confidentiality of information. Enumerators were trained to adopt an informal/flexible approach (so that their rapport with respondents could develop as interviews progressed) and to adopt a non-prejudicial approach. During the survey, project supervisors carried out routine quality control checks on the conduct of interviews and on all completed questionnaires.

Comparison of ICVI and FTFI methods

The statistical significance of observed differences in reporting between the ICVI and FTFI methods in each survey round was assessed using standard statistical χ² tests. Age, marital status, education, socioeconomic environment, and recent in-migration adjusted odds ratios and 95% confidence intervals for reporting a range of risk behaviours for HIV transmission, by interview method, were calculated using logistic regression analysis in STATA version 7 (Stata Corporation, College Station, TX, USA). Sample sizes and absolute levels and changes in the proportions of men and women reporting each type of behaviour were examined to see whether these might account for changes in the effects of ICVI. Changes in the relative completeness and internal consistency of the data collected using the two methods were calculated to facilitate assessment of whether such changes might contribute to changes in the observed effects of ICVI. Adjusted odds ratios for the effect of ICVI were recalculated after omitting cases with internal inconsistencies in reporting. Illiterate respondents and polygamous males were excluded from all analyses—the latter because these men generally belonged to churches in which having multiple wives is not a socially proscribed behaviour.

Results from previous analyses of data from round one of the survey on interrelations between HIV infection and behaviour—reported primarily using the ICVI method—were reviewed for plausibility.

RESULTS

Study participants

In rounds 1 and 2 of the survey, respectively, 93% (3375 of 3638) and 93% (2366 of 2541) of sexually experienced men and 91% (4009 of 4420) and 91% (3325 of 3650) of sexually experienced women were sufficiently literate to be interviewed by the ICVI method. Omitted from the analysis were 5.2% and 3.7% of men in rounds 1 and 2, respectively, who reported more than one current wife. Of the remainder, 77.4% and 87.1% were interviewed using the ICVI method in rounds 1 and 2, respectively. The equivalent figures for women were 73.1% and 78.3%. In each case, the higher percentages interviewed using the ICVI method in round 2 reflected the decision to interview all new literate respondents in this way.

In round 1, the average ages of ICVI and FTFI interview respondents were similar—28 years for men and 30 years for women. In round 2, ICVI interview respondents were slightly younger—29 years versus 32 years for men and 32 years versus 35 years for women—because of the use of the ICVI method for all new previously underage respondents. In both rounds, men interviewed by ICVI were more likely to be living on estates (43% v 28%) and less likely to be living in roadside trading centres (29% v 47%) than men interviewed in FTFIs. Relatively more of the female ICVI interviewees lived in rural villages (39% v 33%) and fewer lived in roadside trading centres (34% v 41%). These differences arose through inconsistencies in the deployment of the field staff assigned to carry out FTFIs in round 1. Differences in marital status and education levels were small in round 1. However, in round 2, relatively fewer male ICVI interview respondents were widowed or divorced (5% v 10%) and comparatively more of their female counterparts had received secondary school education (55% v 43%).

In round 2, 97% (680 of 701) of men and 99% (1105 of 1112) of women who reported not having started sex were sufficiently literate to use the ICVI method. Of these, 95% of both men and women were interviewed in this way. Of the remainder, 50% (42/84) had been interviewed by FTFI at round 1.

Comparison of reported behaviour by data collection method and time period

Odds ratios (ORs) adjusted for sex, age, marital status, socioeconomic location, education level, recent in-migration, and survey round for reporting a range of risk behaviours for HIV infection for men and women interviewed by ICVI versus FTFI method are shown in figure 1. Survey respondents were more likely to report each of the risk behaviours examined if they were interviewed using the ICVI method, although the difference was not statistically significant for unprotected sex with a casual partner given the relatively small sample size (n = 679).

However, there was evidence for differences in the effect of ICVI by sex and between the two survey rounds (table 1). In round 1 of the survey, men were consistently more likely to report risk behaviours when interviewed by the ICVI method. The results for all the behaviours shown were statistically significant except in the case of unprotected sex with a recent casual partner for which the sample size is small. In round 2, men who were interviewed by ICVI method were also more likely to report risk behaviour for all indicators. However, the magnitude of the OR was smaller than in round 1 in three out of five instances and—perhaps, in part, due to smaller sample sizes—the difference in reporting between methods was statistically significant for only one indicator (new partner in last year) and borderline significant for two others.

For women, respondents interviewed in round 1 were more likely to report a new sexual partner in the last year, more than one partner in the past month and more than one current partner if interviewed by ICVI method. In round 2, the difference in reporting remains for having a new sexual partner in the last year but disappears for having multiple partners in the past month and currently. In contrast to round 1, in round 2, there are suggestions in the data that women interviewed by ICVI method are more likely to report that their last or previous sexual partner in the past month was a casual partner and to report unprotected sex with a recent casual partner than those interviewed by FTFI.

Levels and trends in the frequency of HIV associated risk behaviours

Differences in reporting between data collection methods can be more difficult to detect if the behaviours measured are rare. Similarly, differences can become more difficult to detect over time if behaviours are becoming less common. This possibility is explored in table 2. Of the behaviours examined, having a new sexual partner in the past year and having had unprotected sex with a recent casual partner (for those having had such a partner) are the most commonly reported for both men and women at each survey round and for each data collection method. In addition, many men report that one of their two most recent sexual partners was a casual partner.

The ICVI results indicate that most of the behaviours examined are significantly less commonly reported in round 2. The exceptions are reports of the last or previous partner in the past month being a casual partner for women and reports of unprotected sex with a casual partner for both sexes. The latter also appear to be becoming less common after adjustment is made for confounding variables but the differences are not statistically significant given the much smaller sample sizes available. A similar trend is evident in the FTFI data although some differences are not statistically significant and the trend in the less common behaviours is inconsistent for women.

Item response rates and internal consistency

A further possibility is that the differences in reporting by data collection method seen between rounds 1 and 2 of the survey could reflect changes in the standard of data collection between survey rounds. In particular, respondents may have made more errors in completing the voting tokens in round 1 than in round 2. If these errors are biased towards greater apparent reporting of relatively uncommon risk behaviours, this could lead to an apparent reduction in these risk behaviours over time. This possibility is explored in table 3.

In the top panel of the table, we examine the levels of missing data by survey round in order to assess possible changes in the overall quality of reporting. The results suggest that there may have been a slight improvement in the completeness of the ICVI data between survey rounds. However, omissions—either through interviewer carelessness or respondent refusal to answer—remain more common than in the FTFIs and the overall level of missing data is quite low.

In the bottom panel of table 3, results from the internal consistency checks are presented. These suggest there was little change in the level of mistakes made by respondents using the ICVI method between the two survey rounds. The internal consistency of information obtained from respondents interviewed by FTFI remained high except that a greater number of mistakes made by respondents on the questions on time since last sexual intercourse and/or number of sexual partners in the past month went undetected by the field staff in round 2 than had been the case in round 1.

In round 2, it was also possible to examine the consistency of reports on sexual debut by comparing responses to the initial question on age at first sex (always asked by FTFI) with responses to questions on numbers of regular (duration of 12 months or more) and non-regular sexual partnerships (asked by ICVI or FTFI in accordance with the interview method selected for the respondent in question). The results provide no evidence for increased error—or, for that matter, reduced social desirability bias—associated with use of the ICVI method, although it should be noted that the sample sizes for the FTFI method were small. Finally, for women, it was possible to compare responses for questions on experience of sexual debut and experience of a pregnancy in the past three years. These, again, indicate a high level of internal consistency for both methods. The overall level of self-reported pregnancy among women who stated that they had not had sex in round 2 (2.7%) is similar to that found among women tested using a commercial pregnancy kit in round 1 (1.6%).

Table 4 shows, for the two survey rounds combined, the effect of removing cases with internally inconsistent reports on the adjusted odds ratios for reporting the selected risk behaviours when the ICVI method is used rather than FTFIs. All the behaviours remain more commonly reported when the ICVI method is used. However, the degree of increased reporting is reduced in most instances and the difference is reduced to borderline statistical significance for multiple current sexual partners and the last or previous sexual partner being reported as a casual partner.

Substantive results on HIV and associated sexual risk behaviours

Consistency of results with previously hypothesised or observed relations between HIV infection and sexual behaviour would support the reliability of the data obtained primarily through use of the ICVI method. Figure 2 shows the results on the relations between reported number of lifetime sexual partners and HIV prevalence for men and for women obtained from the data collected in round 1 of the survey in Manicaland. As expected,¹⁵ the results show a steady increase in the adjusted OR for HIV infection (relative to the chance of infection given one lifetime partner) with increasing number of lifetime sexual partners for both men and women. The higher OR for women with 10 or more lifetime partners than for men in the same group may reflect a larger average number of sexual partners for the women in this open ended group. Similarly, previously predicted results on the relationship between age difference in sexual unions and exposure to HIV infection^16,17 have been obtained from data collected in the study.¹⁸

DISCUSSION

Social desirability bias poses difficulties for the accurate measurement of indicators of sexual behaviour and sexual behaviour change^8,19 needed for the evaluation of HIV/AIDS programmes. Intuitively, therefore, more secret methods of data collection should reduce bias and provide more reliable information. Findings from studies in both more and less developed countries indicate that this can be the case.^9,20,21 However, most methods are expensive and require technological infrastructure that is absent from the rural areas in which most sub-Saharan African populations live.

In an earlier paper, we described a secret voting procedure (ICVI) for collecting personal information. We presented findings from the baseline survey of a longitudinal population based study in predominantly rural areas that suggested that this inexpensive and practical method could also increase disclosure of proscribed personal behaviour.¹³ In the current paper, we have presented results from the second round of this study that suggest that this greater propensity to disclose behaviours associated with HIV transmission may be reducing over time. This does not appear to be due to temporal changes in the higher level of undetectable errors associated with the method.

To some degree, the changing effect of the ICVI method reflects overall declines in reports of risk behaviour. However, it is difficult to know whether such declines follow from genuine reductions in risk behaviour (due to some combination of behaviour change and selective HIV associated mortality) or from temporal changes in social desirability bias. The latter seem likely to occur as HIV/AIDS programmes are sustained and intensified but the scale and net direction of the changes is unclear. People may feel more reluctant to admit to behaviours that they know they have been told repeatedly can carry a high risk of HIV transmission—especially to interviewers in an HIV/AIDS survey. The consequences of breaches of confidentiality were already severe in many cases (for example, for married women), but can only have become more serious as knowledge and awareness about HIV/AIDS increases. At the same time, HIV/AIDS programmes have led to more open discussion about sexual behaviour which may, to some degree, have offset reluctance to discuss sexual experiences. The net effect may also vary by sex and between behaviour indicators. Thus, for example, for a man, having a new sexual partner in the past year may be considered more acceptable than having more than one partner in the past month. For behaviours measured over longer time periods, recall bias probably obscures the relation between interview method and social desirability bias. We have therefore restricted our primary analyses to measures of recent sexual behaviour.

How these changes have affected the relative propensity of our study respondents to disclose HIV associated risk behaviour in ICVI and FTFI interviews is difficult to assess. Respondents interviewed in ICVI interviews may have been disproportionately less likely (relative to FTFI respondents) to report proscribed sexual experiences at the time round 2 was conducted than at round 1 but this does not seem very plausible. Another possibility is that greater familiarity with and more positive attitudes towards the survey team and growing confidence in the confidentiality of information provided for the research could have led to a relative increase in disclosure of socially proscribed behaviours in the FTFIs.

On balance, and taking into consideration the high mortality and declines in cases of STIs treated at hospitals and clinics in the study areas, it seems likely that at least some genuine reduction in individual risk behaviours for HIV transmission has occurred. The resultant lower overall levels of risk behaviours taken in combination with the smaller sample sizes obtained in round 2 could explain why we could still detect differences for behaviours that remained common (for example, having had a new partner in the last year or one of up to two partners in the past month being a casual partner) but were unable to do so for those that had become quite rare (for example, multiple sexual partners in the past month and currently). For rare behaviours, measurement of differences in reporting between methods can require unfeasibly large sample sizes but may cease to be meaningful.

Adaptations of the ICVI method are currently being applied elsewhere (for example, table top screens in Peru and voting envelopes and bags in South Africa). Other related methods have also been developed that are thought to reduce social desirability bias either by providing greater confidentiality (for example, by conducting interviews in specially constructed huts²²) or through more in depth interviews in which emphasis is placed on building rapport and where more extensive internal consistency checks can be conducted during the interviews.²³ However, very few formal evaluations of applications of such methods in developing country settings have been published. In Karnataka, India, a polling booth method was compared with standard household interviews:²⁴ 13% (72 of 566) versus 2% (41 of 2228) of men reported experience of commercial sex in their lifetime and 32% (90 of 282) versus 19% (264 of 1422) of men and 8% (22 of 272) versus 2% (49 of 2102) of women reported having had extramarital sex.

Further evaluation of low technology methods for collecting data on sexual behaviour should be an important research priority and should include studies that employ alternative designs. The latter might include comparisons of information provided by couples or by respondents asked the same questions in two or more different ways, the order of administration being selected at random. In longitudinal studies, data on the incidence of HIV, HSV-2,²⁵ or other biomarkers might be compared with reports of sexual behaviour between survey rounds collected using different data collection methods. Development and evaluation of methods for collecting reliable data on sexual behaviour from young people must be a particular priority.

CONTRIBUTORS

SG designed the study, conducted the data analysis, and drafted the manuscript. CN, MM, and PM contributed to the study design, organised the data collection and contributed to the write up. CM organised the data processing and assisted with data analysis. PW assisted with the data analysis and preparation of the manuscript.