Publicly funded sexually transmitted disease clinics are the primary locale for public health prevention and control of sexually transmitted infections (STIs) in the United States.¹ STI clinics have an important role in assessing disease trends among high risk groups and in providing behavioural interventions for their patients. Evaluating clinic based interventions, however, is challenging because it requires measuring a range of sensitive, stigmatised, and illegal behaviours, and these self reported measures are vulnerable to non-trivial levels of reporting bias.

In 1992 it was predicted that audio computer assisted survey interviewing (ACASI) would improve the validity and quality of sensitive self reported measures made in population based surveys.² Research conducted during the past decade generally supports this prediction in comparing ACASI with traditional interviewer administered questionnaires (IAQs) or self administered paper and pencil interviews.^3,4 Studies conducted in the general population demonstrate greater consistency in responses and fewer skipped questions than paper and pencil interviews.⁵ Furthermore, the increased privacy afforded by ACASI enhances reporting of sensitive and illegal behaviours associated with HIV transmission, and decreases over-reporting of desirable or prescribed behaviours, such as consistent condom use.

Less is known about the impact of interview mode on the quality of data collected in more focused populations. The primary objective of the Druid Project was to assess differences in self reported risk behaviours by interview mode (ACASI versus IAQ) among adolescent and adult patients attending an inner city STI clinic. We also evaluated variation in reporting by interview mode across various patient subpopulations. We hypothesised that new patients, or those attending an STI clinic for the first time, would be more likely to report risky behaviours in ACASI than in a less private interview mode (IAQ)—that is, their reporting patterns would be similar to that of the general population. However, measures of sexual risk might be less affected by interview mode among “socialised” or repeat patients who are accustomed to being questioned about their sexual history and other risk taking during clinic visits. This social desirability effect may also influence repeat patients to over-report protective behaviours prescribed by clinic personnel, such as consistent condom use, in IAQ relative to ACASI. The Druid Project provides an empirical basis for assessing the quality of self reported data on HIV related risk behaviours in a population known to be at greater risk of HIV in a context where reporting of detailed information on sexual history and risk behaviours is expected.

BACKGROUND

Both audio-CASI and video-CASI (also referred to as text-CASI) have been found to improve reporting of sensitive behaviours among adolescents (but not adults) when compared to paper self administered questionnaires (paper SAQs). Among a national sample of US adolescent males, respondents assigned to the ACASI mode were three times as likely as those completing paper SAQs to report sex with a prostitute or use of drugs or alcohol during their last heterosexual intercourse, and 3.8 times as likely to report any same sex sexual contact, yielding an estimate more consistent with retrospective reports made by adult males regarding their sexual experiences before adulthood.²

Wright et al⁶ and Aquilino⁷ have reported similar results comparing video-CASI to paper SAQs for the measurement of drug use among adolescents, and they report that the effect is moderated by the presence or intrusion of parents during the interview. Little or no difference has been found between reporting of drug use⁶ and sexual behaviour⁸ between video-CASI and paper SAQs among adults.

Within population based surveys, comparison of ACASI with IAQ has yielded consistent results. A methodological experiment conducted among a sample of adults aged 18–45 years residing in Cook County, Illinois, for example, found that survey respondents assigned to ACASI were 4.2 times as likely as subjects assigned to the IAQ condition to report ever having anal sex and 1.7 times as likely to report cocaine use in the past month. For every sex partner reported by IAQ respondents, respondents assigned to ACASI reported, on average, 1.3 lifetime partners.³

A few empirical studies have explored the effects of ACASI on the quality of data collected among subpopulations at high risk of STI. Des Jarlais and colleagues⁹ compared responses to a range of risk behaviours among 1481 injection drug users attending syringe exchange programmes. ACASI respondents were more likely to report injection with rented or bought used equipment, renting or selling used works, and paying partners for sex than participants assigned to IAQ (odds ratios = 2.1 to 2.3). Participants in ACASI were 0.5 times as likely to report always using condoms with casual sex partners and 0.6 times as likely to always use alcohol wipes.

Like participants in population based surveys, concerns about self presentation may lead to under-reporting of HIV risk within certain settings, such as STI clinics.¹⁰ In addition to the usual pressures to under-report sensitive behaviours, clinic patients may be concerned that their responses affect how they are treated by clinicians, as well as the services they are offered. The context of the assessment, therefore, may influence perceptions of question sensitivity and the formation of “appropriate” or socially desirable responses.

The effectiveness of ACASI within an institutional environment is not clear. Jennings and colleagues,¹¹ for example, compared ACASI with IAQ in a survey of HIV risk behaviour among adolescents in a substance abuse treatment programme. Interviewer administration was found to significantly increase reports of alcohol and drug use and sexual behaviour, relative to ACASI. The authors concluded that within an environment in which a suitable level of rapport had been established between the interviewer and respondent, reporting of sensitive behaviours may be enhanced by modes that increase rather than decrease interpersonal interaction. These findings raise questions about the potential utility of ACASI within the STI clinic environment as well.

METHODS

The Druid Project

Between July 2000 and May 2002, 2633 patients attending an urban STI clinic were approached by two study interviewers to assess eligibility for participation in the project. Patients were eligible if they were between 15 and 39 years of age, non-HIV positive, and English speaking. HIV positive patients were excluded because they are likely to have undergone extensive questioning about their behaviours in the past, potentially confounding the effect of patient status (new versus repeat) on self reporting of sensitive behaviours. Written informed consent was obtained from all study subjects before administration of the interview. Subjects received a $30 food coupon for their participation. The study protocol was approved by the institutional review boards of the Research Triangle Institute, Baltimore City Health Department, and the Johns Hopkins medical institutions.

Experimental design

Following consent, subjects were assigned to either ACASI or IAQ. Our sampling strategy required recruitment of similar numbers of patients in each of four strata: female new patients, male new patients, female repeat patients, and male repeat patients. We defined new patients as those who had never sought care at either the study facility or any other STI clinic. Patients were classified as repeat patients if they had sought care in the past at any STI clinic. Interviews were conducted before the patient was seen by a clinician in a private examination room. Interviewer and patient gender were matched.

Survey instrument

Participants in the study completed a detailed behavioural survey on sexual risk practices, previous STIs and symptoms, partner characteristics, drug and alcohol use, and condom use. The interview instrument contained 80 “core” questions (128 total questions if no skip patterns were executed, plus several gender specific questions, excluding female douching behaviours, male circumcision). The questionnaire was adapted from survey instruments implemented in previous behavioural surveys^12,13 and was designed for administration in approximately 20 minutes. For these analyses, questions were classified into six categories: knowledge of STIs (six items), STI history and symptoms (15 items), sexual practices (19 items), sexual partners (six items), condom use (five items), and alcohol/drug use (nine items). We expected to detect higher reporting of potentially sensitive sexual behaviours such as same gender sex, anal sex, and forced or paid sex among respondents assigned to ACASI. On the other hand, we predicted that socially desirable behaviours within the clinic population would be more frequently reported by respondents assigned to IAQ. Such “clinically prescribed” behaviours would include, for example, consistent condom use, knowledge of various STIs, seeking treatment for STI symptoms, and communicating with a partner about STI risk or diagnosis. A null effect of interview mode was predicted for items on STI history and symptoms, frequency of sexual contact with partners, and ever use of alcohol or drugs. For specific sexual practice—for example, oral sex, one night stand, we expected that “ever” doing these may not be sensitive within the clinic context, but that frequent or recent use might be more resistant to accurate self report.

Analyses

Our analyses assess the impact of interview mode (ACASI versus IAQ) on reporting of a range of sexual behaviours, condom use, and other behaviours associated with the STI/HIV transmission. Adjusted ORs were calculated using multivariate logistic regression models controlling for the impact of variation in the sociodemographic composition of respondents assigned to the two modes—age, marital status, education, race, patient status, and health insurance coverage. These analyses were conducted using Stata, version 6.0.¹⁴

Log linear models (SPSS, version 6)¹⁵ test whether the impact of ACASI on reporting of risk behaviors was equivalent across subpopulations defined by patient status, gender, and age. Models were fitted to three way tables of interview mode (M, ACASI, or IAQ), subpopulation (S, for example, patient status: new or repeat), and reported behaviour (B, for example, anal sex: yes or no). Log linear models were constrained to fit all two way marginals ((MS) (MB) (SB)) in these three way tables, a procedure statistically equivalent to fitting a logistic regression with reported behaviour as the outcome and testing for an interaction between interview mode and the subpopulation in the prediction of the outcome. Likelihood ratio χ² statistics were calculated for each model. Failure to obtain a statistically adequate fit indicated the presence of a statistically significant interaction—that is, the association or impact of mode on reporting of the behaviour varied across subpopulations.

RESULTS

Of the 2633 patients approached in the clinic, 971 declined to participate. An additional 110 left the clinic before the interview could be completed, and 193 failed to meet eligibility criteria or were previously interviewed. Over 60% (61.7%) of females approached completed the study compared to 41.7% of males (p <0.001); there were no differences between participants and non-participants in terms of age, reason for visit, or patient status. Nine HIV positive patients were mistakenly enrolled in the study and excluded. In all, 1350 eligible patients consented to participate in the study and completed the survey interview for a response rate of 55.3%.

The majority of subjects were African-American (94%) and single (66%). Subjects were stratified by sex and assigned to either ACASI (n = 677) or IAQ (n = 673). There were no statistically significant differences in measured demographic characteristics for either male or female respondents by assignment to interview mode (table 1). Respondents assigned to ACASI completed the interview, on average, in 19.3 minutes compared to 18.8 minutes for those assigned to IAQ (p = 0.04).

Estimates of HIV/STI related risk behaviours

The majority of subjects (96-98%) were familiar with more common STIs—for example, gonorrhoea, chlamydial infection, herpes, syphilis, and 61% reported having heard of pelvic inflammatory disease, or PID. Knowledge of genital phlemoria, a fictitious disease, was reported by a minority of subjects (18.8%), but more frequently in ACASI (adjusted OR = 2.7; data not shown).

Table 2 compares reports of STI history and symptoms by interview mode. Mode of survey administration made little difference in reports of previous STI history or symptoms, although clinical reports generally were higher in ACASI than IAQ. Subjects were more likely to report to the interviewer than to the computer that they had sought medical attention for two common symptoms, discharge (adjusted OR = 0.3) and dysuria (adjusted OR = 0.5), a “clinically prescribed” behaviour.

Nearly one in four females reported same gender sexual experiences, compared to one in 10 males; however, reports did not vary significantly by interview mode (table 3).

Patients assigned to the ACASI mode were significantly more likely to report anal sex (adjusted OR = 1.4) and paid sex (adjusted OR = 1.5) and to report more recent occurrences of several behaviours—for example, sex five or more times in the past 2 weeks (adjusted OR = 1.5); anal sex three or more times within the past 30 days (adjusted OR = 2.0); giving oral sex to a partner in the past month (adjusted OR = 1.4); and, sex during menstruation in the past month (adjusted OR = 2.0). Having at least one new partner in the past 6 months (adjusted OR = 1.6) and sex without use of a condom in the past 24 hours (adjusted OR = 1.9) were also reported more frequently in ACASI. Consistent with the notion that interviewer presence motivates positive responses to socially desirable behaviours, subjects assigned to the IAQ mode were more likely to report they felt very comfortable telling their main partner about their STI (adjusted OR = 0.7). On the other hand, they also were more likely to report no condom use in the past 3 months (adjusted OR = 0.7).

Unlike previous studies of the general population, neither reports of lifetime use of illicit drugs—injecting drugs and use of cocaine—nor more recent use of these drugs varied by interview mode (table 4). We note, however, that our estimates are based on a small sample size. Contrary to our expectations, respondents in ACASI were 2.2 times as likely as those assigned to IAQ to report never drinking alcohol.

Variation in mode effects by patient subpopulation

To explore the possible interaction of interview mode with patient status, age, and gender, we fitted hierarchical log linear models for each of our measured sexual and health behaviours. The impact of survey mode on the reported prevalence of each behaviour was equivalent between new and repeat patients; contrary to our hypotheses, there was only one significant mode patient status interaction (sex during menstruation, p<0.05).

Log linear modelling revealed little variation in the impact of interview mode by age (15–24 and 25–39). However, tests of the homogeneity of the effect of mode by gender yielded multiple significant results, indicating a relatively increased mode effect for female subjects (table 5). The impact of ACASI was stronger among females than among males for reporting of previous STIs (history of chlamydial infection, adjusted OR = 1.3 v 0.7; herpes, adjusted OR = 1.8 v 0.6) as well as the reporting of previous STI symptoms (genital sores, adjusted OR = 1.9 v 1.0; genital warts, adjusted OR = 2.3 v 1.1).

Relative to males, female respondents also were more likely to report several sexual behaviours in ACASI than in IAQ: sex five or more times in the past 2 weeks (adjusted OR = 2.0 v 1.1), sex during menstruation (adjusted OR = 1.5 v 1.0), and one or more new partners in the past 6 months (adjusted OR = 2.1 v 1.2). Log linear models of the homogeneity of the impact of ACASI by sex yielded a significant test statistic for these interactions.

DISCUSSION

The findings of this study provide further evidence of the utility of ACASI in improving the quality of behavioural data. In general—but with some exceptions—the STI clinic attendees who were assigned to ACASI were more likely to report risky sexual practices, and less likely to report protective behaviours than subjects assigned to IAQ. Such information is critical to our understanding of the factors associated with STI infection and, consequently, to the development of preventive interventions to reduce their acquisition and spread. Furthermore, these data extend our understanding of the potential application of ACASI beyond population based surveys to the clinical context where we expected that disclosure of sensitive behaviours might be less difficult and mode differences less pronounced among patients who had previously attended an STI clinic.

Contrary to our expectations, whether the respondent was a first time or repeat clinic patient had little effect on the mode differences observed within this population. One possibility is that the effects of ACASI administration influence a range of subjects similarly. Males and females, however, did not respond to ACASI equivalently. In fact, the only subject attribute we observed that appeared to interact consistently with interview administration mode was subject gender.

The variation in mode effect by gender that we observed is admittedly difficult to interpret. Perhaps females within the clinic environment considered a range of behavioural assessment items more sensitive than did males. It also is possible that the privacy afforded by ACASI relative to the IAQ mode was felt more strongly by females within our study than male participants, a supposition supported by our earlier qualitative work and several sexual behaviour surveys of men and women in low and middle income countries.^16–18 An alternate explanation may be that males and females responded differently to our study interviewers. The study matched interviewer and subject gender, but used a single interviewer of each gender. As a result, effects of subject gender may be confounded with the characteristics of the particular interviewer who administered the questionnaire. Systematic differences between the two interviewers in interviewing techniques or interpersonal approach could have contributed to differences in subject self reports. In future studies, multiple interviewers of each gender should be employed, and the effect of within gender interviewer effects assessed.

The fact that ACASI had little effect on reported use of illicit drugs may not be surprising. Based on our earlier qualitative phase,¹⁹ it appeared that subjects believed that the use of ACASI was beneficial in maximising privacy, and therefore might reduce embarrassment in discussing sensitive topics with interviewers. However, the use of an ACASI system does little to ameliorate concerns about data confidentiality or potential uses of information once collected. As such, the benefits may be greater for potentially embarrassing but not necessarily illicit behaviour (for example, sexual behaviour) than with illicit behaviour such as drug use. Similarly, mode of survey administration made little difference in reports of previous STIs or STI symptoms. Arguably, patients might expect to be asked these questions as a routine component of their STI clinic visit and not find them particularly embarrassing in the context of the clinic.

As a final caveat, we acknowledge several additional potential threats to the generalisability of our results. Our results include an unknown level of response bias, given the overall response rate of 55%. We note, however, that this rate is within the range of those obtained by population based behavioural surveys, such as the Behavioral Risk Factor Surveillance System and a methodological experiment conducted as part of the second National Survey of Sexual Attitudes and Lifestyles.^5,20 More generally, we acknowledge that patients attending a STI clinic represent a unique population, in terms of risk behaviours and in their motivations for responding to potentially sensitive questions. Whether similar results would be observed in other high risk populations remains to be determined.

There were indications that the ACASI system was well received among participants; both study interviewers reported that subjects reacted positively towards use of the computer, and despite low average levels of education and in some cases very limited literacy, they were able to use the system easily without assistance. From a practical standpoint, it appears that a touch screen ACASI system could be incorporated into research or data collection systems used in STI and other medical clinics. Once programmed, the system requires relatively little administrative attention and reliably encodes and stores information. Overall, the promise of ACASI systems may not be limited to population surveys, but may extend to a range of social and physical environments and to various population subgroups for whom information generally regarded as sensitive is gathered by self report.