Validity of coital diaries in a feasibility study for the Microbicides Development Programme trial among women at high risk of HIV/AIDS in Mwanza, Tanzania
- Caroline F Allen1,
- Shelley S Lees2,4,
- Nicola A Desmond1,2,
- Geoff Der1,
- Betty Chiduo2,
- Ian Hambleton2,4,
- Louise Knight2,4,
- Andrew Vallely3,4,
- David A Ross4,
- Richard J Hayes4
- 1Medical Research Council Social and Public Health Sciences Unit, University of Glasgow, Glasgow, UK
- 2National Institute for Medical Research Mwanza Centre, Tanzania
- 3African Medical and Research Foundation
- 4London School of Hygiene and Tropical Medicine, London, UK
- Correspondence to: Dr Caroline F Allen Caribbean Health Research Council, 25a Warner Street, St Augustine, Republic of Trinidad and Tobago; and
- Accepted 26 June 2007
- Published Online First 27 July 2007
Objectives: To compare coital diaries and face-to-face interviews (FFIs) in measuring sexual behaviour among women at high risk of HIV. To assess the effect of differing levels of support from researchers on reporting in coital diaries and FFIs.
Methods: Three groups of 50 women were randomly selected from a cohort of food and recreational facility workers participating in a microbicide trial feasibility study and received differing levels of researcher support. Minimum support involved delivering and collecting coital diaries weekly; medium support included a weekly FFI and discussion of concerns; intensive support also included an unscheduled mid-week visit when diaries were checked and concerns addressed. All respondents participated in an exit FFI, including questions on sexual behaviour over the four-week study period and study acceptability.
Results: Sexual behaviours were generally reported more frequently in coital diaries than weekly or exit interviews. Vaginal and anal sex, male and female condom use, vaginal cleaning and lubrication, sex during menstruation and sex with irregular and regular partners were reported more frequently in coital diaries than exit interviews. In coital diaries, level of support was associated with reporting of vaginal sex and cleaning. In exit interviews, support level was associated with reporting of vaginal sex, vaginal cleaning and sex with regular, irregular and commercial partners. Women with minimum support reported least satisfaction with the research process. Women with intensive support were most likely to report that they informed someone about their study participation and that they completed diaries daily.
Conclusion: Compared with FFIs, coital diaries resulted in higher reporting of socially stigmatised activities, and sexual behaviour reporting varied less by level of support. More researcher support enhanced study acceptability.
Coital diaries are data collection tools in which sexual practices of research subjects are recorded on a daily or other periodic basis or after each sex act. They have been used in sexual health studies to examine the efficacy or acceptability of products1,2 and track behaviour.3–5 Suitable sexual behaviour data collection methods are needed for developing countries, where most HIV infections occur. Low literacy and a desire to conceal socially proscribed activities may hamper the measurement of sexually transmitted infection (STI) risk behaviour.6 Valid sexual behaviour data collection tools are necessary to test the efficacy and acceptability of HIV prevention technologies, but are difficult to devise.7,8
Our study formed part of a feasibility study leading to a phase III trial of vaginal microbicides for HIV prevention, currently being conducted by the Microbicides Development Programme in six sites in sub-Saharan Africa.9 Accurate data on the frequency of sexual practices are needed to show the behavioural conditions under which trial results were achieved. Among the aims of the feasibility phase was to develop data collection tools for use in the trial. This paper assesses the validity of coital diaries and the acceptability of coital diary study implementation by:
Comparing the results of coital diaries with face-to-face interviews (FFIs);
Assessing the sensitivity of coital diary results to different levels of support to participants by researchers;
Assessing whether the acceptability of using coital diaries varied by the level of support provided.
Research subjects were selected from among 1573 women enrolled into the feasibility study in Mwanza, Tanzania, who worked in food or recreational facilities (cooked food or local beer vending, bars, restaurants, guesthouses and hotels).9 Women in these occupations in Tanzania are at high risk of HIV/STI.10–12
The feasibility study included structured FFIs at enrolment (n = 1573), ethnographic fieldwork in 68 food or recreational facilities, focus group discussions on sexual behaviour (six with food or recreational facility workers, two with male customers) and nine open-ended interviews on vaginal hygiene practices. For the coital diary study, these informed the variables chosen, the design of pictures and instruments and the interpretation of results.
After pre-testing of five diary designs, a design with pictures and tick boxes was selected. The coital diary covered 7 days and could be used to record up to four sex acts per day. It was 16 cm by 13 cm to fit into a pocket or handbag, and contained seven data entry sheets (representing each day of the week), and information sheets with explanations for each picture. The diary text was translated into Kiswahili and back before production. Fig 1 shows a collection entry page.
Coital diary evaluation study
We assessed the impact on diary responses of three levels of support by researchers. Diary results were compared with data from FFIs covering two different recall periods (previous week or previous four weeks), either of which FFI might be employed at regular intervals during microbicide trials. The acceptability of the study methods was assessed by interview.
In discussions among Microbicides Development Programme coordinators, it was debated whether FFIs on sexual behaviour should aim to record frequencies over the past four weeks or one week. The preference initially was for four weeks, because a longer time period was thought likely to offer a more representative picture of sexual behaviour, and repeatedly interviewing each week would be more costly. In this paper we therefore focus primarily on comparisons between coital diaries and FFIs covering four weeks.
We selected a subsample of 150 women from participants who were due to attend for a three, six, nine or 12-month follow-up visit to mobile study clinics during a six-week interview period (March to April 2004; N = 538). The sample size of 150 participants was chosen according to logistical and human resource constraints, given a number of competing demands on the research team in meeting feasibility study objectives. Recruitment data suggested that there were few differences in women’s characteristics or STI prevalence between the 10 wards of Mwanza City from which women were recruited to the feasibility study. For each level of researcher support, five wards were randomly chosen and a random sample of 50 women was selected from eligible participants in these wards. To reduce travel between interviews, we ensured a minimum of 10 participants in each ward for the three support levels combined.
Participants were asked to complete diaries for 28 days. Levels of researcher support were defined as follows:
Minimum – Participants received a scheduled weekly visit to their home or workplace to deliver and collect coital diaries. A FFI (exit interview) was conducted a few days after the final visit, including questions on frequency of sexual behaviour over the entire four-week study period and on diary study acceptability.
Medium was the same as minimum support, plus during the weekly visit an FFI was administered relating to the previous 7 days and, if participants raised concerns about diary completion, these were discussed.
Intensive – An unscheduled weekly visit was added, at home or the workplace. During this, fieldworkers checked that participants were completing the diary daily and asked about and helped them with any concerns related to the diary.
Questions asked in weekly and exit interviews were closed-ended, with the exception of some questions concerning interviewee experiences of completing diaries. Responses were recorded manually on questionnaires.
All participants signed or thumb-printed an informed consent form. Ethics clearance was obtained from the National Medical Research Coordinating Committee, Tanzania and the London School of Hygiene and Tropical Medicine, United Kingdom.
Coital diary and FFI data were summarised as the frequency of each behaviour (a count) and the number of women reporting one or more occurrence of each behaviour (a percentage). Paired statistical tests were used to compare responses on coital diaries, four-week FFIs and weekly FFIs within women, whereas comparisons of researcher support strategies were between women and so were unpaired. Reported frequencies of behaviours were not normally distributed; they were positively skewed, with many women reporting no occurrences of the behaviours in question. Non-parametric methods were therefore used for statistical analysis.
For comparisons between coital diaries and the four-week FFI, the Wilcoxon signed rank test was used for behaviour frequencies and the McNemar test was used for the percentage of women reporting one or more occurrence. These tests were repeated to compare coital diaries with weekly FFIs.
For comparing data by level of researcher support, the Kruskal–Wallis analysis of variance was used to compare behaviour frequencies. Chi-squared tests were used for the percentage of women reporting one or more occurrence and to analyse the acceptability of the study by support level.
Kappa was used to measure agreement between pairs of instruments in reporting at all, and Spearman’s rho to assess agreement in frequencies reported.
In the first week, interviewers mistakenly used a version of the weekly questionnaire that did not contain questions on vaginal cleaning or sex by partner type. Full four-week data are therefore only available on the weekly interview for other variables.
Characteristics of evaluation study sample
Participants in the coital diary study were selected to be representative of the feasibility study population (n = 1573). Differences could be introduced at two selection stages: the choice of sample during our six-week recruitment window (N = 538) and the selection of participants from this sample (N = 150). Feasibility study enrolment questionnaire data showed that diary study participants were similar to the feasibility study population regarding most sociodemographic characteristics, prevalence of HIV, herpes simplex virus type 2 and syphilis (data not shown). Of the coital diary study sample, 23% were aged under 25 years and 35% were over 35 years, compared with 30% and 25% of the feasibility study population (p = 0.004). Sixty-six per cent were cooked food or local beer vendors, 19% were guesthouse or hotel workers and 15% were bar, restaurant or grocery workers, compared with 61%, 18% and 21% of the feasibility study population (p = 0.02). Twelve per cent of the coital diary study participants had secondary education or above, 20% were illiterate, 21% were HIV positive and 75% were herpes simplex virus 2 positive.
Of 150 women recruited, all accepted the invitation to participate. During the four weeks of data collection, however, six women dropped out. A total of 144 participants completed an exit interview after four weeks, 141 completed all four coital diaries, and 93 completed all four weekly interviews (weekly interviews were only administered to the 100 women who received medium or intensive support; fig. 2).
Comparisons between coital diaries and FFIs
Reported frequencies of behaviours tended to be higher on coital diaries than exit interviews; this was so for vaginal sex, anal sex, male and female condom use, vaginal cleaning and lubrication, sex with an irregular partner and with a regular partner. Substantial differences were found for vaginal sex, male condom use, vaginal cleaning and sex with an irregular or regular partner (p<0.005). Comparing coital diaries and exit interviews, on diaries greater numbers of respondents reported any occurrence of male condom use, female condom use, vaginal cleaning, sex during menstruation and sex with an irregular partner. Very pronounced differences were found for vaginal cleaning, sex during menstruation and sex with an irregular partner (p<0.001; table 1).
The online appendix presents comparisons of coital diaries with weekly interviews. Whenever differences were considerable (p<0.05), the frequencies and percentages of individuals reporting were higher on coital diaries than weekly interviews. The differences were, however, generally smaller between coital diaries and one-week recall FFIs than between coital diaries and four-week recall FFIs. The online appendix can be viewed on the Sexually Transmitted Infections website (http://sti.bmj.com/supplemental).
Section 2 of the online appendix provides an analysis of agreement between data collection instruments.
Level of support
Over the four-week study period, level of support appears to have had little impact on the reported frequency of behaviour in coital diaries, although lowest frequencies were reported in the medium support group for vaginal sex, contraceptive use, vaginal cleaning and sex with a regular partner. For vaginal cleaning, there was a markedly lower average frequency with medium support (p<0.001). In exit interviews, however, somewhat larger differences in frequencies reported by support level were found for vaginal sex, sex in exchange for money or gifts, sex with a regular partner, sex with an irregular partner and vaginal cleaning. For all but the last two of these, frequencies increased as the level of support increased. Strong relationships between reported frequencies and level of support were found for vaginal sex, vaginal cleaning and sex with a regular partner (p<0.001). In exit interviews, in each case when differences were found to be significant at the p<0.05 level, the number of acts reported was highest with intensive support (table 2). For eight of the 10 variables measured on both instruments, differences in frequencies between coital diaries and exit interviews were smallest either with medium or with intensive support. For vaginal sex, the positive difference between coital diary and exit interview frequencies fell as the level of support increased (F-test, p = 0.023).
On coital diaries, there were increases in the number of respondents reporting one or more event as the level of support increased for vaginal sex, contraceptive use and sex with a regular partner. For vaginal cleaning, the number of individuals reporting was substantially lower with minimum support (p<0.001). On exit interviews, there were increases in the number of respondents reporting one or more event as the level of support increased for vaginal sex, sex in exchange for money or gifts, and sex with a regular partner (p<0.05 in each case). For vaginal cleaning and sex with an irregular partner, the number reporting was lowest with minimum support (p<0.001 and p = 0.010, respectively). On exit interviews, the number reporting was generally highest with intensive support, except for the vaginal cleaning variable when reporting was roughly equal between minimum and intensive support (table 2).
Evaluation of coital diary and survey process
Less than 5% of participants with intensive support said they found any diary pictures difficult to understand, unacceptable or embarrassing. In contrast, several pictures were reported embarrassing by more than 5% of participants in either the minimum or medium support group. Women with minimum support reported least satisfaction with the research process and were most likely to report that someone had discovered their diary. Women with intensive support were most likely to say they informed someone about their participation in the study and that they completed their diaries daily (table 3).
This study has shown a general pattern of more frequent reporting of sexual behaviour in coital diaries than in FFIs. The level of support from researchers apparently affected the frequencies of acts reported and the numbers of individuals reporting on a greater number of sexual behaviour variables in four-week FFIs than in coital diaries. Level of support was associated with the acceptability of the diary design and completion processes.
Strengths and weaknesses of the study in assessing the validity of coital diary data
Assessing the validity of quantitative sexual behaviour data collection tools among high-risk populations generally entails comparisons between self-reporting methods.13 Validation through partner reports is largely impractical. Frequency data cannot be validated using biomarkers.8 This study used several types of comparison between self-report methods to assess validity, including measuring reporting patterns, using qualitative methods to explain patterns, assessing stability under different research conditions and examining strength of agreement.
For socially sensitive behaviours, higher reporting is argued to indicate higher validity and lower social desirability bias.14–16 We found a general pattern of higher frequencies and reporting in diaries than exit interviews, including that of sexual behaviours found in qualitative research during our microbicides feasibility study to have low social acceptability (male condom use, sex with an irregular partner, sex during menstruation). Qualitative methods may be used to help explain patterns in quantitative data, with reference for example to prevailing social norms.8,17 Condom counselling was provided in the feasibility study, but qualitative research revealed that the use of male condoms symbolised sexual infidelity and commercial sex and was unacceptable when a partner wished for a long-term relationship, or when women wished to please their partner sexually. This suggested that social desirability bias, if it existed, would tend to deflate condom use reporting.
Diaries encourage frequent data recording, reducing recall bias. The importance of recall is suggested by the pattern of reporting we found, with higher reporting on coital diaries but larger differences with four-week interviews than with weekly interviews.
Several studies in developing countries have explored the potential of coital diaries18–20 or other methods to reduce social desirability bias.6,8,21 These have generally shown lower reporting of socially sensitive behaviours in FFIs, but have not analysed the impact of the level of support on reporting. The stability of findings under differing research conditions (such as researcher support) is a further criterion for validity assessment.22 Although the picture is not clear-cut, coital diary results were generally more consistent than exit interviews in the face of differences in support. Frequencies reported in exit interviews were generally highest with intensive support, and differences between diaries and exit interviews were smallest with medium or intensive support. If we accept both higher reporting of socially proscribed behaviours and stability in results as methods of assessing validity, coital diaries emerge as more valid, because reporting on coital diaries was higher and exit interview results only tended to converge with coital diary results when researchers provided more support.
The problem of reactivity occurs when awareness that behaviour is being measured influences the behaviour being studied.23 As in other coital diary studies, different tools were administered to the same respondents, so the use of one may have influenced reports on others; this is a variant of the reactivity problem. Future studies may be strengthened by administering different methods to different random samples.
“Construct validity” refers to whether variables measure what they are supposed to measure.16 A method to assess this is agreement between items measuring the same construct.18,24 The online appendix shows and discusses agreement between coital diaries and FFIs.
Given the sample size (less than 50 per group), our study may have been only moderately powered to detect some differences of potential importance. Substantial differences were found despite this, however, and these were consistent with expected patterns according to the literature and our qualitative research. Previous studies comparing coital diaries with other methods of recording sexual behaviour have been conducted with samples of less than 150,18,24–30 in contrast with several studies comparing other self-report methods, which have used larger samples,6,8,21 Further attention should be paid to the validation of coital diaries, using larger sample sizes.
Utility of coital diaries in evaluating sexual health technologies
The need for quantitative sexual behaviour data collection methods appropriate for developing countries is likely to increase with the growth in clinical trials of STI prevention technologies. An advantage of daily diaries is that they permit the collection of per-sex-act data, showing the co-occurrence of behaviours.16,25 For example, in interpreting microbicide trial results showing changes in HIV incidence, we may wish to know, for those sex acts in which a microbicide was used, what number were accompanied by behaviours that may influence the impact of the microbicide on HIV transmission. Using coital diaries, one can calculate the proportion of reported sex acts with microbicides in which, for example, either condoms or vaginal cleaning, or both, were reportedly used. Recalling individual events and proportions in FFIs covering longer periods may be challenging.
The generation of per-sex-act data is costly, however, because data on each individual sex act must be entered separately. Computer data entry for diaries may be more time-consuming than for FFIs.
We found that reportedly the coital diary study was more acceptable to participants with higher support. In our study, however, medium support cost 23% more than minimum; intensive support added a further 15%. As there was little impact on reporting in coital diaries, the advantages of higher support appear to be in acceptability. The burden to participants of daily completion over extended periods should also be considered.18
Our results support the validity of coital diaries compared with four-week FFIs, because on coital diaries there was higher reporting of socially proscribed behaviour, somewhat greater stability of results in the face of differing levels of researcher support, and patterns of reporting were consistent with expectations from qualitative research. The burden and costs of coital diary completion must, however, be considered. Regular data collection with random samples of participants in HIV prevention trials may be worthwhile to collect per-sex-act data and estimates with higher validity than may be possible using FFIs.
This research forms part of the Microbicides Development Programme, funded by the UK Department for International Development and the Medical Research Council. The authors would like to thank Daniel Wight of the Medical Research Council Social and Public Health Sciences Unit for useful comments. Permission to publish this study was granted by the National AIDS Control Programme, Tanzania.
Contribution of authors C.F.A. co-designed the study and designed terms of reference for the consultant, advised and supervised its execution, analysed the data and wrote the paper. S.S.L. was the consultant employed to carry out the study, including management of the field team, logistics, data entry and preliminary analyses. She contributed substantially to writing the paper. N.A.D. was social science coordinator responsible for this study at the National Institute for Medical Research in Mwanza. She co-designed the study and jointly supervised its execution, contributed to the analysis and commented on drafts of the paper. G.D. designed and advised on statistical analyses in the paper. B.C. managed and supervised the field team and contributed to analyses. I.H. and L.K. managed data coding and entry and conducted some statistical analyses. A.V. contributed to study supervision and management and gave comments on drafts. D.A.R. contributed substantially to the study design and writing up, especially via comments on drafts. R.J.H. co-designed the study and contributed substantially to analysis and writing, especially in commenting on drafts of the paper.
Published Online First 27 July 2007
Funding: This research forms part of the Microbicides Development Programme, funded by the UK Department for International Development and the UK Medical Research Council.
Conflict of interest: None.
- 5Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, 50 Beale Street, 12th Floor, San Francisco, San Francisco, CA 94105, USA;
- Published Online First 27 July 2007
The challenges of accurate measurement of self-reported sensitive behaviours in STI/HIV prevention research are highlighted by considerable epidemiological research1 and two recently completed Phase III HIV prevention trials whose null findings underscored the urgent need to improve measurement of adherence.2 Imperfect ways to assess the accuracy of self-report exist. Current biomarkers, which validate specific behavioural assessments,3,4 are not substitutes for measuring behavioural frequencies, partnership characteristics, and co-occurrence of behaviours (eg, microbicide use and sexual activity).
Sexual-behaviour diaries attempt to reduce two misclassification biases: recall and social desirability.5 Few studies have assessed diary use in sub-Saharan Africa. In this issue, Allen et al6 report on an innovative pictorial diary designed to collect sexual event-level data. They examine diary validity through multiple reliability assessments, including a 1-month retrospective interview completed at the end of the diary period, and compare the effects of three-levels of interviewer support on diary reports.
Most reliability assessments posit that higher reports of sensitive behaviours (the “more is better” hypothesis) are more accurate. In the context of most HIV-prevention trials, this interpretation is likely appropriate. Analysis of the male-condom-use finding in Allen et al’s study—that higher reports of using condoms is more accurate—seems counterintuitive at first review. Through qualitative research, they ascertained that reports of using condoms within long-term relationships was stigmatised and concluded that reporting no condom use was the socially desirable response. In contrast, reported condom use in prevention trials is typically much higher than that found in the general population (likely due to enhanced condom counselling and availability; trial participation; and over-reports due to social desirability bias). A study designed for analysis by partner type (eg, regular vs exchange) would strengthen Allen et al’s conclusion, as it is unlikely that reports of no condom use with exchange partners would be perceived as socially desirable as well.
The effects of interviewer support on reporting accuracy are difficult to interpret. The percentage of participants reporting any vaginal sex and sex with regular partners increased with more intensive support and was higher in the diary. What accounted for this pattern? It is unlikely that having vaginal sex with regular partners was stigmatised. Could the increased rapport with interviewers have resulted in a willingness to report sexual activity, or, rather, did participants feel obliged to report something during the twice-weekly interviewer contact, thereby resulting in an over-reporting of sexual activity? Also affected by intensive support were sex with irregular and casual partners, where, if “more is better”, then interviewer rapport seems to have increased honesty.
This study demonstrated high completion of a pictorial diary that could be adapted readily for use in other low-literacy populations. I would hesitate to recommend a diary method with enhanced interviewer support, however, until further research considers the trade-off between increased accuracy in reporting due to interviewer rapport and increased social desirability bias resulting from the loss of privacy that accompanies regular contact with an interviewer. This balance is likely to vary across cultural settings and population groups; certainly the growing literature of measurement studies conducted in sub-Saharan Africa suggests that interview mode does influence social desirability bias.7,8,9,10,11
This paper highlights several methodological lessons: offering some level of support to participants increased diary completion, which likely reduced recall bias; understanding what behaviours are sensitive and/or stigmatised within a particular population group is critical to interpretation of behavioural self-reports; and the role of the interviewer in influencing reporting is complex and warrants further study to guide the design of future STI/HIV prevention trials.