Introduction

The Importance of Understanding Sexual Partnerships and Sexual Mixing

The prevalence of sexually transmitted infections (STI) has been shown to vary by gender, age, and ethnic group (Aral, 2000; Fenton et al., 2001; Garnett et al., 1996; The UK Collaborative Group for HIV & STI Surveillance, 2007). STI transmission is determined by numbers and characteristics of sexual partnerships. Risk increases with increasing partner numbers, but is also related to partner’s STI risk and protected or unprotected sex (Anderson, May, Boily, Garnett, & Rowley, 1991; Aral, 2000; Aral et al., 1999; Fenton et al., 2001; Garnett et al., 1996). STI transmission risk varies according to whether partnerships are formed between people from similar (“assortative”) or different (“disassortative”) prevalence and sexual activity groups (Garnett et al., 1996). For example, young women with older male partners are at increased risk, relative to young women who form partnerships with men of a similar age (DiClemente et al., 2002; Ford, Sohn, & Lepkowski, 2001; Miller, Clark, & Moore, 1997).

In addition to the characteristics of sexual partners and partnerships, their timing is important. Concurrent partnerships are those in which an individual has two or more partnerships which overlap in time, as opposed to “serial monogamy” where one partnership ends before another one starts. Concurrency has the effect of placing all those in the partnership network at risk and can be thought of as effectively increasing the network of partners, because a larger number of people are at risk if an STI is introduced. The proportion of partnerships which are concurrent has an impact on transmission at the population level (Morris & Kretzschmar, 1995; Watts & May, 1992).

Understanding the characteristics of sexual partners and partnerships in the general population is therefore important for our understanding of STI transmission dynamics and can inform appropriate targeting of interventions. However, while concurrency is a particularly important parameter to measure for STI epidemiology, measurement is problematic because of the difficulties of remembering accurately and the social undesirability of reporting such partnerships in the context of a societal norm against “unfaithfulness.”

Challenges of Collecting Data on Sexual Partnerships Through Surveys

The collection of sexual behavior survey data is recognized as challenging for a variety of reasons (Catania, Gibson, Chitwood, & Coates, 1990; Meston, Heiman, Trapnell, & Paulhus, 1998; Wight & West, 1999). The need for detailed data on sexual partners’ characteristics and the number and timing of sexual partnerships adds to this challenge, for several reasons.Footnote 1

First, the highest burden is often placed on a subgroup of particular interest. Participants with multiple recent partners may be asked questions repeatedly, for each partner, but typically up to a certain number (e.g., their two or three most recent partners) (Juarez & Martin, 2006; Kraut-Becher & Aral, 2006; Luke, 2005), increasing interview duration and, potentially, recall difficulties. Researchers must balance the epidemiological benefit of collecting detailed information against the risks of over-burdening some participants and jeopardizing the validity of the data. Participant fatigue or irritation may decrease accuracy of responses, introduce non-response or even cause the interview to be terminated. Bias may be introduced where these risks relate to particular participant characteristics (including particular partnership histories).

Second, the questions involve participants thinking about their partnerships with specific individuals (which may have since ended), which may be more emotionally sensitive than thinking about their sexual behavior in general.

Third, while ability to recall information about sexual behavior can depend on the salience, recency, and social significance of the experience, this may vary between participants’ partnerships. Participants may be asked for information which they may have never known (a partner’s exact age, for example).

Fourth, in order to avoid asking directly about “affairs” or non-monogamy, which can be an especially sensitive topic, measures of concurrency can be derived indirectly, and potentially inaccurately, based on approximate partnership dates.

Finally, gathering these data from general population samples means that questions and response options must be appropriate not only to participants with a wide range of sexual behaviors and experiences, but also to a range of sexual partnerships, from brief, anonymous sexual encounters to very long, committed relationships. Much of the recall error and bias in survey data may be attributable to aspects of questionnaire design (as well as interviewing methods) (Friedenreich, 1994); hence, measures taken to improve design will increase the accuracy and comparability of data to be collected about varied partnerships, from varied participants.

The British National Surveys of Sexual Attitudes and Lifestyles (Natsal) are the largest probability surveys of sexual behavior undertaken anywhere in the world to date. Two surveys have been undertaken, a decade apart, and this article describes part of the development work for the third survey, Natsal-3. Natsal-1 and Natsal-2 sampled the age groups 16–59 and 16–44, respectively, while Natsal-3 seeks to sample 16–74 year olds. A key objective of probability surveys such as Natsal (Fenton et al., 2001; Johnson et al., 2001; Johnson, Wadsworth, Wellings, & Field, 1994; Wellings et al., 2001; Wellings, Field, Johnson, Wadsworth, & Bradshaw, 1994) is to provide data on sexual partnerships and sexual mixing in the general population. We sought to address the methodological issues discussed above in the development of Natsal-3, specifically its “most recent partnerships” (MRP) module. This module included new partnership-specific questions as well as questions asked in previous Natsal studies (Erens et al., 2001; Wellings et al., 1994).

We used cognitive interviewing to test the new and revised questions and the overall flow of this module and to learn how people understand concepts such as concurrency. Cognitive interviews explored the ways in which participants understood survey questions and formulate responses, and this study took place prior to a pilot in which the survey procedures and survey would be tested as a whole.

Method

Participants

A total of 32 cognitive interviews were conducted. The cognitive interviews were administered in two consecutive phases in 2008–2009, with different targeted recruitment strategies. Phase A involved interviews with 22 participants aged 16–74 years who had taken part in the National Centre for Social Research (NatCen) Omnibus survey in February–March 2008 and indicated their willingness to assist with further studies. For practical reasons concerning the areas the interviewers working on this study could cover, participants were interviewed in Scotland, North East England, North West England, and Yorkshire.

Participants were recruited into quotas which ensured a gender and age balance reflective of the expected Natsal survey population; no further selection criteria were employed for Phase A. Phase B comprised interviews with ten participants recruited via an email advertisement sent out to NatCen staff. The request to pass the advert onto friends and family aimed to seek people who had had more than one sexual partner in the last 5 years, in order to test the module with people more likely to find its repetitive nature burdensome. Naturally, some Omnibus participants (Phase A) also met this criterion.

The purposive sampling strategy was designed to reflect the diversity of the proposed survey population, covering characteristics anticipated to influence reactions and response to the questions, including: gender (9 men and 13 women in Phase A, 5 and 5 respectively in Phase B); age (in Phase A, 13 aged 16–58 and 9 aged 59–74 years; in Phase B, 10 in the younger age range); and number of recent sexual partners (in Phase A, 2 out of 22 Phase A participants reported two or more partners in the last 5 years; in Phase B, all 10 participants reported this). In terms of participant’s own concurrency, only one participant in Phase B reported this and so answered this question. All ten Phase B participants answered the question about partner’s concurrency. Neither of these questions was asked in Phase A.

Measures and Procedure

As in previous Natsal studies, all sexually experienced participants in Natsal-3 were asked questions about their most recent partner’s characteristics, regardless of how long ago this partnership occurred. If participants reported two or more partners in the 5 years prior to the survey interview, the question loop repeated for their second and third most recent partners. If, earlier in the questionnaire, a male participant reported sexual partners of both genders, but only reported details of female partners among his ≤3 most recent partners in the 5 years prior to interview, the question loop was repeated for his most recent male partner (and vice versa for female participants). Figure 1 shows the MRP question loop and the wording of questions and responses are provided in the Appendix Table 1.

Fig. 1
figure 1

Most recent partner “loop” showing question order. Shaded boxes were new questions, clear boxes existing questions (which in some cases had been modified from Natsal-2). Questions were specific to each sexual partner/partnership, so for “first sex” read “first sex with this partner.” Question order in the final questionnaire was as in Phase B, with one additional question (whether the sexual partnership was oral sex only, i.e., not vaginal or anal sex), and the question on partner’s world region of birth removed

Full size image
Table 1 Question wording
Full size table

The cognitive interviews took place in participants’ homes or researchers’ offices and were audio-recorded. After giving informed consent, participants completed a selection of questions from the Natsal-3 draft survey (including the entire draft MRP module and other questions not reported here). In order to test the questions as they would be administered in the “real” survey, and because routing between questions was complex, cognitive interviews used the same method of administration for the questions as planned for Natsal-3, i.e., computer-assisted self-interview (CASI) for the entire MRP module. To help ensure privacy during completion, interviewers did not look at the CASI responses at any point. Periodically, participants were instructed to stop completing the questions and the interviewer asked about the preceding questions.

The main cognitive interviewing techniques were probing (the interviewer asked specific questions to gain an understanding of how the participant went about answering the questions) and think aloud (the participant was asked to voice their thoughts as they completed the questionnaire) (Collins, 2003). At the end of the MRP module (as with other groups of questions not reported here), cognitive interviewers used retrospective probing, for which some probes were pre-scripted. Interviewers showed the questions on flash-cards in order to remind participants to which question they were referring. As probing tends to be less burdensome on participants than think aloud (Collins, 2003), and for reasons of confidentiality, interviewers did not ask participants to think aloud as they completed the CASI (although if participants spontaneously told the interviewer what they were thinking this was not discouraged).

Examples of the cognitive interview probes included: (1) “What do you think this question is getting at?” (2) “How did you feel about being asked this question?” (3) “How easy or difficult did you find this question to answer?” (4) “Were you able to find a suitable answer option, or do you think there were any options missing?” (5) “Were you able to provide [the requested information] or did you guess?” (6) “How accurate would you say your guess was?” In addition, and specific to the MRP module, interviewers asked about the question order and flow, as one aim was to test whether the question order was logical for participants. Through these methods, cognitive interviews assessed: (1) acceptability; (2) whether questions were understood as intended; (3) understanding of key concepts, including concurrency; (4) ability to recall the information sought (including recall strategies and judgments made in formulating answers) and to provide an answer. Interviews lasted 1–1.5 h each (including sections of the questionnaire not reported here) and participants were given a £20 voucher as a token of appreciation.

Structured notes were made by the interviewer upon completion of each interview. These were analyzed using Framework, which allows a case-and-theme-based structure to be derived (Ritchie, Spencer, & O’Connor, 2003). A matrix was created, listing the questions across the page, and cases (participants and brief demographic characteristics) down the page. Under each question, researchers summarized how the participant understood the question, recall strategies used, judgments made in formulating an answer, any problems, and the answer itself. Therefore, data could be read as case records for each participant, or question by question, across all cases.

Ethical approval for this study was provided by NatCen’s Research Ethics Committee, ref i9699.

Results

Overview and Flow of Questions

Participants found the loop of questions about their most recent partners (Fig. 1, Appendix Table 1) somewhat taxing, but for the most part logical and straightforward. Some participants who completed the question loop more than once commented that they would have liked to know how many times the partner loop would be repeated, because the burden of recall that this might involve was distracting for them.

Using a Nickname to Identify Partners

At the beginning of the question loops, the questionnaire instructed participants to enter a name, initials or a nickname for each partner, which would be deleted at the end of the module. This was a new addition to the Natsal-3 survey, intended to help participants focus on each partner in turn and was tested in Phase B (it was not tested in Phase A, where the majority of participants were answering about just one partner).

Most participants were comfortable with this and reported that it was helpful for recall. However, some participants had not realized that they could invent a name (and did not wish to enter a partner’s real name; in one case, a participant found it an unpleasant reminder that she had not known the name of a sexual partner). Another participant requested additional confirmation that the names would not be used by researchers. It was important to emphasize that names could be made up and would be deleted, as after this reassurance was given, all Phase B participants were comfortable entering names, nicknames or initials.

The word “partner” appeared in the introduction to each question loop (except the first) and caused some confusion. Due to its social significance, this word was not used elsewhere in the module, where partners were referred to as “the person you had sex with” or by nickname/initials.

Sexual Partners’ Characteristics

Participants were asked each partner’s age at the time when they first had sex with each other, partner’s ethnicity, and where the partner was born (country or world region). Although partners’ ages had been asked in previous Natsal surveys (the other two items were new), participants spontaneously expressed difficulty in answering this accurately for some partners.

Questions on partners’ ethnicity and region of birth aimed to provide epidemiologically important data, since prevalence of STIs differs by ethnicity and country of birth (Health Protection Agency Centre for Infections, 2008). Broad response categories were provided and, if participants selected “Black”’ or “Black British,” they were asked a further question, reflecting differences in the prevalence of HIV and other STIs between people of Black Caribbean and Black African origin in the UK (Health Protection Agency Centre for Infections, 2008). Questions about partners’ ethnicity did not offend or upset participants and were readily answered for brief and longer-term partnerships (informed guesses were made, usually based on appearance, if participants were unsure). One participant commented that the ethnic background of her partners did not matter to her and queried the purpose of this question.

In contrast, it was much more difficult for participants who had not known their partners for long to answer the question about partners’ country/world region of birth. In addition, participants queried the purpose of the question: one participant, for example, commented “Why should I know, and why would I want to know where that person was born?” (referring to partners she did not know very well). Some participants who did not know the answer reported that it made them feel “guilty” and as if they were being judged for not knowing this information. In Phase A, participants were asked whether each partner was born in the UK and if not, they were asked to choose the region of birth from a list. In Phase B, the wording was changed, intending to avoid the suggestion that participants ought to know the answer: participants were asked whether they knew the partner’s country of birth and, if so, they were asked to choose from a list of world regions including the UK (see Appendix Table 1). Despite wording changes, the question was still perceived by some as “out of the blue” and “strange” (and the same participant who had queried the purpose of the ethnicity question, asked why country of birth was asked in addition to ethnicity).

How Participants Met Their Partners

The question about how participants first met each partner listed 16 response options, including the new option “chat room, social networking site or online gaming” (Appendix Table 1). The question was tested to explore whether response options were comprehensive. The question was clear and easy to understand, with participants consistently understanding the term “met” as when they first spoke to each other or had some kind of social exchange. Some minor queries were raised, such as what to do if multiple categories applied e.g., “university” and “at a social event.” Despite the length of the list (which was commented upon), participants reported no difficulty in selecting a response.

No participants chose “chat room, social networking site or online gaming,” so participants in Phase B were specifically asked what they thought it meant. Participants seemed to look at the three examples in this response separately, rather than considering it as an overall “online” category (excluding online dating, which was listed separately). Some of the meanings participants attached to “online gaming” were not necessarily as intended, with participants universally thinking of online poker rather than other gaming sites such as “Second Life” or “World of Warcraft.” However, it should be noted that as they had not reported meeting partners this way, participants were being asked to examine this category somewhat artificially.

How Long Participants Had Known Their Partner When They First Had Sex with Them

This question was clear and participants generally understood it as intended, counting from the time when they first met until when they first had sex. Some participants reported using life or employment circumstances to assist recall. In one case, a participant counted from the first date with a partner whom she had known for longer, while another female participant admitted tending to round up from her initial estimation, in order to select a response, “my inner prude coming out.” Perceived accuracy varied between participants and partnerships and the time ranges provided were considered helpful.

Partnership Dates

Ability to recall dates depended on the recency and salience of the experience, among participants of all ages. Not surprisingly, months were more difficult to recall than years. Recall strategies included thinking about the season and other life events. In Phase A, participants were given a “can’t remember” response option, yet were able to recall accurately the month and year of most recent sex, particularly when it was very recent. As Phase B included participants who had multiple recent partners, the wording was adjusted, asking participants to “estimate the month if you can’t say exactly,” and this was found to be helpful.

Reporting Partnership Status at Most Recent Sex

Participants were asked their relationship status with each partner when they most recently had sex with each other. This was a new question for Natsal-3 (Appendix Table 1).

One response option grouped marital, civil, and cohabiting partnerships together (“we were married/in a civil partnership/living together as a couple at the time”) and occasionally the cohabiting part was overlooked. Seeing the term “married,” some participants who were cohabiting with a partner selected “we were in a steady relationship […]” instead.

The term “steady relationship” was received in different ways: some found it appropriate, compared with alternatives, but others found it “weighted” or unfamiliar (“it sounded American”). Meanings participants attached to “steady relationship” covered the following themes: monogamy, length of relationship, social recognition of the relationship, frequency of seeing each other, purpose, and “whether it’s going somewhere,” all broadly in line with the information sought by researchers. Participants also proposed alternatives including “long-term relationship,” “committed relationship,” or just “relationship.” The term “steady relationship,” though not particularly popular, was intuitively and consistently understood. Problems distinguishing between “steady” and “not steady” relationships appeared to be due to partnerships being in transition between the two states, rather than comprehension. Where partnerships were in the early stages, “on the rocks” or ongoing but casual and non-committed, finding a response was more difficult, while the question was easy to answer for stable relationships, including marriages.

Reporting Likelihood of Having Sex Again with Partners

This question aimed to measure whether a partnership was ongoing and used to route later questions and to weight partnership-level data in analyses (Copas, Mercer, Farewell, Nanchahal, & Johnson, 2009; Mercer et al., 2009). It was not asked when no sex was reported in the previous year. In Natsal-2, whether or not a partnership was ongoing was estimated during analysis, based on responses to three other questions. Asking a direct question is likely to be more accurate than making such assumptions and was found to be easy to answer and inoffensive; some participants who were married or in relationships found it humorous. The only minor issue concerned the distinction between a participant’s desire—wanting to have sex with a partner again—and the perceived likelihood of this happening, with one participant commenting that he would have liked to answer “hopefully!” Between Phase A and Phase B, a “don’t know” category was added and this was found to be helpful.

Reporting Own Concurrency and Perceptions of Partners’ Concurrency

These questions, developed while Phase A was in the field, were tested in Phase B. Own concurrency (i.e., a participant having overlapping sexual partnerships) was assessed from partnership dates (month and year), with an extra question to check whether there was any overlap where this could not be ascertained from the dates provided. Just one participant met these criteria and was therefore routed to this question. This person understood the question well and reported no problems.

Questions about each partner’s concurrency (i.e., whether the partner had any other sexual partners during their sexual partnership with the participant) focused on the last 5 years, to match with the period upon which the module focused, to aid recall and to standardize the question across partnerships of different lengths. There were four versions of this question for each combination of: partnerships beginning within the last 5 years or before this time; and partnerships which have ended or were ongoing (see Appendix Table 1 for question wording). It was difficult to explore each version in turn as participants were asked different versions for different partners and it was difficult to distinguish between the different question versions during probing. However, no participants reported finding the question confusing, so it is likely that its most complex version did not cause major comprehension issues.

As anticipated, participants generally found partners’ concurrency an uncomfortable topic. Reassuringly, however, even participants who reported that their partners had had concurrent partnerships were not offended by these questions.

Regarding accuracy, while some participants considered their perception of a partner’s concurrency as “factual” and their response very accurate, participants who did not know for certain still considered their answers reasonably accurate. We acknowledge that the accuracy of these data will be questionable, as it is impossible to know for sure that a partner did not have sex with others. The research team considered that there was no real difference between “probably not” and “no,” yet participants preferred a graded scale (i.e., the addition of “probably not”).

Participants were asked what they thought these questions were trying to ascertain, and answers (including: “going behind your [partner’s] back,” “cheating on someone,” “having an affair”) were generally consistent with the epidemiological concept of concurrency (UNAIDS Reference Group on Estimates Modelling, and Projections: Working Group on Measuring Concurrent Sexual Partnerships, 2010). However, no participants interpreted the question as including sex with another partner while “on a break” from a relationship, although this scenario is important for STI transmission and thus important to measure.

Definitions of Sex

In Natsal-3, at the start of the CASI, participants were given a definition of sex, inclusive of vaginal, oral, and anal intercourse. To explore whether participants were using this definition throughout the MRP module, after completing the module, participants were asked to write down their own definitions and in Phase B they were asked to indicate whether they were thinking of vaginal, oral, and anal sex when they answered the MRP questions. Although participants understood the definition provided, this was sometimes at odds with their own definitions, which did not always include oral sex, stating for example: “I don’t see [oral sex] as ‘sex’” and “for me the definition of sex is vaginal intercourse.” In the cognitive interviews, some participants revealed that they reverted to their own definitions when answering questions about their most recent partners, leaving out partners with whom they only had oral sex.

Discussion

Statement of Principal Findings

Our cognitive interview findings suggest that survey participants sampled from the British general population are likely to be able to answer many questions about their recent sexual partners, providing detailed data to inform analyses of sexual partnerships and sexual mixing in this population. Even highly sensitive questions about sexual partners, such as those on concurrency, were generally acceptable and did not cause offense among people of different age or gender, or among those who had had multiple recent partners. Our findings also demonstrated the potential of cognitive interviewing to explore aspects beyond question wording, such as the use of nicknames to assist participants in answering about multiple partners. While most participants had just one or a few partners during the recall period, the diversity of sexual behavior required that a balance be struck between asking about a limited number of partners and asking about all partners in a specified time period. Our findings suggest that, in this context, it is likely to be feasible and acceptable to ask detailed questions about up to three or four recent partners.

Strengths and Weaknesses

Cognitive testing allowed refinement of existing questions and testing of new questions and the MRP module as a whole. This process helped to address potential problems, before testing the survey and survey procedures as a whole in the larger-scale pilot. Our research contributes to a small but growing literature on the use of cognitive interviews in sex survey research (e.g., Deardorff, Tschann, & Flores, 2008; Edwards, Thomsen, & Toroitich-Ruto, 2005; Macdonald et al., 2008; Mavhu, Langhaug, Manyonga, Power, & Cowan, 2008; McCabe, Tanner, & Heiman, 2010), and as far as we are aware it was the first to focus on questions about sexual partners and partnerships.

Certain questions were reached through routing from other questions, applied to certain partners only, and therefore were tested on a subset of participants. We would expect major problems with a question to be revealed even with a few participants. However, problems specific to certain unusual participant or partner/partnership characteristics are less likely to surface. That said, several questions had been successfully used before, and the sampling was designed to reach people with a range of characteristics, particularly older age groups which were not surveyed in previous Natsal studies and those with multiple recent partners.

Although we concluded that participants did not find the questions offensive, the full version of the module (including questions about own and partners’ concurrency) was only asked of participants in Phase B, who were younger (aged under 40), and had responded to an advertisement seeking people who had had more than one sexual partner in the last 5 years. This purposive sampling strategy sought to test the questions among varied participants and those who would be completing the question loop repeatedly (rather than to be representative of Natsal participants). We may have missed problems the questions posed for other groups of people. Cognitive interview participants may also have been particularly willing to assist with research or particularly comfortable with revealing this type of sensitive information, compared to the eventual Natsal-3 survey participants. As with any pre-testing, it should not be assumed that participants in the main survey will react in the same way. Indeed, cognitive interviews to pre-test survey questions are carried out not to replace but to complement a pilot, in which some assessment of the response among a larger and possibly more “representative” sample of participants can take place. However, it is reassuring that the subsequent pilot study has supported the conclusions of the cognitive interviews in terms of the feasibility of asking these questions and their acceptability to participants (Phelps, Ogunbadejo, & Nicholson, 2009).

Targeting participants who had had multiple recent partners (Phase B) proved a successful way of testing the questions for a variety of partnership types (for instance, different durations and from casual to cohabiting partnerships), each participant describing at least two partnerships. However, while we succeeded in sampling participants with a range of characteristics, sampling participants whose partner(s)/partnership(s) had particular characteristics would have been challenging, because we could not have used the questions which were being tested in the recruitment process. This led to some limitations: most partners were reported to be white, for example, and so the question on partners’ ethnicity was not tested extensively among people who had partners of different ethnicities.

Meaning of the Study

Participants were generally happy to answer questions about sexual partners’ demographics and none were perceived as too intrusive. However, when participants did not know certain details about a particular partner, the feeling of being “caught out,” as if judged as not really knowing this person, was uncomfortable for some participants who had not known their partners for long. This was a particular problem for the question on partner’s country/region of birth and, compounding this problem, the purpose of the question was not clear to participants. For these reasons, and because it seemed likely to be answered inaccurately or not at all for newer and casual partnerships, this question was removed from the Natsal-3 questionnaire.

Ordinarily, sex survey researchers seek information about behavior and participant characteristics, with the implicit assumption that participants normally know, or at some previous point knew, the information necessary to respond. Questions about participants’ partners may seek information never known to the participant. Researchers need to be aware of this; although information requested in a question may not seem sensitive, lack of knowledge about a partner can be sensitive in itself. In the case of partner concurrency, there were particular concerns about validity. However, we were reassured that participants considered their responses to be accurate and retained this question due to its epidemiological importance.

Further changes were made to the Natsal-3 questionnaire in order to reduce frustration and item non-response, where cognitive interview participants indicated difficulties in giving an accurate response. For instance, wording was changed to instruct participants to “estimate if you can’t say exactly” the timing of events within the 5 years prior to interview and their partner’s age when they first had sex with them. Reducing the amount of missing data for the latter variable was important, as in Natsal-2 14 % of non-regular partners’ ages were missing. Further minor changes were made to question wording and order.

Where few problems with a question were reported, no changes were made if it was judged that while a change could improve a question for some participants, it could cause problems for others. For example, the term “steady relationship,” though unpopular, was understood consistently and alternatives were considered equally or more ambiguous.

In some cases, potential improvements might result in loss of comparability with previous Natsal surveys, so although informed by the cognitive interview findings, the final decisions were referred back to the study team. For example, repeatedly emphasizing the Natsal definition of sex (by providing a definitions card or presenting the definition at the start of the module) in Natsal-3 might introduce bias, if participants to Natsal-1 and Natsal-2 were discounting partners with whom they had only had oral sex.

Conclusions

Through testing the whole MRP module, our findings revealed problems with questions which had been used in previous Natsal surveys that had not been identified. This demonstrates the potential of cognitive interviewing to reveal problems not identified by expert panels or through previous use of questions in the field suggesting that cognitive testing of questions which have already been field-tested remains valuable in improving question wording and in ascertaining whether topics are acceptable, understood and can be answered with reasonable accuracy. It also supports the approach Natsal-3 development work took in using cognitive pre-testing to complement a conventional pilot.