Response rates

One of the major problems with mailed research is that response rates tend to be low. Response rates will vary depending on the type of survey and the persistence of the investigators in terms of both the number and type of contacts with the subjects.^1,3,8,9

Higher response rates are seen when repeat mailings are sent to subjects. In their study investigating non-response bias in postal surveys, Tennant and Badley report an 87% response rate after four mailings of the survey.⁸ The first mailing saw a 57% response, increasing to 73% after the second mailing and 81% after the third.

Macleod et al surveyed a mixed population of 200 subjects, requesting mailed urine samples to screen for C trachomatis infection.³ They report a response rate of 93% from the 68% of the original subject group who were confirmed to be resident at the address registered with their general practitioner. To achieve this response rate they sent out two mailed packages by recorded delivery and subsequently telephoned or visited non-responders.

The request made by the study can influence participation. Etter et al ⁹ set out to establish whether asking subjects to provide a saliva sample in conjunction with a survey would influence response rate. They found an 11% lower response rate than when participants were asked only to complete a questionnaire related to smoking habits. It was proposed that the lower response may have arisen as a result of participants' concern that tests other than those specified may have been performed on the samples, such as HIV testing or drug screening. Alternatively, participants may not have felt adequately compensated for providing a specimen.

A low response rate will also be seen if address databases are not regularly updated, particularly where highly mobile populations such as students are being surveyed. Macleod et al ³ reported that 32% of subjects aged 18–45 were no longer living at their GP registered address in their postal survey. Others have also reported the inadequacy of family practitioner committees' lists, and noted that screening programmes will fail if population registers are not improved.¹⁰

Incentives

Numerous incentives designed to improve response rates have been evaluated, and it would appear that a combination of these incentives is most effective.

Spry et al¹¹ found that prenotification in combination with a lottery incentive significantly improved response rates and that prenotification by telephone, although more expensive, was more effective than by postcard. The lottery incentive alone did not increase response rates significantly. Reminders in the form of a letter or postcard, or as a repeat mailing of the questionnaire, have been shown to increase response. This effect tends to decline after the second mailing.¹¹

It is generally perceived that shorter questionnaires are more likely to be completed than longer ones, and some investigators reduce the length of questionnaires in an attempt to enhance response at the expense of the amount of information obtained. However, Spry et al¹¹ compared an eight page survey with a two page survey and found that the length of these questionnaires did not appear to influence response rates. Hoffman et al¹² found similar results.

A meta-analysis of mail survey response rate by Fox et al reported that the largest increase in response rate was seen with university sponsorship of the study, prenotification by letter, and stamped return postage.¹³ First class outgoing postage and the colour of the questionnaire were other factors identified.

It is important to note that incentives that are effective in one population may not be transferable to another. A number of postal surveys have incorporated in their design a means of investigating which incentives provide the best response. A “randomised phase” is initially undertaken and analysed. The incentive inducing the greatest response is then adopted for the “most effective intervention” stage of the survey.¹⁴

Bias in postal surveys

As with any other study, bias can influence the results of postal surveys. Sampling bias can be introduced to a postal survey if subjects selected are not truly representative of the population targeted. The study may aim to represent the general population—for example, subjects selected from GP patient lists, or a select population such as university students or adolescents attending a genitourinary medicine clinic. Sampling techniques and random selection procedures are used to determine a representative study sample. Within that sample, certain people may be more likely to respond to a particular survey than others. This may result in selection of a subset of the targeted population being represented.

The possibility of incentives introducing bias needs to be considered. Incentives may introduce selection bias by encouraging a certain subgroup of the population to respond to the survey, or they may introduce information bias by influencing the participants' responses.

Prevalence studies

The objective of many postal surveys is to determine prevalence. Incomplete response to a survey can introduce uncertainty as to the true prevalence of a condition. Simple extrapolation of the results assumes that there is no difference in prevalence between the responders and the non-responders and consequently may lead to an underestimation or overestimation of the prevalence. This may be particularly important in surveys dealing with sexually transmitted infections where the prevalence of a condition may be strongly influenced by a small “core group” at high risk of acquiring an STI. If this group is underrepresented or overrepresented among respondents there will be a significant impact on the results.

Mailing clinical specimens

There are specific issues which need to be addressed when requesting clinical specimens from subjects via the postal services. However, few studies that have involved the postage of pathological specimens have described the packaging used during transport. A discussion of this would aid future researchers aiming to undertake similar studies.

In order for clinical specimens to be carried by postal services in the United Kingdom, the Royal Mail currently require packaging to comply with the International Air Transport Association (IATA) Dangerous Goods regulations and use UN packing instructions 602 for the carriage of samples likely to contain infectious substances and those for microbiological analysis. These regulations apply to all countries. Some countries such as the United States also have their own national regulations. Of note with the new 602 packaging is the specification that the external dimensions of the package must be a minimum of 100 mm in the smallest overall dimension (table 1). This is too large to allow posting though a normal letter box, necessitating that packages are collected from and delivered to a post office, or that a courier service is used. This may prove to deter subjects from participating in studies. Future postal surveys involving specimen collection will also be more expensive than those previously conducted. However, there is currently a debate as to whether these regulations are necessary for all diagnostic specimens. The formerly used UN 650 packaging specifications can be compact enough to allow postage and may still provide sufficient safety for transport of these specimens.

Sampling and diagnostic accuracy

Posting of specimens can affect the reliability of certain diagnostic tests. Dabbs² found that using cotton rolls rather than chewing gum for collection of saliva specimens resulted in an elevation of testosterone levels on testing. Posting of the specimens resulted in an elevation of female salivary testosterone levels, but not those from male subjects. Adequate investigation needs to be undertaken to ensure that factors such as unpredictable temperature storage, delays in collection, and the transport medium for the specimen will not adversely affect the test results.

Parker et al¹⁵ assessed the stability of specimens from 1017 women for chlamydia testing by Gen-Probe PACE 2 sent via the US postal system compared with those transported in a controlled environment (by courier). They reported a 99% agreement in results between the two groups even after 50% of the mailed specimens had been subjected to temperatures above 31°C and 44% had been in transit for more than 3 days. These data suggest that swabs remain stable during postal transit, but these findings may not apply to other specimens such as urine.

For studies of sexually transmitted infections, specimens from the urogenital tract are usually required. Consideration therefore needs to be given to the subjects' ability and willingness to provide such specimens and to how comparable the sensitivity and specificity of self testing compares with physician obtained samples. Ostergaard et al ⁵ found that the diagnostic efficacy of home obtained and mailed vaginal flush and urine specimens was comparable with physician obtained and mailed endocervical and urethral swabs when testing was by ligase chain reaction.

Ethical issues

Obtaining ethical approval for postal surveys can be difficult, especially when the survey covers a large geographical area involving numerous committee—for example, with national questionnaires. The requirements of each individual ethics committee may vary, resulting in numerous submissions and the associated administration and time costs. This is aptly described by Middle et al who completed 1095 protocols and 1116 application forms to undertake a postal survey of a sample of children born in 1988 in England and Wales.¹⁶ Although the newly formed multicentre research ethics committee simplifies the procedure, submissions to local ethics committees are still required.

Another ethical issue that arises when asking participants to provide a clinical sample for a specified test or to report a test result is how to contact the respondent in the event of a positive result. If participants are asked to provide contact details and do not do so, or are not contactable from the information given, the investigators can be left in a difficult position. The participant will be expecting to be contacted if the test result is significant, but on the other hand may consider further action by the investigator (for example, contacting their general practitioner) a breach of confidentiality. Such issues need to be given considerable thought before embarking on such studies.

Postal survey can provide valuable information relating to health issues. However, the design of the study needs to minimise the introduction of bias and give adequate consideration to ethical and practical issues.