Postal research: too many problems?
Postal research is a valuable means of collecting health related information. Although the majority of mailed research is in the form of questionnaires, postal services have also been used to report the results of home tests or to obtain clinical specimens.1–7 This approach has been shown to be valuable in certain screening programmes. For example, home testing for glycosuria and subsequent reporting of the results is a simple and effective way of population screening for diabetes mellitus.1 For genitourinary medicine physicians the concept of postal screening may provide an acceptable method of screening low risk populations for certain sexually transmitted infections. A number of studies have already investigated the potential for postal screening for Chlamydia trachomatis infection using mailed specimens including first void urine and vaginal flush samples.3–7
On the surface, postal research would appear to provide a simple, cost efficient means of reaching a widely dispersed population, many of whom would not normally attend a healthcare setting. However, there are a number of factors that need to be considered in the design of the survey and careful interpretation of the information obtained is essential to ensure the validity of results.
One of the major problems with mailed research is that response rates tend to be low. Response rates will vary depending on the type of survey and the persistence of the investigators in terms of both the number and type of contacts with the subjects.1,3,8,9
Higher response rates are seen when repeat mailings are sent to subjects. In their study investigating non-response bias in postal surveys, Tennant and Badley report an 87% response rate after four mailings of the survey.8 The first mailing saw a 57% response, increasing to 73% after the second mailing and 81% after the third.
Macleod et al surveyed a mixed population of 200 subjects, requesting mailed urine samples to screen for C trachomatis infection.3 They report a response rate of 93% from the 68% of the original subject group who were confirmed to be resident at the address registered with their general practitioner. To achieve this response rate they sent out two mailed packages by recorded delivery and subsequently telephoned or visited non-responders.
The request made by the study can influence participation. Etter et al 9 set out to establish whether asking subjects to provide a saliva sample in conjunction with a survey would influence response rate. They found an 11% lower response rate than when participants were asked only to complete a questionnaire related to smoking habits. It was proposed that the lower response may have arisen as a result of participants' concern that tests other than those specified may have been performed on the samples, such as HIV testing or drug screening. Alternatively, participants may not have felt adequately compensated for providing a specimen.
A low response rate will also be seen if address databases are not regularly updated, particularly where highly mobile populations such as students are being surveyed. Macleod et al 3 reported that 32% of subjects aged 18–45 were no longer living at their GP registered address in their postal survey. Others have also reported the inadequacy of family practitioner committees' lists, and noted that screening programmes will fail if population registers are not improved.10
Numerous incentives designed to improve response rates have been evaluated, and it would appear that a combination of these incentives is most effective.
Spry et al11 found that prenotification in combination with a lottery incentive significantly improved response rates and that prenotification by telephone, although more expensive, was more effective than by postcard. The lottery incentive alone did not increase response rates significantly. Reminders in the form of a letter or postcard, or as a repeat mailing of the questionnaire, have been shown to increase response. This effect tends to decline after the second mailing.11
It is generally perceived that shorter questionnaires are more likely to be completed than longer ones, and some investigators reduce the length of questionnaires in an attempt to enhance response at the expense of the amount of information obtained. However, Spry et al11 compared an eight page survey with a two page survey and found that the length of these questionnaires did not appear to influence response rates. Hoffman et al12 found similar results.
A meta-analysis of mail survey response rate by Fox et al reported that the largest increase in response rate was seen with university sponsorship of the study, prenotification by letter, and stamped return postage.13 First class outgoing postage and the colour of the questionnaire were other factors identified.
It is important to note that incentives that are effective in one population may not be transferable to another. A number of postal surveys have incorporated in their design a means of investigating which incentives provide the best response. A “randomised phase” is initially undertaken and analysed. The incentive inducing the greatest response is then adopted for the “most effective intervention” stage of the survey.14
Bias in postal surveys
As with any other study, bias can influence the results of postal surveys. Sampling bias can be introduced to a postal survey if subjects selected are not truly representative of the population targeted. The study may aim to represent the general population—for example, subjects selected from GP patient lists, or a select population such as university students or adolescents attending a genitourinary medicine clinic. Sampling techniques and random selection procedures are used to determine a representative study sample. Within that sample, certain people may be more likely to respond to a particular survey than others. This may result in selection of a subset of the targeted population being represented.
The possibility of incentives introducing bias needs to be considered. Incentives may introduce selection bias by encouraging a certain subgroup of the population to respond to the survey, or they may introduce information bias by influencing the participants' responses.
The objective of many postal surveys is to determine prevalence. Incomplete response to a survey can introduce uncertainty as to the true prevalence of a condition. Simple extrapolation of the results assumes that there is no difference in prevalence between the responders and the non-responders and consequently may lead to an underestimation or overestimation of the prevalence. This may be particularly important in surveys dealing with sexually transmitted infections where the prevalence of a condition may be strongly influenced by a small “core group” at high risk of acquiring an STI. If this group is underrepresented or overrepresented among respondents there will be a significant impact on the results.
Mailing clinical specimens
There are specific issues which need to be addressed when requesting clinical specimens from subjects via the postal services. However, few studies that have involved the postage of pathological specimens have described the packaging used during transport. A discussion of this would aid future researchers aiming to undertake similar studies.
In order for clinical specimens to be carried by postal services in the United Kingdom, the Royal Mail currently require packaging to comply with the International Air Transport Association (IATA) Dangerous Goods regulations and use UN packing instructions 602 for the carriage of samples likely to contain infectious substances and those for microbiological analysis. These regulations apply to all countries. Some countries such as the United States also have their own national regulations. Of note with the new 602 packaging is the specification that the external dimensions of the package must be a minimum of 100 mm in the smallest overall dimension (table 1). This is too large to allow posting though a normal letter box, necessitating that packages are collected from and delivered to a post office, or that a courier service is used. This may prove to deter subjects from participating in studies. Future postal surveys involving specimen collection will also be more expensive than those previously conducted. However, there is currently a debate as to whether these regulations are necessary for all diagnostic specimens. The formerly used UN 650 packaging specifications can be compact enough to allow postage and may still provide sufficient safety for transport of these specimens.
Sampling and diagnostic accuracy
Posting of specimens can affect the reliability of certain diagnostic tests. Dabbs2 found that using cotton rolls rather than chewing gum for collection of saliva specimens resulted in an elevation of testosterone levels on testing. Posting of the specimens resulted in an elevation of female salivary testosterone levels, but not those from male subjects. Adequate investigation needs to be undertaken to ensure that factors such as unpredictable temperature storage, delays in collection, and the transport medium for the specimen will not adversely affect the test results.
Parker et al15 assessed the stability of specimens from 1017 women for chlamydia testing by Gen-Probe PACE 2 sent via the US postal system compared with those transported in a controlled environment (by courier). They reported a 99% agreement in results between the two groups even after 50% of the mailed specimens had been subjected to temperatures above 31°C and 44% had been in transit for more than 3 days. These data suggest that swabs remain stable during postal transit, but these findings may not apply to other specimens such as urine.
For studies of sexually transmitted infections, specimens from the urogenital tract are usually required. Consideration therefore needs to be given to the subjects' ability and willingness to provide such specimens and to how comparable the sensitivity and specificity of self testing compares with physician obtained samples. Ostergaard et al 5 found that the diagnostic efficacy of home obtained and mailed vaginal flush and urine specimens was comparable with physician obtained and mailed endocervical and urethral swabs when testing was by ligase chain reaction.
Obtaining ethical approval for postal surveys can be difficult, especially when the survey covers a large geographical area involving numerous committee—for example, with national questionnaires. The requirements of each individual ethics committee may vary, resulting in numerous submissions and the associated administration and time costs. This is aptly described by Middle et al who completed 1095 protocols and 1116 application forms to undertake a postal survey of a sample of children born in 1988 in England and Wales.16 Although the newly formed multicentre research ethics committee simplifies the procedure, submissions to local ethics committees are still required.
Another ethical issue that arises when asking participants to provide a clinical sample for a specified test or to report a test result is how to contact the respondent in the event of a positive result. If participants are asked to provide contact details and do not do so, or are not contactable from the information given, the investigators can be left in a difficult position. The participant will be expecting to be contacted if the test result is significant, but on the other hand may consider further action by the investigator (for example, contacting their general practitioner) a breach of confidentiality. Such issues need to be given considerable thought before embarking on such studies.
Postal survey can provide valuable information relating to health issues. However, the design of the study needs to minimise the introduction of bias and give adequate consideration to ethical and practical issues.
Copies of the IATA Dangerous Goods Regulations, 41st edition, 2000, and the IATA Infectious Substances Shipping Guidelines detailing the IATA 602 packaging regulations can be purchased from Danvers International, 36 Court House Road, Finchley, London N12 7PJ, UK (tel: +44 (0) 20 8445 3929; fax: +44 (0) 20 8445 2745).