Editor,—In the recent editorial by Bates and Rogstad¹ the authors describe the problems associated with conducting postal research including response rates, use of incentives, bias, mailing clinical specimens, and ethical issues. We would like to add that there are other important issues to consider when undertaking questionnaire research.

The effectiveness of incentives to increase response rates remains controversial. Kalantar and Talley² recommend using a lottery incentive as it increases response rates after the first mailing. However, differences between groups were not large, and decreased during follow up and disappeared by the fourth mailing. Koloski et al³ found that the use of lottery tickets increase response rates, but may be limited when using them with long questionnaires (32 pages). Moreover, they compared the length of questionnaire (28 v 32 pages) which, while being different, did not reach statistical significance.

The most important aspect of postal research is the questionnaire itself! While high response rates are desirable, it is critical that the information provided by participants is of high quality. The quality of the data may differ between short and long questionnaires and to our knowledge this has never been validated. When participants fill out a long questionnaire they may rush or mark incorrect responses purely because they have lost interest because of the length of the questionnaire. Conversely, if a questionnaire is too short, it may be deemed “unimportant” and not worth completing.

The real question is, is there any real difference in the size of the length of the questionnaires used in this study? In comparison with a four or 10 page questionnaire they are still long. Studies are lacking which highlight the threshold or optimal length of questionnaires.

Figure 1 shows a theoretical model of how response rate may perform according to questionnaire length. Part A represents low response rates due to questionnaires of short length; part B is the optimal questionnaire length giving the best response rate; and part C shows the poor response rate due to questionnaires of excessive length.

The presentation of the questionnaires will also influence the response rates to postal surveys. Questionnaires that are professionally printed and designed are more likely to be taken seriously by participants compared with two pages stapled together.

Other reasons for an increased response rate include the importance of assuring participants of their confidentially and this can improved even further if the steps taken to keep subject data confidential is explained. Respondents may want or expect their answers to be treated strictly in confidence, especially if the topic area is threatening or embarrassing. The researcher should not promise greater confidentiality than he/she can provide remembering that coders and data processors may have access to the information.

Ethics of repeated follow ups is of concern. Some individuals do not like receiving multiple mailouts and this can be a problem if they complain. The respondents' privacy and dignity should be respected. A dilemma may sometimes arise when the need for the researcher to obtain the “informed” consent of respondents conflicts with the need for respondents not to know so much that the results are biased.

One thing is certain; the greater the number of follow ups completed the higher the response rate will be. There can be problems associated with undertaking multiple follow ups, particularly when individuals complain about the number of letters and/or questionnaires they receive. However, this can easily be solved by stating on the initial cover letter if they do not wish to be contacted further, to contact the researchers and tell them so they can be removed from the mailing list. By using some of these techniques researchers should be able to obtain increased response rates and higher quality questionnaire data.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

A theoretical model of how response rate may perform according to questionnaire length