Background This study assessed the comparability of respondent-driven sampling (RDS) as a sampling and recruitment method by comparing two cross-sectional surveys conducted among MSM in Zanzibar using RDS in 2007 and 2011.

Methods We conducted community-based behavioural surveillance studies in Zanzibar using respondent-driven sampling (RDS) to recruit 509 MSM in 2007 and 344 in 2011. We used crude and RDSAT-adjusted descriptive statistics to assess differences between the samples.

Results Compared to 2007, participants in 2011 were significantly younger (31.4% vs 9.9% were younger than 19, p < 0.001), more likely to have been tested for HIV in the last year (53.7% vs 10.6%, p < 0.001), ever tested (68.2% vs 18.8%, p < 0.001), and less likely to have injected drugs in the last 3 months (1.0% vs 23.2%, p < 0.001). In 2011, 12 (2.6%) tested positive for HIV; in 2007, 65 (12.3%) were positive (p < 0.001).

We found a statistically significant lower HIV infection rate among participants aged 20–24 in 2011 compared to 2007 (5.0% vs 13.4%, p < 0.001) and among those who reported having ever tested for HIV (3.0% vs 10.5%, p = 0.02).

Conclusion The number and magnitude of differences in the characteristics of the two samples suggests that the two rounds of RDS likely sampled different subsets of the Zanzibar MSM population, limiting their comparability and ability to assess trends over time. Similar findings have been reported with repeated RDS surveys in other settings. Our results highlight the continuing challenge in obtaining representative data among key populations affected by HIV to make evidence-based policy and programme planning decisions. While repeated bio-behavioural cross-sectional surveys using the same methodology in the same population are the backbone of surveillance in key populations, we advocate caution in implementation and interpretation of repeated RDS surveys and that other sampling approaches (e.g., Time Location Sampling) be tested.