Article Text
PDF
Abstract
Objectives We undertook a prospective evaluation of the Qualpro Syphicheck-WB rapid syphilis test to measure its diagnostic performance and utility as a point-of-care (POC) screening test among female sex workers (FSWs) in Bangalore, India.
Methods From August 2008 to May 2009, FSWs without a laboratory-confirmed history of syphilis attending STI clinics in Bangalore underwent POC syphilis screening using finger-prick whole blood, with onsite treatment if indicated. Serum samples were collected for local laboratory offsite rapid plasma reagin (RPR) testing and reference laboratory RPR, Treponema pallidum haemagglutination assay (TPHA), and rapid syphilis testing. FSWs who participated in standard offsite RPR screening from August 2007 to May 2008 in the same clinics formed the comparison group for treatment coverage.
Results Of the 1617 women who underwent POC syphilis testing, 7.4% had laboratory evidence of active syphilis with reactive RPR and TPHA, and 3.7% had an RPR titre ≥1:8. Compared with the reference RPR and TPHA, the sensitivity and specificity of the POC syphilis test were 70.8% (95% CI 62.7 to 79.0) and 97.8% (95% CI 97.1 to 98.5). Because of the low rate of women returning for their test results after offsite RPR screening, the proportion of women with active syphilis who were appropriately treated rose from 44.8% to 68.3% with the use of POC syphilis screening (p=0.003).
Conclusion The Syphicheck-WB test utilising finger-prick whole blood has a relatively low sensitivity in detecting active syphilis. However, among hard-to-reach populations who may not return for follow-up treatment, POC screening with this assay could still confer an advantage over offsite RPR testing with respect to treatment coverage.
- Syphilis
- diagnosis
- point of care systems
Statistics from Altmetric.com
Introduction
The laboratory diagnosis of syphilis in low-resource settings has traditionally been based on a sequence of screening the patient's serum with a non-treponemal test: rapid plasma reagin (RPR) or Venereal Diseases Research Laboratory (VDRL), followed by a treponemal-specific confirmatory test. With a need to increase access to treatment, there has been a call to integrate rapid, technologically appropriate syphilis testing at venues with limited diagnostic resources.1 2 Operational requirements to perform the screening tests (RPR or VDRL) are barriers to standard testing in such settings, as refrigeration is required for the storage of reagents, centrifugation for serum separation and an electric rotator for flocculation. Further needs include trained laboratory personnel experienced in conducting and interpreting the tests. Although venous blood samples can be collected, frozen and transferred to an offsite testing laboratory, the time it takes to report results can delay or even obstruct treatment, as patients may not return to the clinic for follow-up (especially in the absence of symptoms).3–5
Recent evaluations of simple, point-of-care (POC) syphilis tests have been conducted and provide potential solutions to the above limitations.6 7 These rapid tests have favourable storage properties and do not require electricity. However, at present, all available assays detect treponemal-specific antibodies, which may remain detectable for life; thus, they are unable to differentiate between active and latent (or treated) infection. Although the potential for over-treatment of patients with non-infectious syphilis exists with the use of these rapid syphilis tests as the single POC measure, they may have advantages in resource-poor settings and may be particularly important among mobile and hard-to-reach communities such as female sex workers (FSWs),8 where rates of return for follow-up treatment may be low.
Current recommendations detail the operational aspects of POC syphilis tests in resource-constrained settings and call for assessments of these tests on screening-programme efficacy.9 As part of a syphilis-screening programme for FSWs attending clinics managed by non-governmental organisations (NGOs) in Bangalore, south India, under the India AIDS initiative (Avahan) of the Bill & Melinda Gates Foundation, we undertook an evaluation of the utility of POC syphilis testing. We compared POC test performance against RPR and Treponema pallidum haemagglutination assay (TPHA) test results on serum from a reference laboratory. We also examined test acceptance and treatment coverage during the POC protocol and during the standard protocol, where syphilis testing by RPR was conducted in an offsite laboratory using venous blood. We were then able to compare test acceptance and treatment coverage of the standard and POC screening approaches.
Methods
Study setting and procedures
Four NGO-affiliated STI clinics (fixed and mobile site camps) accessed by FSWs in Bangalore participated in the study between August 2008 and May 2009. Demographic and behavioural data were collected at the time of registration with the NGO, and clinical data were collected at initial and follow-up visits. Each site also functioned as a centre for educational, support and collectivisation activities for FSWs, irrespective of whether their primary motive for attendance was medical. In addition to symptom-based treatment for other STIs, the staff offered POC testing (a rapid TPHA; Qualpro Syphicheck-WB, Goa, India) to eligible FSWs. FSWs were excluded if they had a previous laboratory-confirmed diagnosis of syphilis as documented within the intervention programme. Clinic attendees provided informed written or witnessed verbal consent prior to participation. The clinic nurse collected whole blood via finger-prick (after discarding the first drop) and performed the test according to the manufacturer's instructions for whole-blood samples. Two readers (nurse, social worker or physician) blinded to the other's interpretation recorded the result after 15–20 min. At the same time, 5 ml of venous blood was collected by a nurse or laboratory technologist and transferred to the local offsite laboratory for RPR testing, as well as to a reference laboratory for RPR, TPHA and repeat POC testing on serum. The reference laboratory (St John's Research Institute, Bangalore) conducted the Syphicheck-WB test as per the manufacturer's instructions for serum, and results were recorded by two independent readers (laboratory technicians) blinded to the clinic results. RPR testing was by SPAN Diagnostics, Surat, India, and TPHA testing by Immutrep TPHA, Omega Diagnostics, Scotland.
To assess differences in test acceptance and treatment coverage between the POC and standard (offsite RPR) protocol, we used a historical comparison group comprising women without a prior diagnosis of laboratory-confirmed syphilis who attended the same clinics under the standard testing protocol between August 2007 and May 2008. A comparison of test acceptance was restricted to first-time clinic attendees. The standard protocol included offsite RPR and reference RPR testing on all submissions and restriction of TPHA testing to reference RPR positive samples. The study was approved by the St John's Medical College Institutional Ethics Review Board, Bangalore.
Data analysis
The sensitivity and specificity of the field POC test, the offsite RPR test and the serum rapid syphilis test (reference laboratory) were compared with each other (McNemar test) against the following gold standards: (1) reference laboratory TPHA; (2) combination of the reference laboratory TPHA and RPR (active syphilis); and (3) a combination of the reference TPHA and high-titre RPR (titre ≥1:8). Analysis was conducted using Stata 9.2 (Statacorp). Statistical significance is reported at the 5% level. A detailed description of the study protocol and data analysis is provided in the web appendix.
Results
Study population and test acceptance
At their first clinic or site camp visit during the implementation of the standard protocol, 1017 of 5391 FSWs (18.9%) underwent syphilis screening (table 1). Three additional first-time attendees were excluded from the historical comparison group on the basis of prior laboratory-confirmed syphilis. Women who solicited for clients in brothels and were experiencing symptoms of an STI were more likely to accept testing, whereas those reporting a larger number of sexwork-related working days per month were less likely to undergo screening.
Socio-demographic characteristics of first-time female sex worker clinic attendees offered syphilis screening under the standard (offsite RPR) and point-of-care (POC) protocols
During implementation of the POC protocol, 78 women were excluded from POC testing on the basis of a previous laboratory-confirmed diagnosis of syphilis; all accepted venous sampling for offsite RPR. Of the remaining 4871 eligible women, 714 had been attended the clinic. Of the 4157 first-time attendees offered testing, 1117 (26.9%) accepted. Reasons for refusal included unwillingness to undergo a venous blood draw (66.4%) or finger-prick (3.0%) due to perceived pain, discomfort, or undisclosed reason, concern about (clandestine) HIV testing (4.7%), and prior syphilis testing at outside clinics (6.2%); the remainder provided no specific reason. Characteristics associated with acceptance at first visit were similar across the two screening programmes (table 1). Of the 34 (1.1%) eligible first-time attendees who declined to participate in the study but accepted offsite RPR testing, two were diagnosed as having active syphilis (based on the standard protocol). They were included in the comparison of test acceptance (in the declined POC testing group).
Acceptance of syphilis screening was greater during the POC protocol period than during the standard protocol period (8.0% absolute increase, p=0.002). There were no significant baseline differences between the two groups of women who were screened, with the exception of a shorter duration of time between initial registration with the NGO and first clinic visit for women in the POC protocol group (p<0.001). The magnitude and direction of the difference in test acceptance persisted after adjustment for covariates, including time from programme inception (in weeks), and factors predictive of declining to be tested (presence of STI symptoms, place of solicitation, and reported days per month engaged in sex work), suggesting that the increase in test acceptance by these groups alone is insufficient to explain the small but significant overall difference across the groups. Among clinic attendees who had been previously screened for syphilis since programme inception (April 2007), 96/147 (65.3%) and 510/714 (71.4%) during implementation of the standard and POC protocols, respectively, agreed to a repeat screening, significantly higher than first-time attendees (p<0.001).
Diagnostic performance of POC testing
During the POC protocol, 4871 consecutive and eligible clinic attendees were offered POC syphilis screening, and 1627 agreed to participate. The 47 women who declined POC testing but accepted the offsite RPR were excluded from the analysis of diagnostic performance because under the standard RPR protocol, only reactive RPR samples underwent TPHA testing. One patient declined to provide a finger-prick blood drop after a failed attempt, and nine venous blood samples were of insufficient quantity for reference testing. None of these 10 patients' samples were reactive by the other tests. Of the 1617 patients included in the analysis of diagnostic performance, 14.9% were positive by reference TPHA and 7.4% by reference TPHA and RPR, and 3.7% had TPHA-confirmed syphilis with an RPR titre ≥1:8. At the clinic level, two samples were interpreted as inconclusive by one reader (and negative by the alternate reader). The inter-reader agreement with rapid syphilis test interpretation was 99.6% (κ coefficient 0.975) at POC and 99.8% (κ coefficient 0.981) at the reference laboratory. The sensitivities and specificities of the field (finger-prick whole blood) POC test, local laboratory RPR test and reference laboratory (serum) rapid syphilis test against the three gold standards are shown in table 2. The specificities are high and similar (p>0.1), whereas the sensitivities are significantly different between the field POC and offsite RPR for all three gold standards (p <0.05). The rapid syphilis test had a higher sensitivity when performed on serum (in the reference laboratory) than with finger-prick whole blood, against each reference (p<0.05).
Sensitivity and specificity of field point-of-care (POC), reference rapid syphilis and offsite rapid plasma reagin (RPR) tests against reference gold standards (N=1617)
Genital ulcer disease was noted in four women, all of whom tested positive by clinic POC and local laboratory RPR. There was inadequate power for subgroup analysis by stage for primary and secondary syphilis (N=9). Whole-blood POC sensitivity and specificity (by each of the gold standards) were not significantly different by test performer (nurse or social worker), test reader (nurse, social worker, physician), diagnostic venue, study month, duration of time between opening and use of buffer solution, kit lot number, maximum documented temperature (range, 23–30°C) to which the kit was exposed to prior to test use, or socio-demographic characteristics. Field POC sensitivity (but not specificity) was significantly different between the four clinic teams (with and without adjustment for diagnostic venue), and ranged from 62.5% to 92.3%.
Treatment coverage between POC and standard protocols
Treatment coverage was compared for the 1617 eligible clinic attendees who accepted POC testing (and had received POC as well as reference laboratory tests) and the 1113 eligible clinic attendees who accepted RPR screening during the period of the standard protocol. In the historical comparison group, 44.8% (95% CI 35.8 to 54.3) of women with a diagnosis of active syphilis (reference RPR and TPHA positive) received ≥1 dose of treatment, with a median time from testing to first treatment of 11 days (range 0–317). Solicitation in public places and testing at a site camp were independently associated with higher loss to follow-up (table 3). There were no baseline differences among FSWs who were diagnosed as having active syphilis during the two study periods. Under the POC protocol, 115 FSWs (97.5%) who tested positive by the clinic POC test (N=118) received same-day treatment, corresponding to ≥1 dose treatment in 68.3% (95% CI 60.0 to 76.0) of women with active syphilis, as determined by the reference laboratory. There were no significant differences in socio-behavioural characteristics between the women who did and did not receive appropriate (same-day) treatment within the POC protocol. Increased treatment coverage for the POC syphilis protocol compared with the standard protocol remained significant after adjustment for study month, and decreased in magnitude (absolute difference 16.4%, p=0.02) after adjustment for place of solicitation and diagnostic venue.
Socio-demographic characteristics of female sex worker clinic attendees who received treatment for active syphilis during the standard (offsite rapid plasma reagin (RPR)) and point-of-care screening protocols
POC-guided therapy led to treatment in 33 women who were RPR-negative (29 TPHA-positive and 4 TPHA-negative). This corresponds to potential inappropriate treatment for non-infectious syphilis or false treatment in 28.0% (95% CI 20.6 to 36.7) by the field POC test, compared with a 7.5% (95% CI 6.0 to 9.1) false treatment rate with a reactive offsite RPR result under the standard protocol. Approximately half of all women (54.9%; 95% CI 52.2 to 57.5) tested under the POC protocol (excluding those screened in the last month of the study) returned for follow-up within 30 days, comparable with the historical control group (51.9%, 95% CI 49.0 to 55.9; p=0.10).
Discussion
Although syphilis is a curable STI, access to treatment can be an obstacle to its control, particularly among mobile and hard to reach individuals, who may not return for follow-up after standard testing protocols.8 In programmes involving such communities, effective scale-up of syphilis screening and treatment services requires that this barrier be addressed. Our study found that the POC Qualpro Syphicheck-WB, when conducted on finger-prick whole blood, had a sensitivity of only 70.8% in detecting active syphilis, against a reference laboratory gold standard. This compared with a sensitivity of 100% using a local laboratory RPR test. However, with the population under study, the large number of syphilis-positive women who did not return for follow-up actually resulted in higher treatment coverage with onsite testing. Under the POC protocol, 68.3% of women with active syphilis received at least one dose of intramuscular penicillin compared with only 44.8% of women screened under the standard protocol. We defined potentially inappropriate treatment on the basis of TPHA-positive and RPR-negative results (after a positive POC test), and false treatment on the basis of both RPR and TPHA tests being negative. Together, potentially inappropriate treatment and false treatment of women with positive screening results were greater with POC testing (24.6% and 3.4% respectively) than with the offsite RPR false-treatment rate (7.5%), but this is a lesser concern than undertreatment. In the absence of clinical features, prior treatment history and follow-up serology to negate the possibility that the results correspond to early or late stage (untreated) syphilis, our measure is likely an overestimate of potentially inappropriate treatment rate, as some women with a positive TPHA may have had active syphilis infection in spite of a negative RPR.
A concerning finding was the low overall acceptance of syphilis screening, although there was an increase during the POC protocol period. Test acceptance among sex workers may reflect the level of contact with outreach workers, preclinic awareness of STIs and rapport with clinic staff, or the consequence of recent introduction of invasive testing on a background of symptom-based management for all other STIs. This observation requires further exploration to understand barriers to test acceptance.
As test acceptance was notably higher among women who had previously undergone screening, peer-based education to build clinic rapport and raise awareness of screening may be a useful tool to increase participation. More than half of women who declined testing during the POC protocol period did so because of perceived discomfort from venous blood sampling; few reported that finger-prick bloodletting was a concern, suggesting that acceptance may increase if POC screening is implemented without taking venous blood to verify results.
A limitation of this study is that comparison of treatment coverage and test acceptance was conducted using historical controls, and time-related differences in unmeasured (but confounding) characteristics of the women may have led to overestimating the perceived increase in acceptance of the POC compared with the standard protocol. Treatment coverage comparisons may similarly be affected, although the follow-up proportion at 1 month between the two study periods suggests that attrition-related behaviour was unlikely to be markedly different. Onsite testing facilitated treatment among almost all women and enabled a compensatory increase in treatment among women who were particularly vulnerable for not returning for follow-up.
Our findings are consistent with reports of various rapid syphilis assays showing superior sensitivities when performed on serum, with diminished performance on finger-prick whole blood.6 7 10–12 Possible explanations for this include dilution of antibodies by addition of buffer or coagulation of blood prior to test strip attachment.10 The latter problem has been addressed with the development of anticoagulant-coated capillary tubes which can be used during finger-prick blood collection, resulting in enhanced test sensitivity,10 although this may be assay-dependent.13 This technology, moreover, is more complicated to use and more costly. The sensitivity of the Syphicheck-WB on finger-prick whole blood against the reference serum TPHA was lower than that reported in other studies, and may reflect differences in sample size, as well as notable variability in sensitivity observed between different clinics, a finding that is consistent across other published evaluations.7 Unmeasured variability in humidity may have played a role, although diagnostic performance was unaltered by study month and other, measured, kit and storage properties.
This study focused on a high-prevalence population that is at ongoing risk of reinfection. Use of a treponemal-specific POC screening test, especially a more sensitive one, in this population may lead to overtreatment in women with previously treated syphilis. Although there may be benefit in the face of frequent reinfection, repeated intramuscular injections may deter women from serial testing with a treponemal-specific assay. Another strategy, if resources allow, would be to partner POC with RPR screening (for individuals with a known history of syphilis), or in the absence of RPR availability, a detailed clinical history to discriminate reinfection from an old infection; a difficult task11 among high-risk individuals.
Based on our findings, implementation of a POC syphilis protocol with the relatively low sensitivity reported here, may be advantageous in settings where loss to follow-up is likely to be high (such as site camps or among certain FSW groups), and treatment coverage is thus less than the false-negative rate of the test. Additional advantages include increased test acceptance and ease of use. Cost–benefit analyses in this population also warrant further study. Introduction of any assay however, should be contingent upon evaluating its performance on finger-prick whole blood (if not yet conducted and reported), as it may differ markedly from its performance on serum. Rapid syphilis tests, by providing immediate results and enabling onsite treatment, show promise in the scale-up of syphilis screening programmes in resource-limited settings. Despite recent findings of relatively low sensitivity of various kits in the field, technical improvements (eg, with the use of anticoagulant lined capillary tubes) are possible, and additional research is needed to develop kits with better sensitivity on whole-blood specimens. The use of POC syphilis screening warrants careful community and site-specific consideration.
Key messages
Syphicheck-WB, a rapid treponemal test, has relatively low sensitivity for active syphilis when performed on finger-prick whole blood.
In the presence of low follow-up rates using traditional offsite RPR screening tests, even this less sensitive test may provide improved treatment coverage.
Return-rates with traditional offsite RPR screening should be evaluated when considering the implementation of currently available rapid syphilis tests.
Acknowledgments
The authors wish to thank Swasti and Swati Mahila Sangha, the non-governmental and community-based organisations managing the STI clinics and implementing HIV prevention programmes for and with female sex workers in Bangalore, as well as the clinic staff. We also thank M Sasikiran, B Shetty, H Gururaj, and J Kaur, M Anitha, BN Vijaylakshmi and S Kumar from the Karnataka Health Promotion Trust (KHPT) for assistance with study implementation. We also thank J Stephen for advice on study protocol, P Mrinali for coordinating laboratory testing (St John's Medical College, Bangalore), B Mahapatra (KHPT) for assistance with data management, and R Peeling (London School of Hygiene and Tropical Medicine) and J Hajek (University of Ottawa) for their thoughtful comments on the manuscript.
References
Supplementary materials
Web Only Data sti.2009.038778
Footnotes
Funding This research was funded by the Bill & Melinda Gates Foundation. The views expressed herein are those of the authors and do not necessarily reflect the official policy or position of the Bill & Melinda Gates Foundation. S. Mishra was supported by a Royal College of Physicians and Surgeons of Canada International Travelling Fellowship.
Competing interests None.
Patient consent Obtained.
Ethics approval Ethics approval was provided by the Institutional Ethics Review Board, St John's Medical College, Bangalore, India.
Provenance and peer review Not commissioned; externally peer reviewed.
Linked Articles
- Whistlestop tour