Article Text
PDF
Abstract
Objectives The incidence of HIV and syphilis among men who have sex with men (MSM) in Europe has recently increased. Rapid point-of-care tests (POCTs) for syphilis can improve access to screening. The purpose of this study was to evaluate the performance of two syphilis POCTs compared with laboratory tests among MSM.
Methods The study was undertaken in Verona, Italy. Asymptomatic MSM, potentially exposed to syphilis, were enrolled prospectively. The POCTs evaluated were SD Bioline Syphilis 3.0 and Chembio DPP Syphilis Screen & Confirm Assay on both serum and fingerprick blood. The results of the POCTs were read by the naked eye by two independent readers and their concordance assessed.
Results A total of 289 MSM were enrolled in the study. Based on laboratory tests, 35 MSM (12.1%) were TPPA-positive alone and 16 (5.5%) were both Treponema pallidum particle agglutination test (TPPA) and rapid plasma reagin (RPR)-positive. The specificities of both POCTs were above 99% on both serum and fingerstick blood specimens, while sensitivities varied considerably. The sensitivity of the SD Bioline test was lower on fingerprick blood (51.4% and 54.3%, readers 1 and 2, respectively) compared with that on serum (80.0% and 82.9%). In contrast, the Chembio test exhibited similar sensitivity values for serum and fingerprick samples (57.7% and 64.0% on serum vs 65.4% and 69.2% on fingerprick for the treponemal component; 63.6% on both samples by both readers for the non-treponemal component). The positive predictive value ranged between 100% and 93.9% for the treponemal component of both syphilis POCTs, but was lower (76.3%–100%)%) for the non-treponemal component of the Chembio POCT. The negative predictive value surpassed 90% for both tests on both samples. The agreement between readers was very high (>99%).
Conclusion The diagnostic performance of the syphilis POCTs was lower than expected; however, considering the prevalence of syphilis among MSM, POCTs should be recommended to improve syphilis detection among MSM.
- syphilis
- men who have sex with men
- screening
- point-of-care testing
- evaluation study
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Background
WHO strongly recommends screening for HIV and syphilis, as well as for other STIs among most at-risk populations to reduce the burden of morbidity and mortality associated with undiagnosed and thus untreated infection.1 2
In Europe and in the European Economic Area, the main mode of HIV transmission is unprotected sexual intercourse between men. From 2004 to 2013, the number of new HIV cases among men who have sex with men (MSM) increased by 33%, and in 2013, 42% of all new cases were in this population.3 Furthermore, in this high-risk group, the incidence of syphilis has increased every year since 2008, and in 2014, 63% of new cases were reported among MSM.4
In Italy, available data suggest that in urban areas the HIV prevalence rate among MSM is approximately 10%.3 According to the findings of the EU-funded Sialon II project, an HIV prevalence of 9.6% was estimated among MSM in Verona.5 In addition, in the same study an overall treponemal seropositivity rate of 12.7% was detected and non-treponemal test positivity confirmed with a positive treponemal test (a better indicator of active disease) was 5.1%.5 These findings are consistent with the prevalence rates of HIV and syphilis among MSM reported elsewhere in Europe.3
Although HIV transmission can be influenced by many factors, the presence of a coexisting syphilis infection might lead to increased viral shedding through ulcers and an increased viral load as a result of the concomitant effect of syphilis on the immune system.6 7 Both HIV and syphilis may be asymptomatic for long periods but, if untreated, could lead to continued transmission and severe complications. This chain of events could be tackled through early testing and subsequent treatment. Moreover, the introduction of syphilis screening into existing HIV testing programmes would be cost-effective, time-saving and would have a considerable impact on the prevention of transmission, case finding and personal health.2
Since 2006, WHO has been actively working to promote a more efficient use of existing diagnostic tools for STIs, as well as to support a more efficient adaptation of such tools to different populations and settings. From this perspective, both development of and evaluation initiatives for new STI diagnostic technologies have been promoted to ensure appropriate performance and availability of these technologies to improve diagnostic services for those populations most at risk.2
Point-of-care tests (POCTs) for syphilis are promising tools to improve large-scale screening, especially among hard-to-reach populations, such as MSM, in different settings. The availability of test results within 20 min allows for prompt clinical decisions and reduces the loss to follow-up.8
Therefore, in recent years, a remarkable number of rapid diagnostic tests (RDTs) for syphilis and/or HIV have been developed and several national health systems have scaled up their use as part of their STI testing policies.9
Recently, the WHO STI POCT initiative published a target product profile for syphilis POCTs. According to this profile, a syphilis POCT should have a sensitivity of at least 80% and a specificity >90%. These standards ensure acceptable positive predictive value (PPV) and negative predictive value (NPV) and a major effect in terms of clinical utility in low-income, middle-income countries and, by extension, among hard-to-reach populations, where similar high rates of syphilis prevalence (3%–5%) have been reported.10
The diagnostic performance of some POCTs and their easy and rapid utility make them, under specific conditions, a practical alternative to standard laboratory testing methods.11 Nevertheless, the results obtained with POCTs can be influenced by epidemiological/environmental factors. With regards to the human component, the impact of the user’s characteristics, such as the ability to properly follow the rapid testing procedures (capillary blood taking, correct timing of adding the buffer and reading), could substantially interfere with the performance of the POCT. Performance, therefore, is dependent not only on the technical characteristics of the POCT itself, but also of human factors that can impact on the correct use and interpretation of the POCT.
To our knowledge, the human factor has not yet been comprehensively analysed as a key component of the testing procedure. While many performance evaluation studies of syphilis POCTs have been published, none of them has specifically targeted MSM as a target study population. In our study, the predictive positive and negative values of the POCTs could be estimated not only as a result of testing the sample of men included in the study, but also using the seroprevalence data derived from an integrated bio-behavioural survey recently carried out among the MSM population living in Verona.5
The purpose of this study was to evaluate the analytical and diagnostic performance (including human factors) of two syphilis rapid tests compared with laboratory-based gold standard tests when applied to an MSM population. In particular, laboratory-based treponemal tests were used in comparison with both the SD Bioline treponemal test and the treponemal component of the Chembio test. The Chembio non-treponemal component was compared with a laboratory-based non-treponemal test.
Methods
Study sites and population
Study participants were asymptomatic MSM, potentially exposed to syphilis as a result of risky behaviours, prospectively recruited from the Sialon II Respondent-Driven Sampling survey implemented in Verona, Italy,12 from 2013 to 2014 and MSM attending the Infectious Diseases Unit of the Verona University Hospital screening facility from 2015 to 2016. The prevalence of HIV, TPPA and dually TPPA/RPR seropositivity used for the evaluation of the performance of the POCTs was based on the results of an integrated bio-behavioural survey carried out among the Verona MSM population.5
Men or male-to-female transgenders, ≥18 years, who had sex with at least another man over the last 12 months and who provided witnessed written informed consent were included in the study. Participants could be enrolled in the study only once.
Study participants were given an automatically generated unique bar code to participate anonymously and to link participants to their own test results. A structured questionnaire was used to collect information about demographic and behavioural characteristics, whereas a specific form was used to collect data about the participant’s syphilis history.
The sample size was calculated at an expected prevalence of 10%. This sample size yielded 30 subjects with treponemal positivity, which achieves 85% power to detect a change in sensitivity from 0.58 to 0.85 using a two-sided binomial test and a >99% power to detect a change in specificity from 0.58 to 0.85 using a two-sided binomial test. The target significance level was 0.025 (Bonferroni correction). The actual significance level achieved by the sensitivity was 0.015 and achieved by the specificity was 0.019.
POCTs under evaluation
The tests evaluated in this study were the SD Bioline Syphilis 3.0 (Standard Diagnostics, South Korea) and the DPP Syphilis Screen & Confirm Assay (Chembio Diagnostic Systems, USA).
Both are immunochromatographic assays. SD Bioline Syphilis 3.0 test is a treponemal assay, which detects antibodies of all isotypes (IgG, IgM, IgA) against Treponema pallidum. Chembio’s DPP Syphilis Screen & Confirm Assay can simultaneously detect antibodies against treponemal and non-treponemal antigens.
Chembio RDT employs a unique combination of protein A and anti-human IgM antibody, which are conjugated to colloidal gold particles for the treponemal test. It uses a recombinant antigen of T pallidum and synthetic antigens for the non-treponemal test, separately bound to the membrane’s solid phase.13
In both tests, the presence of the treponemal and, when available, non-treponemal magenta-coloured lines was evaluated independently by the naked eye by two readers who were blind to each other’s results and to the clinical history of the study participants.
The POCTs under evaluation were partially donated by the manufacturers or purchased with external funding, namely from the 2008–2013 EU Public Health Programme, through which the Sialon II Respondent-Driven Sampling survey component has been funded. The manufacturers were not involved in any part of the study (study design, data collection, data analysis, data interpretation and writing of the paper).
Reference laboratory tests
These results were compared with those of the syphilis serological laboratory standard assays. For the treponemal component, two treponemal tests were used: the chemioluminescent assay (CLIA) (ADVIA Centaur Syphilis assay, Siemens Healthcare, Germany) and TPPA (SERODIA-TP·PA, Fujirebio Diagnostics, Sweden). For the non-treponemal component, RPR (Syphilis RPR test, HUMAN Diagnostics Worldwide, Germany) was used as reference test. According to the standard laboratory procedure, the titration for TPPA and RPR was also recorded.
Specimen collection, testing procedures and POCTs results reading
According to the international and local standard guidelines, pre-test and post-test counselling was provided to all participants.
The testing procedures, based on the manufacturers’ instructions for fingerprick whole blood, venepuncture whole blood and serum specimens, were strictly followed.
The required amount of capillary (manufacturers’ pipettes) and venous blood (5 mL) was collected by trained healthcare staff of the Verona University Hospital. With reference to obtaining fingerprick blood samples, the manufacturers’ instructions were followed step-by-step, such as wiping away the first drop of blood following pricking, collecting the required amount of capillary blood using the capillary pipette provided in both test kits and waiting 20 min (measured with a timer for each test) before reading the results.
In addition, a double reader method [Reader 1-Reader 2, (R1–R2)] was adopted. The readers were medical doctors and nurses in the clinical setting and lab technicians in the laboratory setting. All readers were specifically trained as described in the ‘Training and materials’ section.
According to the procedure, (i) R1–R2 assessed the RDT results blindly from one to another using two separate forms for recording the assigned result, (ii) neither R1 nor R2 was informed about the clinical history of the patient and (iii) R1 and R2 changed according to the setting (lab/clinical setting). Because of the potential reading bias due to his/her knowledge of the patients’ syphilis history, the counsellor involved in the pre-test/post-test counselling was always excluded from the result assessment of the counselled participant.
In the clinical setting, syphilis POCTs performed on fingerstick blood were read immediately. Blood tubes were sent, according to routine procedures, to the Microbiology Unit of the Verona University Hospital where they were centrifuged to obtain serum and to perform the laboratory-based syphilis serological tests. The specimens that could not be processed immediately were stored at 4°C and processed within 3–4 days to allow respondents to receive their syphilis serological results, using the bar code provided at enrolment. The evaluation of POCTs’ performance on serum at the Microbiology laboratory was carried out on batches of previously stored (−80°C) serum samples.
During the post-test counselling, in the case of a positive result of serological test for T pallidum infection (syphilis), treatment was made available to participants according to the local protocol.
Training and materials
In order to ensure a proper implementation of the research protocol and to standardise the human component in performing the POCTs, the following actions were carried out: (i) specific on-site prevalidation training of health professionals was undertaken for the reading procedures; (ii) the use of specific training materials (including picture examples of different bars of the RDTs to train readers to ensure correct reading); (iii) a coaching programme to ensure proper monitoring of the validation exercise and (iv) the use of a specific set of standardised materials for the readers (eg, posters summarising the POCT procedures displayed in the blood taking settings, a table mat summarising the procedures nurses had to follow when performing the POCTs, posters outlining the reading procedures of the testing always visible in the reading rooms for R1–R2, clarifying the different interpretations of the tests’ results).
Ethics
Research protocols were submitted to and approved by the local Ethics Committee (Comitato Etico per la Sperimentazione Clinica delle province di Verona e Rovigo). Protocols were also approved by both the WHO Research Project Review Panel (RP2) and the WHO Research Ethics Review Committee (ERC) before initiating data collection.
Anonymity and confidentiality of respondents’ data were guaranteed in line with the local standards, while a bar code system has been instituted to allow an appropriate link between the different types of data collected for each individual (demographic and behavioural information, biological samples).
In particular, to comply with all ethical and legal aspects and to minimise the risk of mistakes, participants were informed both during the pre-counselling session and through the informed consent form that only the results of laboratory-based tests would be used to direct patient management. These laboratory results were made available to each participant within a few days of enrolment. Participants were also provided with a phone number to be used (i) to know whether the lab results were available and (ii) to book an appointment for collection of results.
In addition, in cases of positive results, MSM received further information about the infection and the treatment during post-test counselling. According to local procedures, participants could decide to attend the Infectious Diseases Unit of the Verona University Hospital for clinical follow-up or another centre of care. In each case, prompt referral was guaranteed. The treatment provided was in line with national guidelines and standards.
Finally, in line with the protocol, samples were stored at −80°C at the Microbiology Unit of the Verona University Hospital for 1 year after the end of the study. Samples taken without written informed consent were not used for testing and destroyed immediately.
Statistical analysis
Sensitivity, specificity, PPV and NPV for each rapid test were estimated comparing the POCT results with the gold standard lab tests results.14 The result of each rapid test was compared with the respective gold standard, namely POCT treponemal versus laboratory-based treponemal and POCT non-treponemal versus laboratory-based non-treponemal tests. 95% CIs were also estimated (logit transform).
The concordance between R1–R2 readings was estimated by calculating percentage agreement (concordance) and Cohen’s κ (κ for binary variables).15
Cohen’s κ represents a measure of inter-rater agreement, ranging from −1 to +1, where 0 is the level of agreement that can be expected in case of random chance. According to the literature, thresholds for κ are usually categorised as follows: <0.0 (poor agreement), 0.0–0.2 (slight), >0.2–0.4 (fair), >0.4–0.6 (moderate), >0.6–0.8 (substantial) and >0.8–1.0 (almost perfect agreement).16
STATA V.14.2 was used for all analyses.
Results
Study population
A sample of 289 MSM was enrolled in the evaluation study. The mean age of the participants was 31.4 (median: 29; SD 9.2; min 18, max 65). Among the overall sample, 20 individuals reported a previous syphilis diagnosis before study enrolment (6.8%, IC95% 4.4–10.5). All participants provided bio-behavioural information, fingerstick whole blood for syphilis rapid testing, venous whole blood for HIV and syphilis serological testing.
Results of laboratory-based testing
Based on CLIA (ADVIA Centaur Syphilis assay) and TPPA (SERODIA-TP·PA) testing, 35 samples were found to be positive on treponemal testing (12.1%, 95% CI 8.8 to 16.4), while 16 samples were found to be reactive on RPR testing (5.5%, 95% CI 3.4 to 8.8). All RPR (Syphilis RPR test, HUMAN Diagnostics Worldwide)-positive samples were also TPPA positive. No discordance between the results of the two laboratory treponemal tests was found.
Performance of the POCTs
In line with the availability of the two POCTs, the following testing was performed: 289 (100%) SD Bioline Syphilis 3.0 and 227 (78,5%) Chembio DPP Syphilis Screen & Confirm Assay on fingerprick whole blood, 287 (99.3%) SD Bioline and 205 (70.9%) Chembio DPP on serum.
POCTs sensitivity and specificity, as well as PPV and NPV, compared with the golden standard laboratory tests, varied considerably according to the tests and biological samples (see table 1).
Point-of-care test sensitivity and specificity, positive predictive value (PPV) and negative predictive value (NPV), per specimen (estimated syphilis prevalence: 12.7%; estimated active syphilis prevalence: 5.1%), according to the assessment of the readers (R1, R2), compared with the lab-based golden standard (TPPA for the treponemal component, RPR for the non-treponemal component)
The SD Bioline treponemal test carried out on serum yielded a sensitivity of 80.0% for reader 1 (R1) and 82.9% for reader 2 (R2). With regards to specificity, the performance was 100% for R1 and 99.6% for R2. The PPV of the Bioline test was 100% for R1 and 96.8% for R2, while the NPVs were 97.2% and 97.6%, respectively.
When considering the performance of the Bioline test on fingerstick blood, the sensitivities were 51.4% and 54.3% for R1 and R2, respectively, while specificity was 100.0% for both readers. The test showed high values for PPV (100% for both readers) and NPV (93.4% and 93.8%).
With regards to the Chembio DPP treponemal POCT on serum, the sensitivity values were 57.7% and 64.0% for R1 and R2, respectively, while the specificity values were 99.5% and 99.4. The PPVs (93.9% and 94.4% for R1 and R2) and NPVs (94.2% and 95%) were also acceptable.
On whole blood (fingerprick), the sensitivities of the Chembio DPP treponemal POCT were 65.4% and 69.2%, for R1 and R2, respectively, and a specificity of 99.5% was calculated for both readers. PPVs of 95.1% and 95.2% and NPVs of 95.2% were calculated for R1 and 95.7% for R2.
For the non-treponemal component of the Chembio test on serum samples, a sensitivity of 63.6% was recorded for both R1 and R2 and specificities of 99.5% for R1 and 99.0% for R2. PPV estimates were 86.8% for R1 and 76.3% for R2, while the NPV was 98.1% for both readers.
The results on fingerstick samples (whole blood) showed a sensitivity of 63.6% for both R1 and R2 and specificities of 100.0% and 99.5%, respectively. The PPVs were 100% and 87.9%, while an NPV of 98.1% was calculated for both readers.
RPR and TPPA titration values
In table 2, only positive treponemal reference test has been presented and compared with their respective POCT results. In part A of the same table, TPPA and RPR-positive cases are shown, while in part B, TPPA- positive and RPR-negative cases are listed. Considering only part A, among the 289 MSM recruited in this study, 16 cases were identified as TPPA+/RPR+ (prevalence: 5.5%; 95% CI 3.4 to 8.8). Of these 16 individuals, 4 were also HIV-seropositive. The clinical evaluation provided additional information on the status of the study participants’ T pallidum infections. Six individuals had a syphilis history and treatment, whereas the remaining 10 subjects had not previously been diagnosed with a T pallidum infection. The four HIV-positive individuals were among those who did not have a syphilis history and treatment. In all 16 cases, both the POC treponemal tests (SD Bioline and Chembio treponemal component) yielded a positive result on serum (100%). As far as the Chembio non-treponemal component is concerned, among the TPPA+/RPR+ cases, the test was carried out only on 10 individuals out of 16. In this subsample, three cases would have been missed if only the POCT had been used, considering a RPR titre ≤1:4. One of the cases missed by the test was among the individuals with a syphilis history (possible serofast state), while the other two cases had not been diagnosed previously.
Syphilis rapid tests results on serum and fingerprick blood among CLIA and TPPA-positive individuals: RPR-positive (part A, 16 samples), RPR-negative (part B, 19 samples)
When whole blood specimens were analysed, SD Bioline missed one of the TPPA+/RPR+ cases (TPPA 1:5120, RPR 1:2 and no syphilis history and treatment), while the treponemal component of the Chembio test did not miss any cases (although carried out on only 10 out of 16 participants), while its non-treponemal component missed three cases. The very same cases missed on fingerprick blood specimens were also those missed when serum specimens were tested.
Part B of table 2 shows that the higher the TPPA titre, the better the performance of the POCTs’ treponemal component. This is particularly evident when performance was assessed on fingerprick blood specimens.
R1–R2 concordance (Cohen’s κ) and agreement on RDTs result assessment
Table 3 shows a very high agreement between R1 and R2 for both POCTs through both concordance percentages and Cohen’s κ, the latter ranging from 0.91 to 0.97.16 With regards to the SD Bioline POCT, κ showed an almost perfect agreement between R1 and R2 on both serum samples (κ=0.96) and blood samples (κ=0.91), as well as a high percentage of agreement value (99.30% and 98.96%).
Agreement between reader 1 and reader 2: agreement and Cohen’s κ values (per type of specimen)
For the treponemal component of Chembio DPP, κ value was 0.97 for both serum and blood, and an agreement of 99.51% (serum) and 99.56% (blood) was reported. With regards to the non-treponemal tests, κ values were 0.94 and 0.93 and an agreement of 99.50% and 99.56% was estimated.
Discussion
The use of POC tests is becoming increasingly frequent worldwide in clinical and low threshold testing settings.
The use of syphilis rapid tests has been extensively reported in the literature among different types of populations, particularly among vulnerable populations (eg, pregnant women, sex workers, injecting drug users (IDUs)); however, to our knowledge, MSM have been less studied. Considering surveillance data on syphilis among MSM, the use of syphilis POC rapid tests can potentially represent an important clinical and public health measure to diagnose syphilis cases among hard-to-reach segments of this target population. In this study, a homogeneous sample was enrolled. All participants were MSM living in the same city and very likely belonging to the same community. In addition, the prevalence of TPPA+ and TPPA+/RPR+ for calculating PPV and NPV was based on a large bio-behavioural survey carried out among MSM in Verona in 2013–2014,5 12 and therefore, with this more robust prevalence estimate, providing a more valid indication of the real-life POCTs utility among this population. However, the small sample used for this validation study probably represents the most important limitation in terms of result generalisability and performance assessment; therefore, the results of this study should be cautiously interpreted.
Aware of this limitation, the study team used several methodological and operational features to assure appropriate data collection and proper interpretation of the results. The real-life clinical implementation of this study, with the use of fingerprick blood in the clinical setting and serum in the laboratory, provided a good opportunity to assess the diagnostic performance of the two syphilis rapid tests. The performance on both biospecimens resulted in lower-than-expected performance characteristics compared with previous reports in the literature and in the technical specifications provided by the manufacturers. This is particularly evident when fingerprick blood samples were tested. In addition, considering the titration provided by the laboratory tests, it seems that for TPPA titres >1:1280 the misclassification rate for the two POCTs, both on serum and blood, was extremely low. The same pattern can be seen for the non-treponemal test, where with RPR titres ≥1:8 there was virtually no misclassification compared with the non-treponemal component of the Chembio test. As suggested by some studies in prenatal populations, the detection of RPR titres≥1:8 could be very important for pregnant women as this has been significantly associated with adverse pregnancy outcomes.17 Unfortunately, for other at-risk populations, the use of a decision threshold titre is more problematic owing to possible recent exposure to infection and therefore incubation of early disease. The interpretation of low RPR titres can be even more difficult when patients have previously received treatment with a course of potentially treponemocidal antibiotics for other indications. In fact, 16%–20% of men with a history of adequately treated syphilis can present lifelong low positive RPR titres (ie, they are serofast).18
Based on their STI history, participants with a treponemal-positive result were classified as either a newly diagnosed or a serofast case. The non-treponemal Chembio test missed three TPPA+/RPR+ cases; however in terms of clinical utility, at least one case was serofast and therefore negligible in terms of transmission risk and individual health outcome, while the remaining two were possibly cases of late-latent disease. In these latter cases, the clinical implication for the missed treatment could have been relevant for the patient, but not for the community since the transmission risk of a late-latent syphilis is extremely low.19 It is interesting to note that there was no discordance in these cases when serum and blood test results were compared and that none of the MSM who could be considered serofast was HIV- infected.
Unfortunately, due to unexpected logistical reasons (expiration time and delay in new test procurement), the Chembio POCT results were available only for a subsample of MSM and this represented a further limitation in the interpretation of our findings. However, the TPPA titre and RPR comparisons seem to be in line with the results of previous studies.20–22 Additional evaluation studies should be implemented, including a more detailed assessment on the potential impact of different cut-off (TPPA and RPR titre) on the Chembio DPP Syphilis Screen & Confirm Assay performance, within a POCT approach.
Sensitivity and specificity varied considerably across tests and biospecimens, as well as PPV and NPV. While the analytical performance of RDTs (sensitivity and specificity) is not affected by the characteristics of the population among which the tests are used, PPV and NPV are strongly influenced by the prevalence of the infection in the target population. In our study, the predictive values seem to be acceptable considering the specific evaluation setting and the reference population (members of the MSM community in Verona). This leads us to consider the use of syphilis POCT as potential alternative to standard methods to improve screening practice, particularly outside hospital settings. Additional evaluations are certainly needed to further assess the potential replacement of standard tests with the POCT approach in different scenarios and the potential impact of such a shift.
As expected the performance of the two POCTs evaluated here proved to be inferior to those of the standard laboratory tests; however, when used strategically, in settings where venepuncture is not safe or impossible to perform, or when the population is very mobile or for legal reasons not entitled to receive the standard medical assistance (ie, migrants, illegal migrants, sex workers, drug users), these POCTs can be extremely useful in identifying syphilis cases requiring medical assistance and treatment. Furthermore, even though both syphilis POCTs under evaluation seem to be promising, the Chembio POCT seems to be more informative, particularly on fingerprick blood. In a population with such a high prevalence of TPPA positivity, the availability of a rapid syphilis test with a non-treponemal component could be very useful as it allows clinicians to better distinguish between previously treated syphilis and active disease.
This does not imply that a POCT with only a treponemal component is not useful. The treponemal-only test, together with the patient’s clinical examination and history, would allow the clinician to identify cases never treated, to plan additional testing in subjects potentially non-adequately treated or, in case of a previous syphilis infection, to treat them despite the marginally hazardous overtreatment risk. The overtreatment risk can be balanced with the benefits of the transmission risk reduction to partners and the breaking of the transmission chain within the community. At a personal level, even though the subject has to be treated with penicillin, adverse events associated with treatment are fortunately limited.
The procedures adopted for this syphilis test validation addressed also the aspect of quality assurance of the test reading in order to guarantee an adequate assessment of the POCTs performance. In accordance with the literature,23 for reducing the degree of human interpretation and subjectivity, an ad hoc training and supporting documentations for staff, as well as a coaching activity were developed. This can be considered a relevant feature of this POCT study compared with other studies, where the human component was not specifically accounted for or simply considered a negligible factor. According to our results, once the POCT procedures (as described in the ‘Methods’ section) are strictly followed, the human component does not represent a relevant source of inaccuracy. This is confirmed by the high level of concordance between the readers (Cohen’s κ for R1–R2).
To conclude, we believe that this study could be relevant for setting the agenda of future validation studies. Steps and procedures for the future use of POCTs in clinical and community-based testing services have been piloted, assessed and improved for clinical purposes.
From a methodological perspective, despite the fact that POCTs are easy to use, this technology should be linked to specific training for users together with the use of supporting documentation (eg, posters, procedural dashboard) to reduce misinterpretation of the results due to human subjectivity. From a clinical viewpoint, we can consider the human component as having no significant impact on the performance of the POCTs if the healthcare staff is properly trained on POCT use. The complex experimental procedures used for this validation study have not fully allowed the staff to appreciate the flexibility of POCTs in meeting the diversity of medical needs that can make these tools invaluable for this population. The possibility of performing the tests in a variety of locations, including saunas, bathhouses and other non-conventional facilities, makes the POCTs very attractive and useful for certain healthcare systems. Despite the relatively low sensitivity showed by the POCTs in this study, in our opinion they can provide important diagnostic and treatment opportunities among the MSM population studied. In fact, considering the high specificity (close to 100%) the provision of immediate on-site treatment, guided by a clinical and epidemiological evaluation (syphilis history, previous treatment, exposure assessment), could represent a real benefit both for the individuals and for the community as a whole, with a consequent reduction of infection transmission and overall burden of disease.
Key messages
The point-of-care test (POCT) technology should be linked to specific training for users and the adoption of supporting documentation to reduce misinterpretation of the results due to human subjectivity.
The potential of POCTs can only be realised if properly used.
Although syphilis POCTs may lack some sensitivity compared with laboratory-based tests, their use in non-conventional settings, together with on-site treatment, could contribute to a significant reduction in disease transmission among high-risk men who have sex with men populations.
Acknowledgments
The authors would like to thank all the survey participants for their contribution, as well as the colleagues from the Virology and Microbiology Unit, the Infectious Disease and Tropical Medicine Unit and the Regional Coordination Centre for European Project Management of the Verona University Hospital, the Department of Reproductive Health and Research of WHO, the Department of Clinical Research, International Diagnostics Centre of the London School of Hygiene and Tropical Medicine and the Faculty of Medicine and Health Sciences of the University of Ghent for the support, commitment and enthusiasm in the project implementation.
References
Footnotes
Handling editor Jackie A Cassell
Contributors MM, LG, AZ, MC, IT and RWP participated in the design of the survey and the organisation and implementation of the survey in the survey cities. This analysis was conceived by MM, IT and RWP. Data were analysed by MM, LG and MC. The first manuscript draft was jointly written by MC, LG, MM and AZ. MC, LG, MM, KB, PDB, AZ, IT, RWP, JK, GC, RB contributed writing to the following drafts. All authors read and approved the final manuscript.
Funding This manuscript is partially based on data collected in the context of the Sialon II project, co-funded under the Second Programme of Community action in the field of health (2008–2013) (Work Plan 2010).
Disclaimer The author(s) is(are) staff member(s) of the World Health Organization. The author(s) alone is(are) responsible for the views expressed in this publication and they do not necessarily represent the views, decisions or policies of the World Health Organization.
Competing interests None declared.
Patient consent Obtained.
Ethics approval WHO Research Project Review Panel (protocol number A65), WHO Research Ethics Review Committee (protocol number RPC557) and Local Ethic Committee - Comitato Etico per la Sperimentazione Clinica delle province di Verona e Rovigo (protocol numbers 272CESC, CE2341, CE2408).
Provenance and peer review Commissioned; externally peer reviewed.