Article Text

PDF

Sensitivity requirements for the point of care diagnosis of Chlamydia trachomatis and Neisseria gonorrhoeae in women
  1. P Vickerman1,
  2. C Watts1,
  3. M Alary2,
  4. D Mabey1,
  5. R W Peeling3
  1. 1London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT, UK
  2. 2Department of Social and Preventive Medicine, Laval University, Quebec, Canada
  3. 3Sexually Transmitted Disease Diagnostic Initiative, TDR/WHO, Geneva, Switzerland
  1. Correspondence to:
 Peter Vickerman, London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT, UK; 
 Peter.Vickerman{at}lshtm.ac.uk

Abstract

Background/objectives: Most current tests for Neisseria gonorrhoeae and Chlmydia trachomatis require the support of a laboratory, and results are not usually available before the patient has left the clinic. This delay can lead to patients not returning for treatment and may allow further STI transmission to occur. Current rapid point of care (POC) STI tests and the syndromic approach are less sensitive than gold standard tests, but allow treatment at the initial visit. This study estimates the required sensitivity of POC tests that ensures as many STIs are averted as with current gold standard tests.

Methods: A mathematical model is developed to compare the STIs averted using different STI diagnostic methods. Data from Benin, South Africa, Tanzania, and England are used to explore how the required sensitivity differs between settings.

Results: The required sensitivity of a POC test is low if there is significant STI transmission during the delay in treatment for the gold standard test and/or few women return for treatment. For example, the required sensitivity of a POC test for C trachomatis is 50% (gold standard test sensitivity is 90%) if either 55% return for treatment and there is no STI transmission, or 80% return for treatment and 50% of infected women infect their partner during the delay in treatment. Furthermore, in these settings a POC test of moderate sensitivity can lead to significantly more STI averted than the gold standard test.

Conclusions: These results support the use of moderate sensitivity POC tests in scenarios where many women will not return for treatment, and in populations where the delay in treatment would result in significant STI transmission.

  • women
  • Chlamydia trachomatis
  • Neisseria gonorrhoeae

Statistics from Altmetric.com

The diagnosis of genital gonococcal and chlamydial infections among women in low income countries remains a major challenge.1 Although there are many tests available to detect chlamydial and gonococcal infections, most require laboratory facilities, and so are costly; and the results are not usually available before the patient has left the clinic. Delays of 10–14 days are common between the time that specimens are collected from patients and when patients return for their test results and treatment.2–4 Patients often fail to return for their results leading to fewer patients receiving treatment, with reductions of 58–80% being documented in the literature.2–4 During the delay between testing and treatment further STI transmission may occur, particularly among individuals with multiple sexual partners or in STI discordant partnerships. Thus, although laboratory tests are highly sensitive,5 in practice their high costs and technical requirements make their routine use difficult, and compromise their potential impact on STI control and management.

Consequently, in most resource limited settings the World Health Organization recommends the use of the syndromic approach for the management of urethral discharge in men and vaginal discharge and lower abdominal pain in women.6 While the syndromic approach appears to be satisfactory in men,7 it has several important limitations in women. The recommended flow charts apply to women attending health centres with symptoms while a large proportion of genital gonococcal and chlamydial infections are asymptomatic in women. 1,8 Furthermore, studies evaluating the syndromic approach in women have shown that it generally has a poor sensitivity (30–80%) and specificity (40–80%) for the diagnosis of Neisseria gonorrhoeae and Chlamydia trachomatis, with the sum of sensitivity and specificity almost never exceeding 120%.7–11 These results were obtained both for symptomatic women7,9,10 and when flow charts were adapted and used as screening algorithms.8,11 Such low sensitivities and specificities result in many false diagnoses, massive overtreatment, and many STIs remaining untreated.

The development of cheap, reliable, non-invasive, and easy to use tests for the diagnosis of N gonorrhoeae and C trachomatis among women in developing countries could greatly improve STI control. Such tests would permit the screening of asymptomatic women and would reduce the false diagnosis and overtreatment of symptomatic women inherent in the currently recommended syndromic management algorithms. A rapid point of care (POC) diagnostic test would also enable women to be diagnosed and receive treatment at a single visit as is the case with the syndromic approach. As an incentive, in 1994 the Rockefeller Foundation announced a prize of $US 1 million for the development of such a test. The specifications were a sensitivity of 95%, a specificity of 99%, and the use of non-invasive samples not requiring gynaecological examination with a speculum. Despite substantial investment of research energies, no such test has been developed. The sensitivity of current POC tests for C trachomatis in women range from 55–85% for high prevalence populations3,12 and 25–49% for low prevalence populations.2,13 Currently available POC tests for N gonorrhoeae have not been evaluated, other than microscopic identification of Gram stained endocervical or urethral smears. The sensitivity of microscopic identification is 37–70% for women.5 Despite their low sensitivity, over time it has been recognised that even suboptimal POC tests could have public health value (Wellcome Trust symposium on STD Diagnostics in the Developing World in 2000). Indeed, Mukenge et al and Gift et al have shown that a low sensitivity POC test (Biostar) for chlamydial infection and even the syndromic approach for cervical infection can result in more infected women being treated than a laboratory test involving a return visit.3,4

This paper explores the public health implications of using POC tests in different settings. The analysis builds upon a study by Gift et al that used a mathematical model to estimate the minimum required sensitivity of a POC test to ensure that as many gonococcal and chlamydial infections are treated as when using current gold standard tests in the United States.3 This study develops their mathematical model to estimate the minimum required sensitivity of a POC test that leads to more gonococcal and chlamydial infections being averted than the gold standard test by preventing the delay between diagnosis and treatment and the associated loss to follow up and STI transmission. The findings are applicable to the syndromic approach, which can be viewed as a POC test because treatment is provided at the initial consultation.

METHODS

Model description

Gift et al3 showed that a POC test would have more impact than the current gold standard test if the number of infected people being effectively treated as a result of a POC test is greater than the number being tested, returning for treatment and being effectively treated, using the current gold standard. Assuming the efficacy of treatment is equal in both scenarios, the sensitivity of the POC test must satisfy the following equation:

Math

where Sr is the sensitivity of the POC test, Sg is the sensitivity of the gold standard test, and μ is the proportion of clients that return for their test result (defined as the “return rate”). This illustrates how the required sensitivity of the POC test is directly related to the return rate for the gold standard test. For example, a POC test compares favourably with a gold standard test with 50% return rate if it is more than half as sensitive as the gold standard test. However, equation 1 does not include the STI transmission that may occur during the delay between testing and treatment when using the gold standard test (defined as the “delay in treatment”).

If, on average, each person infected with an STI infects a specific number of sexual partners (Y) during the delay in treatment, then the number of STI averted by the POC test (Ar) is the sum of the number of STI effectively treated and the number of STIs that would have occurred as a result of onward transmission before the patient returned for the result of the gold standard test:

Math

where E is the effectiveness of the STI treatment used. Conversely, the number of STI averted from using the gold standard test (Ag) is simply the number of STI effectively treated:

Math

For the POC test to avert more STIs than the gold standard test, Armust be greater than Ag, and so the sensitivity of the POC test must satisfy the following threshold equation:

Math

In practice, the average number of people an STI infected person infects during the delay in treatment (Y) can be estimated using an established mathematical equation that describes the probability of STI transmission over a fixed period of time,14 and so equation 4 can be rewritten as:

Math

where m is the average number of sexual partners per unit time (excluding the sexual partner they were infected by); y is the proportion of sexual partners who are already infected with the STI; n is the average number of sex acts per partnership per unit time; β is the STI transmission probability per unprotected sex act; e is condom efficacy per sex act; f is the average consistency of condom use; and T is the delay in treatment.

Equation 5 only considers the primary transmission of the STI between the women accessing STI services and their partners. It does not consider the subsequent transmission that would occur from these partners. If this were incorporated, the sensitivity threshold for the POC test would be decreased further.

Scenarios considered

The model in equation 5 can be used with behavioural, epidemiological, and STI service specific data to illustrate how the required sensitivity of POC tests differs between settings. We consider six scenarios, reflecting different levels of sexual activity and STI prevalence (table 1). Three of the scenarios are from sub-Saharan Africa, where the syndromic approach is primarily used. Two scenarios are from the United Kingdom, where laboratory tests are the recommended protocol,5 but where a modified syndromic approach (using symptoms, risk profiles, and microscopy) may also be used. In these clinics laboratory tests are used to confirm the syndromic diagnosis and to enable asymptomatic cases to be diagnosed and asked to return for treatment.15

Table 1

Scenario data inputs, estimates of the average number of males infected with an STI by each infected woman during the delay in diagnosis and treatment (Y), and the required sensitivity of the POC test (Sr)

The scenarios presented have been selected to crudely represent transmission from sex workers to their clients in two cities in Africa (data from Cotonou, Benin16 and Virginia, South Africa17); transmission in long term steady partnerships (data from Mwanza region, Tanzania18,19); transmission from sex workers to their clients in the United Kingdom (data from GUM clinic in London20,21); and transmission from sexually active women attending a GUM clinic (data from London22,23). For the “UK sex worker” scenario we assume that the sex workers usually acquire STIs from their non-paying partners20 and consider the subsequent STI transmission to their paying clients. The UK scenarios are included to illustrate the possible importance of low sensitivity POC tests in settings where gold standard laboratory tests are routinely used. For the two scenarios that focus on “low risk” women the number of sexual partners during the delay in treatment is reduced by one because they were infected by one of their sexual partners. For the two African sex worker scenarios we consider STI transmission to their paying clients and assume that they were either infected by a non-paying partner or a paying client that they are unlikely to see again during the delay in treatment.

Model inputs

The model requires estimates for the per sex act probability of N gonorrhoeae and C trachomatis transmission from females to males, of which there are a number of estimates from the published literature. For N gonorrhoeae the probabilities range from 0.19 to 0.53.24,25 For C trachomatis, three studies produced point estimates of 0.108, 0.09, and 0.14 per sex act,26–28 but did not provide confidence bounds around their estimates. For our analysis we averaged the point estimates from each study to produce a transmission probability of 0.3 per sex act for N gonorrhoeae and 0.11 for C trachomatis. A sensitivity analysis was undertaken to explore the robustness of the projections, with a range of 0.19–0.53 being used for N gonorrhoeae, and 0.07–0.19 for C trachomatis. In the absence of confidence bounds the range used for C trachomatis is taken to have the same relative magnitude as for N gonorrhoeae.

We assume that condom use reduces the per sex act probability of HIV transmission by 90–95%.29 Because of limitations in data,30 we assume that condoms are equally effective at preventing the transmission of C trachomatis and N gonorrhoeae. A point estimate of 90% was used and a sensitivity analysis was undertaken to explore the robustness of the projections to variations in the per sex act effectiveness of condoms from 90% to 95%.

The model also required inputs for the sensitivity, average delay in treatment, and the return rate for the gold standard test. The sensitivity of gold standard tests for C trachomatis (nucleic acid amplification tests) is approximately 88–96% and 83–95% for N gonorrhoeae (culture identification).31,32 We assume that the sensitivity of the gold standard test is 90% for C trachomatis and N gonorrhoeae in our analysis.

There are few published data about the length of delay between testing and treatment for laboratory STI tests and the percentage returning for treatment. Published studies from Benin and the United States report delays between testing and treatment ranging from 10 days to more than 14 days,2–4 with the return rate ranging from 57.6% to 80%.2,4 In our analysis we use conservative estimates for both these variables, assuming that 80% return for treatment with an average delay of 10 days.

RESULTS

Figure 1 shows the relation between the required sensitivity of a POC test for chlamydial or gonococcal infection, the percentage of people returning for their test results (return rate), and the average number of people infected with C trachomatis or N gonorrhoeae by each infected woman during the delay in treatment (using equation 4).

Figure 1

Required sensitivity of a POC test for N gonorrhoeae and C trachomatis, for different return rates for the gold standard test, and for different numbers of males infected with N gonorrhoeae and C trachomatis by each infected woman during the delay between testing and treatment (Y). It is assumed that the gold standard tests for C trachomatis and N gonorrhoeae are 90% sensitive.

From figure 1it can be seen that the required sensitivity of a POC test is low if the return rate is low and/or there is further STI transmission during the delay in treatment. For example, if 50% of women return for their test results, and if half the women infected with C trachomatis infect another person during the delay in treatment, then the required sensitivity is 29%. If fewer women infect their partner during the delay in treatment, or if their return rate is higher, then the required sensitivity of the POC test is greater. However, if the sensitivity of a POC test is greater than 60% then more chlamydial infections will be averted even if the return rate is 100%.

Equation 5 illustrates how the required sensitivity of a POC test is inherently dependent on the underlying patterns of sexual behaviour, condom use, and STI prevalence of the groups accessing services. In general, the required sensitivity decreases for groups with a greater rate of sexual partner change, higher frequency of sexual acts, and lower condom use.

For each scenario, table 1 shows the average estimated number of males infected with C trachomatis or N gonorrhoeae by an infected woman during a 10 day delay in treatment, and conservative estimates of the required sensitivity of the POC test assuming an 80% return rate after 10 days. In addition, table 2 shows estimates of the percentage increase in the number of STI averted from using a POC test with different sensitivities compared to using the gold standard test. Both tables also show the uncertainty in the model predictions as a result of uncertainty in the STI efficacy of condoms and the transmission probability of N gonorrhoeae and C trachomatis.

Table 2

The percentage increase in the number of STI averted from using a POC test of sensitivity 25%, 50%, or 75% instead of using the gold standard STI test in the scenarios presented in table 1*

The results in tables 1 and 2 illustrate that in settings where there is substantial potential for further STI transmission, such as among the sex workers in Benin or South Africa, even a low sensitivity method of detecting STIs (sensitivity 20–50%) could have greater impact than current gold standard tests if the return rate was 80%. Indeed, if a POC test of sensitivity 50% is used instead of the gold standard then at least 41% more chlamydial and 122% more gonococcal infections would be averted among the sex workers in South Africa.

In settings where there is some potential for further transmission, such as for the UK sex worker scenario, the required sensitivity of a POC test is greater than for the African sex worker scenarios and is dependent upon both the return rate for the gold standard test result, and factors affecting the extent of further STI transmission. Indeed, for the transmission of C trachomatis, the gains from using a POC test will be small unless it has a sensitivity greater than 75% (table 2) or the return rate is less than 80%. Conversely, the required sensitivity of a POC test for N gonorrhoeae is lower in this scenario and so the possible gains from using a POC test are greater.

In settings where there is little potential for further STI transmission, such as for the women with steady partners from Tanzania, or women attending GUM clinics in the United Kingdom, the required sensitivity of the POC test is quite high and primarily dependent upon the product of the sensitivity of the gold standard test, and the return rate for the test. The gains from using a POC test in these settings are very low unless the POC test is almost as sensitive as the gold standard test and/or the return rate is very low.

Uncertainty in the efficacy of condoms in preventing STI transmission, and in the transmission probability ofN gonorrhoeae and C trachomatis affects our estimates of the number of men infected by each infected woman during their delay in treatment. This results in uncertainty in the estimated required sensitivity of a POC test (table 1) and the percentage change in the number of STIs averted as a result of using a POC test instead of the gold standard test (table 2). The magnitude of the uncertainty is small when there is little potential for further STI transmission and increases as the potential becomes greater. For example, for an 80% return rate and a 10 day delay in treatment, the required sensitivity of a N gonorrhoeae POC test for the South African sex worker scenario ranges from 19–27% due to the uncertainty in condom efficacy and STI transmission probability (table 1). This uncertainty also results in uncertainty about the predicted impact of a POC test of known sensitivity such that an N gonorrhoeae POC test of 75% sensitivity is predicted to avert between 175% and 293% more infections than the gold standard test. However, it is important to note that this uncertainty does not affect the broad conclusions that can be drawn from tables 1 and 2.

In addition to the uncertainty in these epidemiological parameters, there is likely to be considerable variability in the value of some context specific parameters used in the analysis, such as the return rate for the gold standard test result, the duration between testing and treatment, and the sensitivity of the gold standard test. For this reason we have aimed to produce fairly conservative estimates of the required sensitivity of POC tests by using fairly high estimates for the return rate (80%) and low estimates of the delay in treatment (10 days) in our analysis.2–4 In practice, in many scenarios the return rate will be much lower, and will lead to lower sensitivity requirements for the POC test. For example, if we assume that the return rate among the sex workers in Benin is not 80% but 42% as observed by Mukenge et al4 then the required sensitivity of a POC test for C trachomatis decreases from 43% to 23%. Likewise, if the delay in treatment is longer than 10 days, then there will be a greater potential for STI transmission, and so lower sensitivity requirements. For example, if the delay in treatment for the sex workers in Benin is 20 days instead of 10 days, then the required sensitivity of a POC test for C trachomatis decreases from 43% to 31%. Lastly, if the sensitivity of the gold standard test for either STI is lower than 90% then the required sensitivity of a POC test would be lower and vice versa. For example, if the sensitivity of the gold standard test for N gonorrhoeae is 83% instead of 90% then the required sensitivity of a POC test decreases from 26% to 24% for the sex workers in Benin, whereas if the sensitivity of the gold standard test is 95% then the required sensitivity increases to 28%. In general, the required sensitivity of a POC test increases proportionally to the sensitivity of the gold standard test and the return rate and decreases proportionally to the inverse of the delay in treatment.

DISCUSSION

The study has estimated the minimum required sensitivity of POC STI tests that results in as many STIs being averted as if the gold standard STI test had been used. The results provide estimates for the sensitivity requirements of future POC tests and give criteria for where currently available POC tests can best be used. The findings illustrate that the required sensitivity of a POC test is highly dependent upon the proportion of women that return for the result of a gold standard test, and the potential for further STI transmission during the delay in treatment.

Our results support the use of POC tests in scenarios where it would be difficult to ensure a high return rate, and in populations where there is potential for further STI transmission during the delay in treatment from using laboratory STI tests. These findings highlight the possible value of currently available POC tests for C trachomatis in such settings.2–3,12,13

As POC STI tests are currently expensive, this is of most relevance to industrialised countries. Indeed, given that countries such as the United Kingdom are currently experiencing an epidemic of C trachomatis in young females,33 increased priority needs to be given to the possible incorporation of POC tests within the national guidelines on STI treatment5 and the forthcoming C trachomatis screening programme in the United Kingdom,34 as their use would enable symptomatic and asymptomatic STIs to be treated promptly without massive overtreatment and loss to follow up. However, the appropriate protocols for doing this should be carefully developed as our results also illustrate that current POC tests for C trachomatis are unlikely to have greater impact than gold standard tests when used for diagnosing women with low numbers of sexual partners unless there is a low return rate. For example, pretesting assessments (based on assessments of risk and likelihood to return) could potentially be used to identify women who should be diagnosed using a POC test, and women who can be diagnosed using more accurate laboratory tests. This strategy would reduce further STI transmission by promptly treating high risk women and would reduce overtreatment among lower risk women in clinics where the syndromic approach is used.

Our results are also supportive of the syndromic approach used in developing countries, highlighting that it may be as effective as gold standard STI tests especially in highly sexually active populations. This is contrary to previous studies that have emphasised the low sensitivity of the syndromic approach compared to the gold standard test in women, without considering loss to follow up.8,9

Owing to their increased specificity,3,13 current POC tests could be extremely useful in reducing the overtreatment resulting from using the syndromic approach to manage vaginal discharge in women. Indeed, especially in low to medium risk populations, the low specificity of the syndromic approach in women leads to a poor positive predictive value, a negative characteristic that has led to criticism of the use of syndromic management for gonococcal and chlamydial infection in women.35 Indeed, given that an estimated 20–40% of ever partnered women will experience physical violence from a sexual partner,36 and since cases of violence have been documented following STI disclosure,37 the risks to women associated with these false diagnoses merits serious consideration.

For example, if we consider the syndromic approach used in the sex worker population in Cotonou, Benin (24.7% prevalence of cervical infection),4 where the sensitivity and specificity of the syndromic approach for cervical infection is 48.3% and 74.7%, respectively, then twice as many uninfected women were wrongly diagnosed as having cervical infection (92 women) than those correctly diagnosed (57 women). If the prevalence of cervical infection had been lower, say 5%, then the situation worsens with 10 times more uninfected women being wrongly diagnosed (116 women) than those correctly diagnosed (12 women). In contrast, if a POC test for cervical infection, with sensitivity 63% and specificity 98% (based on the BioStar POC test for C trachomatis3), had been used then only nine uninfected women would have been wrongly diagnosed. This highlights the importance of considering current POC tests for screening programmes in different C trachomatis prevalence settings to avoid massive overtreatment and the social complications of misdiagnosis. This is not a concern when comparing current POC tests for C trachomatis with gold standard tests because they have comparable specificity.3,13,5

Another possible strategy to avoid the social complications associated with misdiagnosis when using the syndromic approach in high prevalence STI settings is mass presumptive treatment.17 This will still result in massive overtreatment but will avoid the social consequences of misdiagnosis and will treat more STIs than using POC tests or the gold standard test. However, recently there has been growing concern that mass presumptive treatment may increase drug resistance38 and the negative effect the strategy may have on an individual’s risk perception.

Key messages

  • Rapid, point of care (POC) diagnostic tests can be important tools for STI control in women

  • In settings where many patients do not return for the results of laboratory tests, a POC test or the syndromic approach can be as effective as a more sensitive laboratory test

  • Even patients who do return for laboratory results may transmit STIs to others in the interval between being tested and returning for treatment. Where this is likely a POC test or the syndromic approach can be as effective as a more sensitive laboratory test

  • Our mathematical model can be used to estimate the sensitivity required of POC tests for N gonorrhoeae and C trachomatis (including the syndromic approach for cervical infections) that will enable them to perform as well as gold standard tests in different populations

The analysis is limited because it primarily considered specific scenarios and has used a static model to estimate the STI transmission during the delay in treatment. Further analysis using dynamic models of STI and HIV transmission are being done to assess the robustness of the findings in high transmission settings; the potential impact of POC STI testing on STI and HIV transmission; the potential cost effectiveness of reducing STI transmission and overtreatment when compared to syndromic management.

Acknowledgments

This investigation received financial support from the UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases (TDR), as part of a large piece of work that was partly funded by the Wellcome Trust. Rosanna W Peeling is employed by Sexually Transmitted Diseases Diagnostics Initiative, TDR/WHO. Peter Vickerman and Charlotte Watts are members of the Health Economics and Financing Programme and the AIDS Knowledge Programme, which are funded by the UK Department for International Development (DFID). David Mabey heads the DFID AIDS Knowledge Programme. Michel Alary was partially funded for this work by the Wellcome Trust and is a senior research scholar of the Fonds de la Recherche en Santé du Québec. The valuable comments contributed by the reviewers were appreciated. The views and opinions expressed are those of the authors alone.

CONTRIBUTORS
 PV and CW undertook the mathematical analysis; PV collated the data for the behavioural scenarios; RP commissioned the analysis and initiated discussions with MA, DM, PV, and CW; PV and CW wrote the paper; MA, DM, and RP were involved in interpretation of the results and drafting of the paper. All authors have reviewed and approved the final version of the manuscript.

REFERENCES

View Abstract

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles

  • Brief encounters
    Helen Ward Rob Miller
  • Editorial
    H Ward J Weber