You are here

  • Home
  • Archive
  • Volume 87, Issue 3
  • Performance of the Focus HerpeSelect-2 enzyme immunoassay for the detection of herpes simplex virus type 2 antibodies in seven African countries

Article Text

Article menu

Download PDFPDF

Basic science
Performance of the Focus HerpeSelect-2 enzyme immunoassay for the detection of herpes simplex virus type 2 antibodies in seven African countries
  1. Andrew Mujugira1,
  2. Rhoda Ashley Morrow2,
  3. Connie Celum1,3,4,
  4. Jairam Lingappa1,3,5,
  5. Sinead Delany-Moretlwe6,
  6. Kenneth H Fife7,
  7. Renee Heffron4,
  8. Guy De Bruyn8,
  9. Brigitte Homawoo9,10,
  10. Etienne Karita10,
  11. Nelly Mugo11,
  12. Bellington Vwalika12,
  13. Jared M Baeten1,3,4,
  14. for the Partners in Prevention HSV/HIV Transmission Study Team*
  1. 1Department of Global Health, University of Washington, Seattle, Washington, USA
  2. 2Department of Laboratory Medicine, University of Washington, Seattle, Washington, USA
  3. 3Department of Medicine, University of Washington, Seattle, Washington, USA
  4. 4Department of Epidemiology, University of Washington, Seattle, Washington, USA
  5. 5Department of Pediatrics, University of Washington, Seattle, Washington, USA
  6. 6Reproductive Health and HIV Research Unit, University of the Witwatersrand, Johannesburg, South Africa
  7. 7Department of Medicine, Indiana University, Indianapolis, Indiana, USA
  8. 8Perinatal HIV Research Unit, University of Witwatersrand, Johannesburg, South Africa
  9. 9AIDS Institute, New York State Department of Health, Albany, New York, USA
  10. 10Rwanda-Zambia HIV Research Group (RZHRG), Kigali, Rwanda
  11. 11Department of Obstetrics and Gynecology, University of Nairobi and Kenyatta National Hospital, Nairobi, Kenya
  12. 12Rwanda-Zambia HIV Research Group (RZHRG), Lusaka, Zambia
  1. Correspondence to Dr Andrew Mujugira, International Clinical Research Center, University of Washington, Box 359927, 325 Ninth Avenue, Seattle, WA 98104, USA; mujugira{at}uw.edu

Abstract

Objective To compare the performance of the Focus HerpeSelect-2 enzyme immunoassay (EIA) with the gold standard herpes simplex virus (HSV) type 2 western blot, among HIV-1-uninfected men and women in east and southern Africa.

Methods 3399 HIV-1-uninfected women and men from seven countries in east and southern Africa were tested for HSV-2 antibody using the Focus HerpeSelect-2 EIA. The performance of the HerpesSelect-2 EIA was compared with the gold standard HSV-2-specific western blot.

Results Two-thirds (2294/3399) of participants were male and two-thirds (2242/3399) were from east Africa. By western blot testing, HSV-2 prevalence was 68%; 59% in men and 85% in women. At the manufacturer's recommended cut-off value of greater than 1.1, the HerpeSelect-2 EIA had a sensitivity of 98.3% and specificity of 80.3%. Receiver operating characteristic plot analysis indicated that the optimum cut-off was 2.1 or greater, with sensitivity 93.9% and specificity 90.5%. Diagnostic accuracy was modestly higher for southern Africa (area under the curve (AUC) 0.979, 95% CI 0.970 to 0.988) compared with east Africa (AUC 0.954, 95% CI 0.942 to 0.965; p<0.001 for southern vs east Africa).

Conclusions The Focus HerpeSelect-2 EIA has acceptable diagnostic accuracy for the determination of HSV-2 serostatus in African HIV-1-uninfected adults. An assay cut-off value of 2.1 or greater results in approximately 90% sensitivity and specificity, against a gold standard HSV-2 western blot. Diagnostic accuracy differed slightly by geographical region.

  • Africa
  • HIV-1
  • HSV-2
  • Focus HerpeSelect-2 EIA
  • western blot

https://doi.org/10.1136/sti.2010.047415

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Herpes simplex virus (HSV) type 2 is the most frequent cause of genital ulcer disease worldwide and is an important risk factor for HIV-1 acquisition.1 The HerpeSelect-2 enzyme immunoassay (EIA) (Focus Technologies, Cypress, California, USA) is a commercially available, type-specific serological test for the detection of antibodies to HSV-2 glycoprotein G (gG) that is frequently used in epidemiological research studies of HSV-2. However, the HerpeSelect-2 EIA has been reported to have poor specificity for serological diagnosis of HSV-2 among some African populations, particularly for samples with index values (ie, the ratio of the optical density of the sample to the optical density of a standard calibrator) in the low positive range (values between 1.1 and 3.4).2–4 Studies comparing the HerpeSelect-2 EIA with gold standard assays, such as HSV-2-specific western blot, have proposed various cut-offs (index values 3.1–3.5) to improve specificity,2 3 5 6 but these were generally done with relatively small populations, only among men or women, or in single geographical areas. We compared the performance of the Focus HerpeSelect-2 EIA with the gold standard for HSV-2 serological diagnosis, HSV-2 western blot, among nearly 3400 HIV-1-uninfected men and women from seven countries in east and southern Africa.

    Methods

    Population and procedures

    Between November 2004 and April 2007, 3408 HSV-2/HIV-1-co-infected women and men and their HIV-1-uninfected heterosexual partners were enrolled in the Partners in Prevention HSV/HIV Transmission Study, a randomised clinical trial of acyclovir HSV-2 suppressive therapy to reduce HIV-1 transmission (http://ClinicalTrials.gov/ number NCT00194519). All participants were 18 years of age or older, and HIV-1-uninfected partners could be either HSV-2 seropositive or seronegative. Couples were from 14 sites in seven African countries in east (Kenya, Rwanda, Tanzania and Uganda) and southern Africa (Botswana, South Africa and Zambia). As previously reported, HSV-2 suppression provided to the HIV-1-infected partners did not reduce HIV-1 transmission risk to their initially HIV-1-uninfected partners.7 For the present study, we assessed the HSV-2 serological status of the HIV-1-uninfected partners from a blood sample collected at the enrolment visit.

    Institutional review boards at the University of Washington and at all collaborating site organisations approved study procedures. All participants provided written informed consent.

    Laboratory methods

    At study enrolment, HIV-1-uninfected partners provided a serum sample for HSV-2 serological testing using the HerpeSelect-2 EIA; 12 laboratories performed the testing for the 14 study sites. The manufacturer's instructions for this assay define a negative result as an index value less than 0.9, an indeterminate result as an index value between 0.9 and 1.1, and a positive result as an index value greater than 1.1. Archived serum aliquots from the enrolment visit were also batch tested at the end of the study at the University of Washington using a HSV type-specific western blot.8 Western blot readers were blinded to the results of the HerpeSelect-2 EIA. All sites participated in an external quality assurance programme using a HSV-2 proficiency panel developed at the University of Washington.9

    Statistical analysis

    The sensitivity and specificity of Focus HerpeSelect-2 EIA results, compared with western blot, were calculated, and receiver operating characteristic (ROC) curves were constructed to describe test performance. Optimal EIA index result cut-offs were identified, aiming to achieve both test specificity and sensitivity of approximately 90%. Individuals with indeterminate western blot results (ie, HSV-2 gG band indistinct or not apparent after pre-absorption against HSV-1 antigens)10 were excluded from sensitivity/specificity and ROC analyses. To assess the effect of between-laboratory variation on our estimates of test performance, we calculated adjusted ROC analyses that controlled for clustering by laboratory.11 Statistical calculations were performed using PASW Statistics 17.0 and Stata 10.1.

    Results

    Of 3408 HIV-1-uninfected participants enrolled in the study, HerpeSelect-2 EIA and western blot results were available for 3399 (99.7%). Of these, 2294 (67%) were male and 2242 (66%) were from east Africa. The median age was 34 years (IQR 28–41). By western blot testing, HSV-2 seroprevalence was 68% (2304/3399), including 85% (944/1105) in women, 59% in men (1360/2294), 68% (1532/2242) in east Africa and 67% (772/1157) in southern Africa. One hundred and nine participants (3%) had indeterminate (n=109) western blot results.

    There was generally strong agreement between HerpeSelect-2 EIA and western blot results (table 1). Using the manufacturer's recommended cut-off (index value >1.1) and compared with western blot, the HerpeSelect-2 EIA had a sensitivity of 98.3% (2264/2304) and specificity of 80.3% (792/986) compared with western blot for determination of HSV-2 infection (table 2). At an index value cut-off of 3.5 or greater, sensitivity was 82.9% (1888/2304) and specificity improved to 95.1% (938/986). ROC plot analysis comparing a range of HerpeSelect-2 cut-offs versus gold standard western blot indicated strong agreement between the two tests: (area under the curve (AUC) 0.962, 95% CI 0.954 to 0.971). An optimum cut-off of 2.1 or greater was identified, giving a sensitivity of 93.9% and a specificity of 90.5%.

    View this table:
    Table 1

    Comparison of Focus HerpeSelect-2 EIA values and HSV-2 western blot results

    View this table:
    Table 2

    Focus HerpeSelect-2 EIA performance compared to HSV-2 western blot among HIV-1-uninfected participants from seven countries in sub-Saharan Africa

    We assessed whether demographic factors resulted in differences in the diagnostic accuracy of the HerpeSelect-2 EIA. By ROC testing, overall test performance was modestly better in men (AUC 0.960, 95% CI 0.951 to 0.969) than in women (AUC 0.944, 95% CI 0.914 to 0.973; figure 1A), although this was not statistically significant (p=0.3). Diagnostic accuracy was modestly greater for samples from southern Africa (AUC 0.979, 95% CI 0.970 to 0.988) compared with east Africa (AUC 0.954, 95% CI 0.942 to 0.965; figure 1B) and this difference was statistically significant (p<0.001). The optimal index value cut-off was 2.7 in east Africa (sensitivity 89.8%, specificity 90.3%) and 1.3 in southern Africa (sensitivity 96.8%, specificity 90.4%). There were small differences in test performance between countries in the same geographical region (table 3). Among samples from Uganda, specificity was 81% at an index value of 2.1 or greater, and the optimal index value was 3.4 (sensitivity 90.4%, specificity 90.1%). Test performance did not differ for those with or without a history of genital ulcer disease in the 3 months before enrolment (AUC 0.903, 95% CI 0.732 to 1.000 vs AUC 0.954, 95% CI 0.945 to 0.962; p=0.50). In addition, age had no effect on test performance: age 25 years or less (AUC 0.968, 95% CI 0.951 to 0.985) vs age greater than 25 years (AUC 0.961, 95% CI 0.951 to 0.970; p=0.38).

    Figure 1
    Figure 1

    Diagnostic accuracy of the Focus HerpeSelect-2 EIA, by gender and geographical region.

    View this table:
    Table 3

    Diagnostic accuracy of the Focus HerpeSelect-2 EIA, by country

    We investigated whether between-laboratory variation accounted for differences in diagnostic accuracy. We recalculated the AUC, 95% CI and p values for the above comparisons with adjustment for clustering by laboratory, and obtained results that were essentially identical to the unadjusted analyses. In the adjusted models, there were no statistically significant differences in adjusted AUC estimates for comparisons of gender (p=0.43), genital ulcer disease history (p=0.51), or age group (p=0.47). A statistically significant effect of region remained present in the analysis adjusting for site laboratory (adjusted AUC 0.978, 95% CI 0.961 to 0.995 for southern Africa vs 0.953, 95% CI 0.939 to 0.966 for east Africa, p=0.02).

    We considered the proportion of individuals with positive and indeterminate western blot results at various low positive HerpeSelect-2 EIA values. For those with EIA results in the manufacturer's indeterminate range (ie, 0.9–1.1, n=62), 29% were positive and 18% indeterminate by western blot. For those with EIA results between 1.2 and 2.4, 55% were positive and 13% indeterminate by western blot. Finally, for those with EIA results between 2.5 and 3.4, 80% were positive and 6% indeterminate by western blot. Overall, of the 109 persons with indeterminate western blots, 75 (69%) had HerpeSelect-2 EIA values greater than 1.1 and 37 (34%) greater than 2.4 (table 1). There were no significant differences in the frequency of indeterminate HSV western blots by gender (p=0.12) or region (p=0.10).

    Discussion

    This multinational study of nearly 3400 HIV-1-uninfected adults represents the largest investigation of the performance of the Focus HerpeSelect-2 EIA to date in sub-Saharan Africa. In this population with high HSV-2 seroprevalence, a Focus HerpeSelect-2 EIA index value cut-off of 2.1 or greater resulted in approximately 90% sensitivity and specificity. There was some variation in optimal cut-off values by subgroups defined by gender and region. Of the approximately 15% of sera that had index values 1.2–3.4, two-thirds were confirmed as HSV-2 seropositive by western blot.

    Previous studies have demonstrated reduced specificity of the HerpeSelect-2 EIA at an index value cut-off greater than 1.1 for samples from Africa4 6 potentially due to cross-reactivity with HSV-112 or HSV-2 sequence diversity.13 In our study population from east and southern Africa, an index cut-off value of 2.1 or greater improved test specificity to approximately 90%, similar to a meta-analysis that suggested index values between 2.2 and 3.5 increased specificity14; in our population, a cut-off of 2.1 or greater also provided a sensitivity of approximately 94%. Studies from Rakai, Uganda, have suggested higher index value cut-offs improve specificity, with an optimal trade-off between sensitivity and specificity identified at index values of 3.2–3.4.3 15 Our results for east Africa in general and Uganda in particular, are consistent with those findings. For samples from southern Africa, we found an index cut-off value of 1.3 provided an acceptable trade-off between sensitivity and specificity, lower than the optimal index value of 3.3 from one previous study from that region.5 Regional differences in test specificity could be a result of cross-reacting antibodies due to possible HSV-2 antigenic differences.3 Importantly, between-laboratory variations in test performance did not appear to account for differences in diagnostic accuracy. We also found gender differences in test performance, although this was not statistically significant; at each cut-off value, test sensitivity was better in women while specificity was higher in men. The potential for differential test performance by gender may merit further investigation.

    Although improving test specificity is important in epidemiological studies and for clinical diagnosis, shipping sera to a reference laboratory for western blot confirmatory testing is neither feasible nor cost-effective. Studies have suggested that sequential serological testing with the Focus HerpeSelect-2 assay plus other commercially available type-specific assays, such as Kalon HSV-2 gG2 ELISA (Kalon Biological Ltd, Surrey, UK) or Biokit HSV-2 rapid test (Biokit USA Inc, Lexington, Massachusetts, USA) could be feasible alternatives to western blot for confirmatory testing of sera that are initially positive on the HerpeSelect-2 EIA,16 although other studies have not demonstrated that serial algorithms improve diagnostic accuracy.6 17 The approximately 90% sensitivity and specificity values we calculated with modified cut-offs may not be sufficiently precise for the use of this assay for serological diagnosis of HSV-2 in all clinical care settings.

    In summary, the Focus HerpeSelect-2 EIA had an acceptable diagnostic accuracy in HIV-1-uninfected women and men from east and southern Africa, compared with gold standard HSV-2 western blot, particularly when using an index cut-off of 2.1 or greater.

    Key messages

    • The Focus HerpeSelect-2 EIA has good diagnostic accuracy in east and southern Africa.

    • Using an index value of 2.1 or greater to define a positive result, compared with gold standard HSV-2 western blot, resulted in approximately 90% sensitivity and specificity.

    • Diagnostic accuracy was modestly better for samples from southern Africa compared with those from east Africa.

    Acknowledgments

    The authors are especially thankful to Anne Cent from the University of Washington Clinical Virology Laboratory for performing HSV western blot testing and to Katherine Thomas for assistance with statistical analyses. The authors are grateful to the study participants for their participation and dedication. They wish to thank the study team members at the research sites and at the University of Washington for their contributions to study implementation and data collection.

    Appendix

    Partners in prevention HSV/HIV transmission study team

    University of Washington coordinating centre and central laboratories, Seattle, USA

    Connie Celum (principal investigator), Anna Wald (protocol co-chair), Jairam Lingappa (medical director), Jared M Baeten, Mary Campbell, Lawrence Corey, Robert W Coombs, James P Hughes, Amalia Magaret, M Juliana McElrath, Rhoda Morrow, James I Mullins

    Study sites and site principal investigators:

    Cape Town, South Africa (University of Cape Town): David Coetzee; Eldoret, Kenya (Moi University, Indiana University): Kenneth Fife, Edwin Were; Gaborone, Botswana (Botswana Harvard Partnership): Max Essex, Joseph Makhema; Kampala, Uganda (Infectious Disease Institute, Makerere University): Elly Katabira, Allan Ronald; Kigali, Rwanda (Rwanda Zambia HIV Research Group and Emory University): Susan Allen, Kayitesi Kayitenkore, Etienne Karita; Kisumu, Kenya (Kenya Medical Research Institute, University of California San Francisco): Elizabeth Bukusi, Craig Cohen; Kitwe, Zambia (Rwanda Zambia HIV Research Group and Emory University): Susan Allen, William Kanweka; Lusaka, Zambia (Rwanda Zambia HIV Research Group and Emory University): Susan Allen, Bellington Vwalika; Moshi, Tanzania (Kilimanjaro Christian Medical College, Harvard University): Saidi Kapiga, Rachel Manongi; Nairobi, Kenya (University of Nairobi, University of Washington): Carey Farquhar, Grace John-Stewart, James Kiarie; Ndola, Zambia (Rwanda Zambia HIV Research Group and Emory University): Susan Allen, Mubiana Inambao; Orange Farm, South Africa (Reproductive Health Research Unit, University of the Witwatersrand): Sinead Delany-Moretlwe, Helen Rees; Soweto, South Africa (Perinatal HIV Research Unit, University of the Witwatersrand): Guy de Bruyn, Glenda Gray, James McIntyre; Thika, Kenya (University of Nairobi, University of Washington): Nelly Rwamba Mugo.

    Data management was provided by DF/Net Research, Inc (Seattle, USA) and site laboratory oversight was provided by Contract Lab Services (University of the Witwatersrand, Johannesburg, South Africa).

    References

    1. ↵
      1. Corey L,
      2. Wald A,
      3. Celum CL,
      4. et al
      . The effects of herpes simplex virus-2 on HIV-1 acquisition and transmission: a review of two overlapping epidemics. J Acquir Immune Defic Syndr 2004;35:435–45.
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Hogrefe W,
      2. Su X,
      3. Song J,
      4. et al
      . Detection of herpes simplex virus type 2-specific immunoglobulin G antibodies in African sera by using recombinant gG2, Western blotting, and gG2 inhibition. J Clin Microbiol 2002;40:3635–40.
      OpenUrlAbstract/FREE Full Text
    3. ↵
      1. Laeyendecker O,
      2. Henson C,
      3. Gray RH,
      4. et al
      . Performance of a commercial, type-specific enzyme-linked immunosorbent assay for detection of herpes simplex virus type 2-specific antibodies in Ugandans. J Clin Microbiol 2004;42:1794–6.
      OpenUrlAbstract/FREE Full Text
    4. ↵
      1. van Dyck E,
      2. Buve A,
      3. Weiss HA,
      4. et al
      . Performance of commercially available enzyme immunoassays for detection of antibodies against herpes simplex virus type 2 in African populations. J Clin Microbiol 2004;42:2961–5.
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Delany-Moretlwe S,
      2. Jentsch U,
      3. Weiss H,
      4. et al
      . Comparison of focus HerpesSelect and Kalon HSV-2 gG2 ELISA serological assays to detect herpes simplex virus type 2 antibodies in a South African population. Sex Transm Infect 2010;86:46–50.
      OpenUrlAbstract/FREE Full Text
    6. ↵
      1. Smith JS,
      2. Bailey RC,
      3. Westreich DJ,
      4. et al
      . Herpes simplex virus type 2 antibody detection performance in Kisumu, Kenya, using the Herpeselect ELISA, Kalon ELISA, Western blot and inhibition testing. Sex Transm Infect 2009;85:92–6.
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Celum C,
      2. Wald A,
      3. Lingappa JR,
      4. et al
      . Acyclovir and transmission of HIV-1 from persons infected with HIV-1 and HSV-2. N Engl J Med 2010;362:427–39.
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Ashley RL,
      2. Militoni J,
      3. Lee F,
      4. et al
      . Comparison of Western blot (immunoblot) and glycoprotein G-specific immunodot enzyme assay for detecting antibodies to herpes simplex virus types 1 and 2 in human sera. J Clin Microbiol 1988;26:662–7.
      OpenUrlAbstract/FREE Full Text
    9. ↵
      1. Lingappa JR,
      2. Kahle E,
      3. Mugo N,
      4. et al
      . Characteristics of HIV-1 discordant couples enrolled in a trial of HSV-2 suppression to reduce HIV-1 transmission: the Partners Study. PLoS One 2009;4:e5272.
      OpenUrlCrossRefPubMed
    10. ↵
      1. Sacks SL,
      2. Straus SE,
      3. Whitley RJ,
      4. et al.
      1. Ashley RL
      . Current Concepts of Laboratory Diagnosis of Herpes Simplex Infection. Sacks SL, Straus SE, Whitley RJ, et al., eds. Washington, DC: IOS Press, 1995.
    11. ↵
      1. Janes H,
      2. Longton G,
      3. Pepe M
      . Accommodating covariates in ROC analysis. Stata J 2009;9:17–39.
      OpenUrlPubMedWeb of Science
    12. ↵
      1. Golden MR,
      2. Ashley-Morrow R,
      3. Swenson P,
      4. et al
      . Herpes simplex virus type 2 (HSV-2) Western blot confirmatory testing among men testing positive for HSV-2 using the focus enzyme-linked immunosorbent assay in a sexually transmitted disease clinic. Sex Transm Dis 2005;32:771–7.
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      1. Ashley-Morrow R,
      2. Nollkamper J,
      3. Robinson NJ,
      4. et al
      . Performance of focus ELISA tests for herpes simplex virus type 1 (HSV-1) and HSV-2 antibodies among women in ten diverse geographical locations. Clin Microbiol Infect 2004;10:530–6.
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. Biraro S,
      2. Mayaud P,
      3. Morrow RA,
      4. et al
      . Performance of commercial herpes simplex virus type-2 antibody tests using serum samples from sub-Saharan Africa: a systematic review and meta-analysis. Sex Transm Dis 2011;38:140–7.
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. Gamiel JL,
      2. Tobian AA,
      3. Laeyendecker OB,
      4. et al
      . Improved performance of enzyme-linked immunosorbent assays and the effect of human immunodeficiency virus coinfection on the serologic detection of herpes simplex virus type 2 in Rakai, Uganda. Clin Vaccine Immunol 2008;15:888–90.
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Morrow RA,
      2. Friedrich D,
      3. Meier A,
      4. et al
      . Use of “biokit HSV-2 Rapid Assay” to improve the positive predictive value of focus HerpeSelect HSV-2 ELISA. BMC Infect Dis 2005;5:84.
      OpenUrlCrossRefPubMed
    17. ↵
      1. Ng'ayo MO,
      2. Friedrich D,
      3. Holmes KK,
      4. et al
      . Performance of HSV-2 type specific serological tests in men in Kenya. J Virol Methods 2010;163:276–81.
      OpenUrlCrossRefPubMedWeb of Science

    Footnotes

    • ↵* Partners in Prevention HSV/HIV Transmission Study Team are listed in the appendix.

    • Funding This study was supported through research grants from the Bill & Melinda Gates Foundation (grant ID 26469) and the US National Institutes of Health (National Institute of Allergy and Infectious Diseases grant P01 AI030731). The funders played no role in design, data collection, analysis, interpretation, or writing of the report.

    • Competing interests None.

    • Patient consent Obtained.

    • Ethics approval This study was conducted with the approval of the University of Washington, Human Subjects Division and all collaborating site institutions.

    • Provenance and peer review Not commissioned; externally peer reviewed.