Original ArticleCross-cohort heterogeneity encountered while validating a model for HIV disease progression among antiretroviral initiators
Introduction
Observational cohort studies have been vital to our understanding of the natural history of human immunodeficiency virus (HIV) disease progression and in the identification of factors associated with prognosis after antiretroviral therapy (ART) initiation [1], [2]. Recognizing the importance of such studies, the National Institute of Allergy and Infectious Diseases at the U.S. National Institutes of Health has sponsored the International Epidemiologic Databases to Evaluate AIDS (IeDEA), a global consortium to combine data and answer questions about the HIV epidemic (www.iedea-hiv.org) [3], [4].
Combining data from heterogeneous sites or applying knowledge learned from one site to another can be challenging, because patient characteristics, clinical practice patterns, and even human immunodeficiency virus type 1 (HIV-1) strains can differ between cohorts. Nevertheless, patient characteristics, especially immunologic status, at the initiation of highly active antiretroviral therapy (HAART) have been shown to be associated with subsequent AIDS-defining events (ADE) and death [2], [5], [6], [7], [8], [9], [10], [11].
Using data from the Collaborations in HIV Outcomes Research/US (CHORUS) cohort, we observed that percent (%CD4) and absolute (aCD4) CD4+ lymphocytes at HAART initiation, prior exposure to ART, and probable route of HIV acquisition (injection drug use [IDU] or non-IDU) were associated with subsequent progression to ADE/death in a multivariable model [12]. Using these patient characteristics, as well as HIV-1 RNA at HAART initiation and demographic features, we created a prognostic model for ADE/death in HIV-infected persons initiating HAART. A calculator that uses this model to predict ADE-free survival probabilities up to 7 years after HAART initiation is available at http://biostat.mc.vanderbilt.edu/HIVSurvivalPrediction.
The goal of this study was to assess how well our model predicted ADE/death in an independent cohort. First, we used resampling techniques to internally estimate our model's predictive ability when applied to independent data. Second, we applied our model to a cohort of HIV-infected persons initiating HAART in Baltimore, MD, the Johns Hopkins HIV Clinical Cohort (JHHCC) Adult HIV Clinic. The JHHCC was specifically chosen to evaluate the predictive model on a cohort different from CHORUS in terms of demographics, common routes of infection, and level of immunosuppression at HAART initiation. Finally, we explored possible reasons for the discrepancies between predicted and observed ADE/deaths in the JHHCC.
Section snippets
Study cohorts
Both the CHORUS cohort and the JHHCC have been described in detail previously [13], [14]. Briefly, our study included persons from both cohorts who were ≥ 18 years of age and initiated HAART between August 1, 1997 and January 1, 2005, and followed them through August 1, 2005. HAART was defined as at least 2 nucleoside reverse transcriptase inhibitors (NRTIs) in combination with at least 1 protease inhibitor (PI) and/or non-NRTI (NNRTI) or at least 3 NRTIs. To be included in this study, persons
Internal model validation
In the CHORUS data, the proportion of all patient pairs in which the ordering of predictions and outcomes was concordant was c = 0.678. Based on the bootstrap, the expected concordance when applying the model to independent data was c = 0.625. This suggests that the model is substantially better than chance at using patient characteristics to order their clinical prognosis (P < 0.001).
Figure 1a demonstrates the predictive accuracy of the model. CHORUS participants were divided into 5 categories based
Discussion
Using re-sampling techniques and an independent data set from JHHCC, we have described the prognostic ability of a model to predict ADE-free survival based on patient characteristics at initiation of HAART. In short, our model was fair at ordering patients' risk of disease progression based on characteristics at HAART initiation, but it was not good at predicting ADE-free survival more than 3 years after HAART initiation in an independent cohort with different characteristics.
Several measures
Acknowledgments
Financial support: Vanderbilt-Meharry Center for AIDS Research (NIH program 930 AI54999) and National Institutes of Health (grant K23 AT002508-01 to T.H., grant K24 A1065298 to T.R.S., and grants K24 DA00432 and R01 DA11602 to R.D.M.)
References (28)
- et al.
Prognosis of HIV-1-infected patients starting highly active antiretroviral therapy: a collaborative analysis of prospective studies
Lancet
(2002) - et al.
Prognostic model for HIV-1 disease progression in patients starting antiretroviral therapy was validated using independent data
J Clin Epidemiol
(2005) - et al.
Plasma viral load and CD4+ lymphocytes as prognostic markers of HIV-1 infection
Ann Intern Med
(1997) - et al.
Cohort profile: the North American AIDS Cohort Collaboration on Research and Design (NA-ACCORD)
Int J Epidemiol
(2007) - et al.
Cohort profile: Caribbean, Central and South America Network for HIV research (CCASAnet) collaboration within the International Epidemiologic Databases to Evaluate AIDS (IeDEA) programme
Int J Epidemiol
(2007) - et al.
Initiation of highly active antiretroviral therapy at CD4+ T lymphocyte counts of >350 cells/mm3: disease progression, treatment durability, and drug toxicity
Clin Infect Dis
(2003) - et al.
Rates of disease progression by baseline CD4 cell count and viral load after initiating triple-drug therapy
JAMA
(2001) - et al.
HIV viral load response to antiretroviral therapy according to the baseline CD4 cell count and viral load
JAMA
(2001) - et al.
Improved outcomes with earlier initiation of highly active antiretroviral therapy among human immunodeficiency virus-infected patients who achieve durable virologic suppression: longer follow-up of an observational cohort study
J Infect Dis
(2003) - et al.
Survival benefit of initiating antiretroviral therapy in HIV-infected persons in different CD4+ cell strata
Ann Intern Med
(2003)
Clinical efficacy of early initiation of HAART in patients with asymptomatic HIV infection and CD4 cell count > 350 × 10(6)/l
AIDS
Rates of disease progression according to initial highly active antiretroviral therapy regimen: a collaborative analysis of 12 prospective cohort studies
J Infect Dis
Absolute count and percentage of CD4+ lymphocytes are independent predictors of disease progression in HIV-infected person initiating highly active antiretroviral therapy
J Infect Dis
Zidovudine and stavudine sequencing in HIV treatment planning: findings from the CHORUS HIV cohort
J Acquir Immune Defic Syndr
Cited by (8)
Survival in HIV-1 patients receiving antiretroviral therapy in Morocco
2018, Revue d'Epidemiologie et de Sante PubliqueResearch progress on the association between HIV antiretroviral therapy and the ontbreak
2020, Chinese Journal of EndemiologyMortality and loss to follow-up among HIV-infected persons on long-term antiretroviral therapy in Latin America and the Caribbean
2015, Journal of the International AIDS SocietyA comparison of seven cox regression-based models to account for heterogeneity across multiple HIV treatment cohorts in latin america and the caribbean
2015, AIDS Research and Human RetrovirusesImpact of definitions of loss to follow-up on estimates of retention, disease progression, and mortality: Application to an HIV program in Mozambique
2013, American Journal of Epidemiology