Cross-cohort heterogeneity encountered while validating a model for HIV disease progression among antiretroviral initiators

Abstract

Objective

To evaluate a model for predicting time to AIDS or death among HIV-infected persons initiating highly active antiretroviral therapy (HAART).

Study Design and Setting

The model was constructed from 1,891 HAART initiators in the Collaborations in HIV Outcomes Research/US (CHORUS) cohort. The model's predictive ability was assessed using internal bootstrap validation techniques and data from 716 HAART initiators at Johns Hopkins HIV Clinical Cohort (JHHCC) in whom HIV disease was, in general, more advanced.

Results

The estimated concordance statistic was 0.632 with the bootstrap method and 0.625 in JHHCC. Mean predicted and observed 3-year AIDS-free survival for JHHCC was 0.76 and 0.73 (95% confidence interval [CI], 0.69–0.77), respectively; mean predicted and observed 5-year AIDS-free survival was 0.69 and 0.57 (95% CI, 0.52–0.62), respectively. Sensitivity analyses showed that the discrepancy between predicted and observed AIDS-free survival after 3 years could be due to differences in lost-to-follow-up rates between cohorts.

Conclusion

The model was fair at using baseline characteristics to order patients' risk of disease progression, but did not accurately predict AIDS-free survival >3 years after HAART initiation. Different variable definitions, patient characteristics, and loss to follow-up highlight the challenges of using data from one cohort to predict AIDS-free survival in an independent cohort.

Introduction

Observational cohort studies have been vital to our understanding of the natural history of human immunodeficiency virus (HIV) disease progression and in the identification of factors associated with prognosis after antiretroviral therapy (ART) initiation [1], [2]. Recognizing the importance of such studies, the National Institute of Allergy and Infectious Diseases at the U.S. National Institutes of Health has sponsored the International Epidemiologic Databases to Evaluate AIDS (IeDEA), a global consortium to combine data and answer questions about the HIV epidemic (www.iedea-hiv.org) [3], [4].

Combining data from heterogeneous sites or applying knowledge learned from one site to another can be challenging, because patient characteristics, clinical practice patterns, and even human immunodeficiency virus type 1 (HIV-1) strains can differ between cohorts. Nevertheless, patient characteristics, especially immunologic status, at the initiation of highly active antiretroviral therapy (HAART) have been shown to be associated with subsequent AIDS-defining events (ADE) and death [2], [5], [6], [7], [8], [9], [10], [11].

Using data from the Collaborations in HIV Outcomes Research/US (CHORUS) cohort, we observed that percent (%CD4) and absolute (aCD4) CD4⁺ lymphocytes at HAART initiation, prior exposure to ART, and probable route of HIV acquisition (injection drug use [IDU] or non-IDU) were associated with subsequent progression to ADE/death in a multivariable model [12]. Using these patient characteristics, as well as HIV-1 RNA at HAART initiation and demographic features, we created a prognostic model for ADE/death in HIV-infected persons initiating HAART. A calculator that uses this model to predict ADE-free survival probabilities up to 7 years after HAART initiation is available at http://biostat.mc.vanderbilt.edu/HIVSurvivalPrediction.

The goal of this study was to assess how well our model predicted ADE/death in an independent cohort. First, we used resampling techniques to internally estimate our model's predictive ability when applied to independent data. Second, we applied our model to a cohort of HIV-infected persons initiating HAART in Baltimore, MD, the Johns Hopkins HIV Clinical Cohort (JHHCC) Adult HIV Clinic. The JHHCC was specifically chosen to evaluate the predictive model on a cohort different from CHORUS in terms of demographics, common routes of infection, and level of immunosuppression at HAART initiation. Finally, we explored possible reasons for the discrepancies between predicted and observed ADE/deaths in the JHHCC.