Article Text
PDF
Abstract
Objective To develop and validate a risk scoring tool to identify those who are at increased risk of chlamydia infection.
Methods We used demographic data, sexual behaviour information and chlamydia positivity results from more than 45 000 individuals who attended Sydney Sexual Health Centre between 1998 and 2009. Participants were randomly allocated to either the development or internal validation data set. Using logistic regression, we created a prediction model and weighted scoring system using the development data set and calculated the odds ratio of chlamydia positivity for participants in successively higher quintiles of score. The internal validation data set was used to evaluate the performance characteristics of the model for five quintiles of risk scores including population attributable risk, sensitivity and specificity.
Results In the prediction model, inconsistent condom use, increased number of sexual partners in last 3 months, genital or anal symptoms and presenting to the clinic for sexually transmitted infections screening or being a contact of a sexually transmitted infection case were consistently associated with increased risk of chlamydia positivity in all groups. High scores (upper quintiles) were significantly associated with increased risk of chlamydia infection. A cut-point score of 20 or higher distinguished a increased risk group with a sensitivity of 95%, 67% and 79% among heterosexual men, women and men who have sex with men (MSM), respectively.
Conclusion The scoring tool may be included as part of a health promotion and/or clinic website to prompt those who are at increased risk of chlamydia infection, which may potentially lead to increased uptake and frequency of testing.
- Chlamydia infection
- risk prediction
- hepatitis C
- HIV
- HIV testing
- homosexuality
- infectious disease
- epidemiology
- sexual medicine
- health informatics
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.
Statistics from Altmetric.com
- Chlamydia infection
- risk prediction
- hepatitis C
- HIV
- HIV testing
- homosexuality
- infectious disease
- epidemiology
- sexual medicine
- health informatics
Article summary
Article focus
The authors created a risk assessment tool that allows people to estimate their own chlamydia risk score based on simple non-invasive variables.
Key messages
The tool described here will potentially provide a simple and cost-effective method of identifying and alerting individuals who would benefit from chlamydia screening.
This tool may be included as part of a health promotion and/or clinic website.
This tool may potentially lead to increased uptake and frequency of testing.
Strengths and Limitations
This is the first study to utilize statistical methods to derive a locally-specific assessment tool using 12 years of data from more than 45 000 men and women.
The Study population was sexual health clinic attendees who are likely to be at higher risk for Chlamydia infection compared to the general population.
Introduction
Chlamydia infection is highly prevalent in young heterosexuals and men who have sex with men (MSM) in Australia, with prevalence estimates of 3–5% in both populations.1 2 The majority of chlamydia infections are asymptomatic. Chlamydia is associated with sequelae such as pelvic inflammatory disease and infertility in women and proctitis in men.3–6 Also in MSM, chlamydia re-infection of the rectum has been associated with an increased risk of HIV seroconversion.7
The number of chlamydia notifications continues to increase steadily each year among MSM and young heterosexual men and young women in Australia,8 9 as in many other countries. A major public health challenge is therefore to identify individuals at risk of chlamydia and facilitate testing and treatment before the development of chlamydia sequelae and onward transmission to others. Clinical guidelines in Australia recommend annual chlamydia testing in <25 year olds, annual HIV and sexually transmissible infection (STI) testing for all MSM and 3–6 monthly testing for high-risk MSM reporting more than 10 sexual partners in the last 6 months, unprotected sex and other specific risk behaviours.8
Clinical risk prediction approaches that can capture a continuous risk spectrum have been used in public health and clinical care decision making and have been proposed as an alternative to diagnosis for some diseases in various contexts.11 12 Our study aimed to develop and validate a simple scoring tool to assess the risk of chlamydia infection using demographic and sexual risk behaviour information collected from over 45 000 individuals who attended Sydney Sexual Health Centre (SSHC) between 1998 and 2009.
Methods
Study population
The study population consists of 45 902 men and women who visited SSHC during the period 1998–2009. A standard medical record form was used to collect demographic and sexual behaviour information from all new attendees and a sexual health screen was undertaken. Since 1998 SSHC has actively triaged those at higher risk of STIs into the service. SSHC also targets sex workers from culturally and linguistically diverse (CALD) backgrounds through interpreter facilitated sex worker clinics.
For this analysis, the demographic and sexual behaviour information was extracted from the medical records system including the anonymous patient identifier, age, gender, postcode, country of birth, date of arrival in Australia (if born overseas), marital status, alcohol use, condom use, number of male/female sex partners in the last 3 and 12 months, sex overseas in the past 12 months, reason for attendance, self-reported past chlamydia diagnoses, perceived HIV status and the current HIV/STI test results.
Statistical analyses
A split-sample method was used to develop a risk equation and scoring system with internal validation for each study population. Participants were randomly allocated to either the development (∼67%) or internal validation (∼33%) sample data sets within each group.
Development data set
Logistic regression was used to create a predictive model based on the development data set which included 11 354, 6800 and 12 700 MSM, heterosexual men and women, respectively. We evaluated a range of socio-demographic and sexual behaviour variables as potential determinants of chlamydia infection including age, country of birth (Australia vs other countries), language spoken at home (English vs others), marital status (married/defacto vs others (divorced/widowed/unknown)), CALD (not born in Australia and not speaking English at home), travellers (not born in Australia and in Australia for less than 2 years or those who identify themselves as ‘travellers’), area of residence, alcohol use, number of sexual partners in the past 3 months, condom use in the past 3 months, sex overseas in the past year, current sex work, reason for presentation, anal/genital symptoms, past chlamydia diagnoses and perceived HIV status. All analyses were stratified by sexual identity (MSM, heterosexual man or woman).
We used descriptive statistics to characterise the groups according to chlamydia status: mean and SD for continuous variables and percentages for categorical variables. Logistic regression was used to create a predictive model based on the development data set. We used all non-missing observations available in the relevant analyses as only a small proportion of observations had any missing data. All analyses were conducted using SAS statistical software v 9.2 (SAS Institute) and STATA 10.0.
Derivation of a screening score
Using the development data sets for the three sub-groups, we investigated a comprehensive list of predictors known to be potentially associated with chlamydia infection in an initial model. Specifically, we included the main effects of all variables listed in table 1. We first analysed the univariate associations between each variable and being diagnosed with STIs in each sub-group separately. Backward elimination was used to reach the final multivariate model, in which factors with the largest p value were sequentially deleted until only significant predictors remained. We then created a weighted scoring system by rounding all regression coefficients up to the nearest integer (ie, the smallest integer greater than the estimate). This method was based on the β coefficients (or log of the ORs) rather than ORs, which can be excessively influenced by only a few factors.11 Once the final model was defined, we created integer weights for each variable. We calculated these weights by multiplying the model coefficients by 10. Using the rounded weights in the risk function, we estimated the participant-specific probabilities of chlamydia positivity and characterised the degrees of risk based on cut-off points of the probability distribution.
Characteristics of the population
Cross-sectional internal validation
The prediction model was evaluated in the three cross-sectional internal validation data sets of 3805 MSM, 5313 heterosexual men and 7084 women. We conducted various analyses to check the sensitivity and robustness of the new screening score. We computed standard validation measures for the proportion of those tested positive for chlamydia infection, sensitivity, specificity, positive likelihood and negative likelihood ratio and the area under the receiver-operating characteristic curve (AUC)13 as discrimination statistics. Akaike information criteria were evaluated as model fit statistics. The Hosmer–Lemeshow goodness-of-fit test was also performed. We also assessed the diagnostic characteristics of different cut-points based on the total score in the development as well as the validation data sets. The purpose of this analysis was to assess whether the combination of risk factors under consideration could predict those at increased risk with acceptable accuracy.
Population attributable risk
We then estimated the population attributable risk (PAR), which estimates the percentage of chlamydia infections that would not have occurred if all the participants had been in the lowest risk (first quintile) category of the risk score. We calculated PAR by using previously described methods14 that were elaborated for this study design and are appropriate for use with multivariate adjusted relative risks.
Ethics approval for the study was obtained from the South Eastern Sydney and Illawarra Area Health Service Human Research Ethics Committee.
Results
Table 1 summarises participant characteristics by group. The overall prevalence of chlamydia was 6%, 7% and 5% for MSM, heterosexual men and women, respectively. MSM were more likely to be Australian born and live in metropolitan Sydney. More than 30% of the females were from CALD backgrounds compared to 13% of heterosexual men and MSM. Approximately 50% of females were also classified as travellers compared to 38% and 27% of heterosexual men and MSM, respectively. Although excess alcohol intake and current smokers were more common among heterosexual men and women compared to MSM, more MSM reported ever injecting drug use. Approximately 50% of women reported being in full time employment and 20% of them identified as being a sex worker. More heterosexual men reported that they had had sex in Asia in the last 12 months. Inconsistent condom use in the last 3 months and presenting with genital or anal symptoms were more common among heterosexual men and women compared to MSM. The primary reason for making an appointment was testing for STI in all groups, however, presentation for HIV testing was more common among MSM compared to heterosexual men and women. Consistent with this, approximately 50% of heterosexual men and women also did not know their HIV status compared to 22% of MSM.
Prediction model
Table 2 presents the final multivariate logistic regression model derived from the development data set for each group. Independent predictors of chlamydia infection in MSM were younger age, inconsistent condom use, increased number of male sexual partners in the past 3 months, anal/genital symptoms and presenting for STI screening or being a contact of an STI case.
Multivariate logistic regression: ORs and 95% CIs for chlamydia infection
Independent predictors of chlamydia infection in heterosexual men were being single, CALD background, being unsure about HIV status, inconsistent condom use, increased number of female sexual partners in the past 3 months, anal/genital symptoms and presenting for STI screening or being a contact of an STI case. The Hosmer–Lemeshow goodness-of-fit test showed no lack of fit for the three fitted models (p>0.21 in all models).
Independent predictors of chlamydia infection in women were being single, CALD background, being unsure about HIV status, inconsistent condom use, anal/genital symptoms, presenting for STI screening or being a contact of an STI case.
Internal validation
The variables age and number of male/female sexual partners required multiple categories to capture the risk gradient, whereas other risk factors were binary. The risk factors collectively yielded an AUC of 0.71 (95% CI 0.69 to 0.73) for MSM, 0.74 (95% CI 0.72 to 0.75) for heterosexual men and 0.72 (95% CI 0.70 to 0.74) for women. No statistically significant interactions were detected between the sexual risk factors and the age groups.
Table 3 shows the odds ratios from the logistic regression models for the quintiles of the risk scores in the development and validation data sets. The ORs (95% CI) of chlamydia positivity for participants in successively higher quintiles of STI score were: 1.79 (1.23 to 2.60), 2.96 (2.10 to 4.15), 4.56 (3.30 to 6.30) and 8.80 (6.43 to 12.02) for MSM; 2.53 (1.76 to 3.63), 4.21 (2.97 to 5.98), 6.82 (4.84 to 9.60) and 14.17 (10.20 to 19.68) for heterosexual men; and 2.50 (1.67 to 3.76), 3.70 (2.51 to 5.43), 4.59 (3.11 to 6.78) and 12.33 (8.55 to 17.78) for heterosexual women. There was a linear trend towards increasing chlamydia positivity with increasing score regardless of group for the development and validation data sets (trend, p value<0.001, all).
ORs and 95% CIs for being diagnosed with chlamydia infection by quintiles of chlamydia risk scoring
We also estimated PARs (95% CI) for the upper four quintiles of the scores. Results showed that 73% (69% to 76%) of infections in MSM, 80% (77% to 82%) of infections in heterosexual men and 78% (74% to 81%) of infections in women would be avoided if the participants who were in the upper four quintiles of the STI scores were in the lowest quintile. Results from the validation data set were consistent with results from the development data set.
We performed additional analyses to assess the diagnostic characteristics of various cut-points of the total score in the overall study population (table 4). For example, among heterosexual men, the predictive value of the screening criteria for a cut-point score of 20 or higher was approximately 10%. Although it is crucial to determine the best cut-point to alert those at highest risk for infection, a cut-point of ≥20 or higher demonstrated excellent sensitivity among MSM and heterosexual males (80.0% and 96.8%, respectively) and acceptable sensitivity among heterosexual women (70.0%).
Selection criteria for screening for chlamydia infection
Discussion
In this study, we have developed a chlamydia risk scoring tool based on data from more than 45 000 men and women who attended SSHC during the period 1998–2009. The tool was validated to accurately identify those at increased risk of chlamydia infection. Our methodology made use of a range of coexisting risk factors that were identified by a rigorous statistical approach in order to accurately determine the most relevant risk factors for chlamydia infection.
Developing a risk assessment tool that identifies, quantifies and characterises risks may lead to improved knowledge about chlamydia and increased testing for STIs. This is particularly relevant because many infections are asymptomatic and individuals may be unaware that they are at risk and/or have the infection. For example, our current study found higher percentages of heterosexual males and females were unsure of their HIV status compared to MSM (47%, 48% and 22% for heterosexual men, women and MSM, respectively) and those who were not aware of their HIV status were determined to be at high risk for chlamydia infection (OR 1.38, p=0.001 and OR 1.54, p<0.001 for heterosexual men and women, respectively).
This study has several strengths. It is the first study to utilise statistical methods to derive a locally-specific risk assessment tool to identify, quantify and characterise the risks of various groups in Australia with acceptable sensitivity. Risk assessment methods or prediction models ideally should be derived from large representative samples. Our study used 12 years of data from more than 45 000 men and women to develop the suggested risk assessment tool. Our risk calculation was based on a statistical method that yielded a systematic scoring system for carefully selected predictors, guided by both scientific evidence and feasibility perspectives. However, our study is limited by its retrospective nature and the self-reported measures of the sexual risk factors and anal/genital symptoms which may be subject to measurement error/misclassification. The study population was based on clinic attendees who are triaged into the service based on risk assessment and/or the presence of symptoms as demonstrated by the positivity of 6%, 7% and 5% for MSM, heterosexual men and women, respectively. When we restricted the analyses to those younger than 20 years of age, chlamydia positivity rates were estimated to be 11%, 8% and 7% for MSM, heterosexual male and females, respectively, compared to 3%–5% among young MSM, heterosexual men and women in community-based studies.8 It is also possible that chlamydia infection might have been acquired prior to the sexual risk behaviour that preceded the clinic visit as chlamydia infection can persist for an average of 12 months if untreated.9 Finally, risk prediction models apply primarily to groups defined by a set of clinically relevant variables rather than directly to individuals. This is a limitation common to all risk prediction models.10 Indeed, prevention of chlamydia infection may require population-based interventions that are beyond the control of individual physicians and patients. Therefore, our risk prediction assessment serves only as a guideline and should not be taken as an absolute definition of high risk.
We envisage that the chlamydia risk scoring tool developed in this study will be adapted for interactive clinic websites and the interface and website will be designed and calibrated for use by relevant populations including people from CALD backgrounds who were at higher risk for chlamydia infection. The screening tool will also be piloted in primary care clinics targeting those at higher risk for infection(s).
In conclusion, we believe the screening tool described here will provide a simple and cost-effective method of identifying and alerting individuals who would benefit from chlamydia screening with notable predictive validity. Self-identification, if widely practiced, could be an effective method of case ascertainment and may encourage uptake of screening.
References
Supplementary materials
Supplementary Data bmjopen-2010-000005
Files in this Data Supplement:
Footnotes
To cite: Wand H, Guy R, Donovan B, et al. Developing and validating a risk scoring tool for chlamydia infection among sexual health clinic attendees in Australia: a simple algorithm to identify those at high risk of chlamydia infection. BMJ Open 2011;1:e000005. doi:10.1136/bmjopen-2010-000005
Competing interests None.
Contributors HW implemented the study, analysed the data and wrote the first draft. RG, BD and AM helped interpreting the data and finalising the manuscript. All authors saw and approved the final manuscript.
Provenance and peer review Not commissioned; externally peer reviewed.