METHODS

Figure 1 shows the process we undertook. The Clinical Effectiveness Group chose quality indicators reflecting the process of clinical management and partner notification for gonorrhoea and chlamydia and decided on the following principles for setting the outcome standards:

• To derive figures for each standard using available published and unpublished evidence and opinions solicited from individual specialists across the United Kingdom.

• The standards should not be aspirational targets unlikely to be achieved in the real world, nor so low as not to detect a failing service. They should reflect targets consistently achievable by well resourced and adequately run clinics.

• To consider factors that may influence clinics’ ability to meet the standards—for example, patient demographics, geography, resources, efficient management, etc.

Table 1 describes the indicators selected to reflect the quality of care. The literature review and evidence synthesis were then carried out to derive numerical standards for the parameters x (cure, treatment or partner notification rates) and y (time limit) for each standard.

Literature search

The box shows the search strategy. Data from unblinded published and unpublished reports were extracted by one reviewer using a standard proforma that recorded details of the publication, study population, numbers of participants, and the parameters x and y for each indicator. Some centres were included in multiple reports so only the most recent estimate from each was included in analyses. Data from multicentre reports were disaggregated where possible to obtain information from individual centres. Where individual clinic data could not be extracted—for example, from the National Gonorrhoea Audit,^8,9 we used unweighted figures, stratified into London and non-London clinics when possible. This means that clinics contributing to the national audit and to another study will therefore be duplicated.

Evidence synthesis

Pooling of data from individual centres

For each parameter and each indicator we calculated the average (with 95% confidence intervals) of results from all relevant studies, weighted according to the number of cases analysed at each clinic. Evidence of variation between estimates from different centres beyond that expected by chance was examined by performing a statistical test of heterogeneity (χ² test). If there was evidence of heterogeneity (p<0.1) the weighted average was calculated using random effects models that allow for variation. We then stratified results according to geographical location (London or large city v other). The exact figures for the standards were rounded up to the nearest 10%.

We compared our results with the opinions of experts who were asked to specify desirable standards for each indicator.

Exploration of between centre variation

We plotted data about numbers of patients cured and contacts treated per case in each centre using “Shewhart’s control charts,” a graphical method that allows different sources of variation in performance to be examined. The control charts were created using methods previously described.⁷ They present data points for each clinic together with three lines: the mean and upper and lower limits. The upper and lower limits represent the limits of common cause variation, taken as three standard deviations from the mean. Points falling outside these limits suggest special cause variation.⁷

RESULTS

Setting standards

The search strategy (box) yielded 25 published and unpublished reports. These included 17 reporting gonorrhoea management, including two from a national audit^8–24 and 14 about chlamydia management (table 2).^{15,16,19,20,23–32}

Gonorrhoea

The 17 reports included results from 50 different departments of genitourinary medicine in 15 reports^10–24 and two reports from the National Gonorrhoea Audit of 155 departments, 22 in London and 133 outside London.^8,9 The majority of reports were from centres outside London.

Cure rates (x)

Seven reports^{8,11,13,17,19,21,23} including 2563 patients contained sufficient data to calculate cure rates (table 3). Two further studies reported the numbers of patients returning for a test of cure but not the results.^15,22 Two multicentre reports did not stratify results by geographical location so we stratified data into London clinics,^21,23 provincial clinics,^13,19 and combined rates.^8,11,17 The results from the National Gonorrhoea Audit⁸ were consistent with those from other centres.

Table 3 shows that cure rates calculated as defined by the Clinical Effectiveness Group (proportion with negative tests of cure out of all patients diagnosed, table 1) ranged from 42.9% to 76.0% and differed between centres (test for between centre heterogeneity, p<0.001). The proportion of patients returning for a test of cure was much lower in London (weighted mean 55.8%) than outside (weighted mean 78.4%). After stratifying by location heterogeneity in cure rates between non-London clinics remained (range 54.5–76.0%).

The efficacy of antibiotic treatment, calculated as the number of patients cured as a proportion of those who returned for a test of cure, was 94.7% (95% CI, 92.8 to 96.6), with no evidence of heterogeneity between studies carried out in and outside London (p = 0.203).

Time between diagnosis and cure (y)

Eight studies reported the average time between diagnosis and test of cure, which was from 2^8,10,11,21 to 4 weeks.^15,19,22,23

Partner notification (x)

Twelve studies reported the number of sexual contacts screened for gonorrhoea^{8,12–14,16,17,19,20,22–24} (table 4) but no study reported the proportion confirmed to have been treated or returning for follow up.

The pooled data show that, overall, 0.58 contacts per case of gonorrhoea are screened (table 4). Fewer contacts per case are screened inside London than outside but confidence intervals for these estimates overlap. Geographical location did not explain all the heterogeneity between studies since strong evidence of heterogeneity remained after stratification (p<0.001, estimates ranged from 0.14 to 0.60 contacts per case in London and 0.23 to 0.99 contacts per case outside London).

Time between notification and screening of contacts (y)

Only one study reported the time between diagnosis and screening of contacts¹⁵ as 4 weeks.

The proposed outcome standards are summarised in table 5. We rounded figures to the nearest 10% for simplicity.

Chlamydia

Fourteen studies reported on chlamydia outcomes in a total of 10 860 patients at 82 different clinics.^{15,16,19,20,23–32}

Satisfactory treatment (x)

Most studies reported the proportion of diagnosed cases receiving treatment but information about satisfactory completion of treatment was much less well documented. One small study from London reported the proportion of patients satisfactorily completing treatment for chlamydia.²³ Of 35 female patients seen in 1 month 11 returned for follow up and eight (22.9%, 95% CI 10.4 to 40.1) had completed treatment satisfactorily. Another study from Durham found that 46/209 (22%, 95% CI 16.6 to 28.2) female patients required re-treatment, particularly if they had been referred by a general practitioner. Three studies (from London, Wessex, and Yorkshire) found that, overall, 63.7% (95% CI 56.6 to 71.7) of patients returned for a test of cure,^23,26,27 with strong evidence for heterogeneity between studies (p<0.001). These studies did not report whether or not treatment had been completed satisfactorily. The Wessex audit, however, reported results of tests of cure performed at the follow up visit in 172 patients from six clinics.²⁷ 71.0% (95% CI 64.5 to 78.0) of patients could be established as being cured, by this definition. Using these data, which estimated the chosen outcome standard indirectly, we set the values for satisfactory treatment of chlamydia at the same levels as for cure of gonorrhoea.

Time from diagnosis to follow up (y)

The time period between diagnosis and follow up was reported in two studies as 4 or 5 weeks.^23,26

Partner notification (x)

Twelve studies reported the number of contacts screened per case of chlamydia diagnosed.^{16,19,20,23–28,30–32} These studies involved 6877 patients in seven single centre studies (2085 patients) and five multicentre audits (4792 patients, 66 centres).

Table 4 shows that overall, 0.61 (95% CI 0.55 to 0.66) contacts per chlamydia case diagnosed were screened. These data could be stratified as large city (London, Birmingham, Manchester) or other clinic locations. In large city clinics 0.43 (95% CI 0.30 to 0.62) contacts per case were screened, compared with 0.64 95% CI 0.58 to 0.70) contacts per case in other clinics. There was strong evidence for between clinic heterogeneity in all settings. These figures were similar to those for gonorrhoea and the same standards were set for both (table 5).

Time from notification and screening of contacts (y)

The time between diagnosis and screening of contacts was 4 weeks in two studies^16,27 and 5 weeks in another.²³

Expert opinion

Table 6 shows the opinions of GUM physicians and representatives of national and regional organisations. When compared with the pooled results of audit data individuals’ expectations for performance were higher than the achieved outcomes. This is exemplified by experience from Scotland (GR Scott, personal communication) where audit standards for gonorrhoea and chlamydia proposed that 90% of patients should be satisfactorily treatment and 50% of partners notified. The results showed that 71% of gonorrhoea and 55% of chlamydia cases achieved the standard for satisfactory treatment and partners of 32% of gonorrhoea and 27% of chlamydia cases were satisfactorily managed.

Variation between centres

Figures 2 and 3 show control charts plotted to display variation in the outcomes of treatment and partner notification between different centres in the reports retrieved from the literature search. Points lying outside the control limits (broken lines) suggest a special cause for this degree of variation. Points lying above the upper control limit show high cure or partner notification rates and knowledge of why these centres perform well could be used to help improve practice in other centres, particularly those below the lower control limit.

In figure 2 the points lying clearly below the lower control limit are in London. Those on the line are outside London. For chlamydia (fig 2B) the horizontal axis shows the numbers of people returning for follow up, which is used as a proxy for the number satisfactorily completing treatment.

Figure 3 shows substantial variation between clinics in the outcome of partner notification. Most points lie within the control upper and lower control limits, however, suggesting common cause variation. Special cause variation is suggested in one London and one non-London centre.

DISCUSSION

We derived standards of management for gonorrhoea and chlamydia in GUM clinics using a process coordinated by an expert professional group that included a systematic review of published and unpublished literature and peer consultation. The evidence based standards were considerably lower than if they had been based on the expert opinion. We found good evidence on which to base standards for satisfactory treatment for gonorrhoea and rates of partner notification for gonorrhoea and chlamydia. There was, however, little empirical evidence about satisfactory treatment for chlamydial infections and the times within which follow up for either infection should be conducted. There was wide variation in outcomes achieved in different clinics, most of which is the result of common causes. The data suggest special causes for variation in the rates of follow up for chlamydia and gonorrhoea cure and these should be investigated further.

Limitations of the data and methods

In any systematic review it is possible to miss reports, particularly unpublished data. The data analysed and reported here may therefore be incomplete but they include a large number of diverse centres across the country. More importantly, the reports retrieved often did not provide the information required. In particular, audits of the management of chlamydial infections collected data about the number of patients returning for follow up but the number of patients requiring re-treatment, which is a better measure of the adequacy of management, was rarely reported. Times over which outcomes were measured were usually not reported. These data should be collected in future audit reports now that standards have been defined.

Some of the reports from multicentre audits did not provide the results from individual centres. They usually presented simple proportions—that is, estimates that were not weighted for the number of observations from each centre. The inclusion of these unweighted data ignores some of the variability between centres, which could bias the pooled estimates.

Choice of standards

For each infection we presented a weighted average of the proportion of cases and the number of contacts per case managed satisfactorily. We found substantial heterogeneity in outcomes achieved between clinics, some of which was due to geographical location. We therefore chose different standards for clinics in London and those outside. The reasons for less complete follow up in London clinics include a more mobile population, greater clinic workload and longer waiting times, which may discourage reattendance. There may be areas in which practice could be changed to facilitate follow up or ascertainment of compliance with treatment, including telephone calls, text messaging, and use of outreach health advisers and nurses. The use of audit should enable clinics to evaluate the effects of changes in practice.

The outcome of contact tracing was based on total numbers of sexual contacts ascertained although it is known that some contacts can not realistically be traced and others have already presented as index cases and been treated. There were insufficient data in the studies reported to determine outcomes for different categories of contacts and most reports did not state the basis for calculating this variable. The wording of the standards reflects current practice, although it is recognised that other measures such as the total proportion of contacts screened may be more relevant for disease control. Future audits should collect data on a range of contact tracing outcomes so that standards can be modified appropriately.

Variability in outcomes between clinics

As discussed above some of the variation between clinics was explained by geographical location. There was, however, residual heterogeneity that was not due to geographical location and that could represent unmeasured factors in the process of care. The control charts (figs 2 and 3) provide information that gives clues as to the source, if not the cause, of variation. In each case the clinics can be divided into three groups.⁷ Firstly, data points above the upper control limit show well performing clinics. Reasons for their success (special causes) could be investigated and modifiable factors could help improve performance in other clinics. Secondly, data points lying between the control limits show variations in performance that are the result of common causes. Examining the underlying processes of care in all clinics could identify modifiable factors, which could be used to reduce variation and improve performance overall. Thirdly, data points below the lower control limit show clinics with special causes responsible for their poor performance. Identifying these factors could help eliminate them and improve quality of care.

The control charts presented here show that most variation in the outcomes related to cure and follow up has common causes but special cause variation also exists. We investigated geographical location as one reason for special cause variation. The stratified analyses complement the findings in the control charts. The lowest cure or follow up rates were from London clinics and these clinics were also below the lower limit of the control chart. The results of contact tracing outcomes also show fewer contacts treated per case in London clinics. Reviews of partner notification processes across clinics and identification of factors associated with improved outcomes would therefore be more likely to reduce between clinic variations.

Monitoring clinic performance

The proposed outcome standards (table 5) can be used in conjunction with the control charts. The standard represents the weighted average performance target. Future national audits could measure performance against best practice targets as well as the standards suggested here by the Clinical Effectiveness Group. The actual performance of clinics will generally lie above or below the average. A control chart plotted for all clinics would then show if performance lies outside the limits, in which case causes of special variation should be sought to explain particularly good or poor performance. The profession as a whole should examine the process of care to look for causes of common cause variation and to try to reduce these to improve the consistency of care. As successive audit rounds and changes in practice occur the average performance target would be expected to increase and the control limits become narrower with fewer clinics below the lower limit.

In summary, we have defined evidence based outcome standards for gonorrhoea and chlamydia. We have also shown how a method for examining variation in performance between clinics can be applied to monitor performance and could improve consistency and quality of care over time.