Unlocking the potential: longitudinal audit finds multifaceted education for general practice increases HIV testing and diagnosis
- 1University College London, Medical School, London, UK
- 2Sexual Health In Practice, Heart of England NHS Foundation Trust, Birmingham, UK
- 3Research Department of Infection and Population Health, University College London, London, UK
- 4Killick Street Health Centre, Islington, London, UK
- Correspondence to Timesh D Pillay, University College London, Medical School, Rockefeller Building, University Street, London WC1E 6BT, UK;
- Accepted 23 August 2012
- Published Online First 8 October 2012
Background This longitudinal study aimed to evaluate the impact of a multifaceted educational intervention (Sexual Health in Practice, SHIP) on general practice HIV testing rates in a high prevalence London area.
Intervention SHIP offered training in sexual health clinical skills to general practitioners (GPs) and practice nurses (PNs) in Haringey. SHIP training aims to break down stigma in sexual health and provide sexual history and communication tools (e.g. differential diagnosis), and provides resources to practices (including condoms).
Design Numbers of GP HIV tests were collected from laboratories for 24 months prior, 19 months during and 5 months after training. Attendance data and practice list sizes were obtained.
Results 39 of 51 practices had at least one trained individual. These `trained' practices conducted an average 526 HIV tests p.a. before training began which rose to a projected 1556 p.a. (on the basis of the last 6 months of data). Testing rates of trained and untrained practices increased from 2.29 to 6.66 and 1.54 to 1.90 tests/1000 registered patients/year (p=0.0016 and p=0.5195) respectively. The rate of positive diagnosis was high in the trained group (18.0 and 16.7 positives/1000 tests before and after training began; p=0.7908). This equates to a rise from 9.5 to 22 new diagnoses p.a.
Conclusions The training intervention has been found to significantly increase general practice HIV testing rates in the absence of financial incentives. Positivity rates are substantially higher than that found in pilots of screening in London, suggesting that the training nurtured and supplemented complex clinical skills.
A quarter of those with HIV in the UK are undiagnosed.1 Those diagnosed late are more likely to die and to develop serious illness.2 There is also a higher risk of onward transmission of the virus, which can be prevented with early treatment.3 HIV testing in the UK must, therefore, increase substantially so as to lead to improvement.
It is recognised that general practice is highly accessible and well used by people, including those in key groups at higher risk of HIV,2 ,4–6 as well as by people with relevant symptoms.5 ,7 In response to UK national HIV testing guidelines,8 screening of newly registering patients in general practices in high prevalence areas has been piloted. However, a Department of Health review of the findings9 concluded that there should be further investigation of models for testing in general practice ‘since not all primary care facilities conduct new patient health checks’. In a recent House of Lords report on HIV,10 ,11 it was noted that ‘professionals, most notably general practitioners (GP), must become more confident and competent in [HIV testing]. Training and education are important tools to use to achieve this … [and] must incorporate efforts to address HIV-related stigma’. Similar problems are identified, and solutions proposed, at a Europe-wide level.12
Guidelines and educational approaches8 ,13–15 have been used to try to increase GP HIV testing rates. Written material is highly valuable, as it brings consensus and sets standards. However, Cochrane evidence reviews conclude that there is no evidence that ‘printed educational materials’ had an impact on patient outcomes,16 and the effects of educational meetings on complex clinical behaviours is small.17 By contrast, multifaceted interventions combining, for instance, didactic and interactive education with other approaches (eg, resource provision, feedback to clinicians, audit and use of local clinical leads) may effect change,18–21 and a close link with the clinical context may help.22 One UK study of a Sexually Transmitted Infection Foundation course23—a mixed interactive and didactic course for those with a clinical interest—found no increase in HIV testing post-training. There is currently no basis for assuming that any intervention will change a complex clinical behaviour, such as GP HIV testing.
In this paper, we report on the effect of a multifaceted educational intervention, SHIP (Sexual Health in Practice, Heart of England NHS Trust) on GP HIV testing rates. SHIP addresses all aspects of sexual health including sexually transmissible infections, health promotion and contraception. With respect to HIV, it aims to remove the barriers to HIV testing and teaches a systematic clinical approach to testing (see online supplementary document).
This paper evaluates the effect of the SHIP intervention on general practice HIV testing rates. For this audit, attendance at the sessions relevant to HIV testing only were considered (see online supplementary document).
SHIP teaches a cohesive clinical model for sexual health care in general practice. This includes approaches to the four situations in which HIV tests may be offered or carried out:
Opportunistic testing for those found to be at risk through history taking
Diagnostic testing for selected patients with relevant symptoms
Screening (eg, those referred for abortion or, in high prevalence areas, seeking contraception).
To provide HIV tests, GPs and practice nurses (PNs) need good clinical knowledge, effective communication skills and also risk assessment and sexual history-taking skills. To apply these, confidence and motivation is needed. There are multiple perceived and actual barriers to all forms of HIV testing in general practice.12 ,15 Overcoming these is vital to improving testing rates.15 ,24
Kirkpatrick outlined a hierarchy for the evaluation of clinical learning in response to educational interventions,25–28 number 1 being the least valuable:
Participation: numbers and types of participants
Reaction of participants
Learning: knowledge, skills, attitudes, confidence, commitment
Changes in practice and behaviour
Outcomes for patients
Return on expectations—what degree of change would indicate success? What is the cost-benefit analysis?
This hierarchy offers a stark reminder that the responses of participants, while they may help improve and develop training, are in fact a weak test of effectiveness.
In an earlier publication,29 SHIP demonstrated the value of its training by the use of Kirkpatrick's outcomes 1 through 3. In Haringey, general practice laboratory HIV testing rates were selected as an outcome measure because they are:
clinically significant, with potential to reduce undiagnosed HIV (if positivity rates are high)
not ‘self-reported’ by practices
an indication of reduction in stigma associated with HIV testing and improved communication in consultations.
An increase in GP HIV testing rates might, therefore, fulfil Kirkpatrick's Levels 4 and 5.
Birmingham SHIP was commissioned by Haringey Primary Care Trust (PCT), North London, to support their Sexual Health Strategy (see online supplementary document). The 51 medical practices within Haringey are in four collaboratives (approximately geographical groups). Haringey PCT has a population of around 270 000 registered for healthcare, and has the 12th highest prevalence of HIV nationally (6.8/1000 adults aged 15–59 in 2009).30 Incentives to attend training included access to free condoms and pregnancy testing kits and locum costs. No incentives were offered for HIV testing.
SHIP training in Haringey included five rounds of four afternoon sessions. These were run in March and June of 2010 and January, June, September and October 2011 (see online supplementary document).
The total registered population in Haringey was 271 697 in April 2011. A PCT list of Haringey practices and clinical staff and their numbers of registered patients (list size) was obtained.
Haringey GP HIV tests are sent to one of three laboratories: North Middlesex, Whittington or Homerton hospitals. Monthly, numbers of tests by practice were requested from laboratory managers for 24 months retrospectively, then prospectively through 19 months of training, and 5 months post-training, totalling 48 months of data. Equivocal results were deleted on the assumption they were repeated. Laboratories were asked to identify repeat positives (ie, with identical patient ID) which were removed.
Relevant training for GPs consisted of two afternoon sessions, the first addressing clinical and communication skills relevant to sexual health, and the second focusing on HIV. PNs attended three afternoon sessions so that sexual health promotion skills could also be taught (see online supplementary document). Training attendance data was collected from sign-in sheets at the training sessions, and time of departure of early leavers noted. For the sake of comparison, practices were defined as either ‘untrained’ practices (those practices with no clinical staff to complete all relevant sessions, even if some had attended one session) or ‘trained’ (any practice with at least one health professional who had attended all relevant sessions). Three practices merged during the follow-up period, and their data was merged and treated as one practice throughout. One practice of <1000 patients closed during the study. It was excluded from analysis.
A linear regression analysis was performed to investigate whether the monthly number of Haringey HIV tests increased over time. An interaction term was fitted to investigate whether the change in the number of tests performed was significantly different once SHIP training was introduced. Next, the rate of HIV testing was calculated by dividing the total number of tests performed in a month by the list size of the practice. These rates were then rescaled to give the rate of testing per 1000 patients per month. The rate of testing at each practice in the period before SHIP was stratified by whether practices went on to become ‘trained’ practices or not. The rates of HIV testing for the latest 6 months of data were then calculated, and compared with the preintervention rates using Wilcoxon and Mann–Whitney tests, as the rates were not normally distributed. The proportion of all HIV tests that were positive before and after the intervention was calculated and compared using a χ2 test. Finally, Poisson regression was used to investigate the impact of SHIP training on the rate of testing taking into account the month that the practice trained. A term was fitted to investigate the average change in the practice rate of testing once they had attended SHIP. Generalised Estimating Equations were included to account for the fact that each practice contributed data in more than 1 month. March 2010 data was a transitional month because relevant skills for HIV testing were taught on the 4th but no one had completed training until the 25th. This month was removed from all statistical data analysis except from the graph of monthly tests, the formal test for interaction, and the Poisson distribution analysis. The statistical package used was SAS V.9.3 (SAS Institute Inc, Cary, North Carolina, USA).
At no point was patient-identifiable information collected, and there was no randomisation.
Table 1 summarises attendance by staff over five rounds of training. Of a total of 150 participants, three participants left a training session early, but were classed as full attendance. By the end of five rounds of training, 19 of the 39 trained practices had multiple attenders. Table 2 compares trained with untrained practices by population coverage.
Results of the linear regression model suggest that, during the 24-month pretraining period, the number of HIV tests performed in Haringey was slowly increasing at a non-significant rate, with an extra 0.1 tests performed per month (95% CI −0.3, +0.6; p=0.59). When considering the time period after the introduction of SHIP, the number of tests performed increased at an estimated extra 3.5 tests per month (95% CI +2.7, +4.4; p<0.0001). A formal test for interaction considering the entire follow-up period demonstrated that introduction of SHIP was associated with a significant increase in the number of HIV tests performed in Haringey PCT (p=0.0004). It can also be observed from figure 1 that testing appears to increase after each training round, apart from round 1. The number of tests conducted dipped in December 2010 and December 2011 (figure 1), probably due to a reduction in non-urgent clinical activity around Christmas.
Table 3 shows that the rate of testing among trained practices increased between the 24 months prior to SHIP and the last 6 months, a statistically significant change (Wilcoxon test p=0.0016). By contrast, there was no evidence of an increase in HIV testing rates in the untrained group (p=0.5195). The total testing rate for Haringey PCT also significantly increased (p=0.0014). Prior to SHIP, there was a significant difference in testing rates between the untrained and trained practices (Mann–Whitney U test p=0.0137).
A Poisson distribution analysis, taking into account the staggered nature of the intervention and using all 48 months of data, showed that there was a significant effect of completion of training on the rate of tests performed. A 14% increase in the number of tests performed is observed every 6 months after training started (rate ratio=1.14; 95% CI 1.07 to 1.22; p=0.0002).
Table 4 illustrates the wide range in diagnosed HIV prevalence within Haringey, with northeast and southeast collaboratives serving extremely high prevalence areas. Estimates of undiagnosed HIV in London were thought to add another 26% to these figures in 2009.30
The number of positive HIV tests in trained practices increased from 19 over the 2 years before training to 39 in the 2 years since training began. There is no evidence of a difference between the rate of HIV positive tests over time (χ2 test statistic 0.07, p=0.7908), though numbers are small (table 5).
The SHIP intervention produced a substantial effect. With the training of 27% of doctors and 22% of nurses in Haringey, SHIP was associated with an increase in HIV testing rates (p=0.0004) and a high rate of positives (16.7/1000). Furthermore, a large proportion of all Haringey practices (20/51) serving 59% of Haringey's registered population, increased their testing rates by more than 50%. The biggest increases in testing rates (350%) were seen in the northeast collaborative, the larger of the two most deprived areas of Haringey, and with the highest prevalence of HIV with only 11 of 15 having had a member of staff trained.
These findings support the conclusion that SHIP, as an educational intervention, led to changes in clinical practice and behaviour (Level 4 Kirkpatrick's Hierarchy). Furthermore, because additional HIV-infected patients appear to have been diagnosed as a result of the SHIP intervention, improved outcomes for patients have been demonstrated (Level 5). It is unusual for clinical educational interventions to achieve this level. It is not possible, however, to state whether any of these HIV diagnoses were achieved early (which would be of greatest benefit to the individual and the population). In addition, only a controlled study, including economic evaluation, could fulfil Kirkpatrick's Level 6—a cost-benefit analysis. The prevalence of HIV in the area would affect any such analysis, although SHIP training may be of value in lower prevalence areas because late diagnosis of HIV is more common,31 and training may benefit other aspects of sexual health.
This longitudinal clinical audit has a number of strengths. The study involved the collation of data over an extended period of time and was able to reveal trends for 2 years before and after training. The selected outcome measure (numbers of general practice HIV tests) is meaningful to patient outcomes. The intervention increased general clinical HIV testing in general practice without financial incentive to test (only to train). Indeed, it is possible that the provision of payment per test, outside a screening programme, does not actually help practitioners overcome some of the substantial barriers to testing.
There are several limitations to this study. The effect of the training intervention was assessed by longitudinal clinical audit rather than a randomised control trial (RCT). However, this was the best method available within the timescales and funding of this commissioned intervention. Furthermore, there are advantages to measuring the impact of an educational intervention in the real-life primary care environment, potentially not measurable with a tightly controlled RCT. The ‘first-come-first-served’ system of enrolment for SHIP training attracted staff from practices with a slightly higher baseline testing rate, meaning untrained practices were not necessarily comparable and could not, therefore, act as a control group. The number of tests per month conducted in Haringey demonstrated a small increase just before training began. This might be explained by SHIP advertising and correspondence from December 2009, as has been shown in the past with educational material sent by post.32 The broader trend does not appear to be explained by other interventions. While, for example, UK National Guidelines for HIV Testing8 have been introduced, neither this nor any other local or national intervention we could find coincided with the introduction of SHIP in Haringey. The intervention was spread over a 19-month period, making ‘before-after’ comparison difficult.
Recruitment to SHIP training varied, and the last course in the period of study showed a dip in attendance, although the most recent training (run after the end of the study period of this paper) showed a return to the usual numbers of attendees.
The 6 months’ data used in some subanalyses as the ‘after’ period may not be representative of a year's testing data, and is likely to underestimate yearly testing rates due to the observed annual fall in HIV testing in December. In addition, while individual GP and PN attendance data was collected, only whole-practice HIV testing data was available. This means that the effect of training may have been diluted in the practice setting, particularly in the larger practices if only one person had completed training. Finally, some practices had not yet sent a clinical member of staff to be trained, and some did, but showed no response to training (some of these performing no HIV tests). This study is not able to elucidate why this was the case.
Unlike other interventions,23 SHIP aims to attract GPs and PNs whether or not they have an interest in sexual health. The evidence of benefit presented here echoes the conclusions of Cochrane reviews on educational interventions, and suggests that a systematic tackling of health professionals’ perceived barriers to testing with the support of peer-educators, are important factors.
It is likely that further gains will be made in HIV testing, if SHIP training continues, although the rate of increase may slow if the most motivated staff have trained first. Thus, 65% of the 20 higher-performing trained practices and 32% of 19 lower-performing trained practices had multiple attenders at training, suggesting that having more than one trained individual in a practice reinforces change.
The cumulative increases in testing over time suggest that the impact of each round of training is sustained. Individual practice HIV testing rates may also be responsive when a new diagnosis of HIV is made in the practice, reinforcing the value of changed clinical practice. Several new positives were found in the northeast collaborative, and this may be one factor in the dramatic increases in testing rates seen in this very high prevalence area. Numbers of new HIV positives are not high enough to enable closer examination of this issue. Qualitative work could give insights into this.
Rates of positives were substantially above levels achieved with screening trials.9 However, while duplicate positives were removed from our dataset by the laboratories, it is possible that some ‘new’ diagnoses of HIV counted here were in fact in patients who already knew they were HIV positive. This cannot be explored here due to the simple study design, but we think it unlikely to account for many—or indeed any—of the cases. However, if some patients used new opportunities for HIV testing to help them ‘disclose’ their HIV status to their GP, this is likely to be a good thing for their future clinical care. The high rates of positivity, and high increases in testing rates in the highest prevalence area, may reflect the nature of the intervention, which aims to teach a range of clinical skills to help clinicians select patients who will be offered a HIV test. These skills include employing strategies for the clinical management of those with relevant symptoms and the use of risk assessment for opportunistic offers of testing.
The findings from this study have implications for policy makers. Commissioning of SHIP training for local practices seems likely to support the implementation of increased HIV testing and diagnosis in the primary care setting. Evidence given here suggests that SHIP training also changes clinical practice most in the areas with the highest need. Considering a recent statement from the House of Lords that the removal of stigma around HIV will be a slow process over a number of years,10 our findings suggest a faster route, at least towards its reduction.
The SHIP measure of HIV testing rates (the number of tests per annum per 1000 registered patients) provides a simple shorthand rate that practice staff can calculate for themselves, and forms a useful basis for discussion and education. Such rates are outside the remit of existing antenatal, new patient or other screening programmes. On the basis of the individual Haringey practices testing at the highest rate, a guide figure for high prevalence areas might be over 30 tests/1000 patients per annum. This proposed benchmark takes no account of population age distribution, or local HIV prevalence. It is simply a guide figure to stimulate reflection for practitioners considering what might be appropriate for their own practice, given its own individual circumstances. It is not intended as a measure of quality of care.
Further investigation into this educational intervention is indicated, perhaps using a step-wedged randomised—or randomised controlled—trial. Areas for further study include a comprehensive cost-effectiveness analysis of SHIP training (encompassing other sexual health outcomes of SHIP); qualitative research to better understand clinicians’ response to training (including non-response); and whether the training can have an impact on early diagnosis of HIV infection.
Sexual Health in Practice is a multifaceted training programme for general practices implemented in a high prevalence area of North London
It was associated with a substantial increase in the number of HIV tests done over a 19-month period of five training rounds (formal test for interaction: p=0.0004).
Numbers of positives identified by general practice rose from an average 9.5 per annum before training to a projected 22 per annum (on the basis of the last 6 months’ data).
The highest increases in HIV testing were seen in the locality with the highest prevalence of HIV.
Thanks to Vicky Hobart and Michele Guimarin for initially commissioning SHIP in Haringey, and Susan Otiti for her continuing support. Special thanks to Christopher Buckingham from Whittington Hospital lab; ZareenaAsif, Josephine Silles and Julie Prendergast at North Middlesex Hospital lab; and John Gooch at Homerton Hospital laboratory for all their help with HIV testing data collection. Thanks too to all the SHIP trainers in Haringey, and all GP and PN training participants. Thanks to the Haringey Learning and Development Team, and the St Ann's Sexual Health team including Helen Munro. Thanks also to Ruth Hutt and Allon Gould for their helpful comments.
Contributors TDP coordinated the writing of the paper; JM contributed information regarding the intervention; CJS managed the statistical analysis; PM provided guidance regarding the broad aims of the paper. All authors wrote parts of the final document.
Funding SHIP training in Haringey was funded by Haringey PCT. SHIP Birmingham funded data collection for this evaluation and funded training of the Haringey peer-educators. The researchers for this article are independent from Haringey PCT. No additional funding was received.
Competing interests None.
Ethics approval The local ethics committee criteria classified this study as a combination of clinical audit and service evaluation, and therefore, formal approval was not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement All authors had full access to relevant data and can ensure its validity and the accuracy of data analysis. Consent for use of data was not given, but the data is anonymised (so that detailed practice data cannot be obtained), and so, the risk of practice identification is very low, and there is no risk of patient identification. The data is available on request from the corresponding author, contact email@example.com. Due to the identifying nature of practice list sizes, only testing rates (not absolute numbers) are available.
Open Access This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/3.0/ and http://creativecommons.org/licenses/by-nc/3.0/legalcode