Introduction The South African National Department of Health sought to improve syndromic management of sexually transmitted infections (STIs). Continuing medical education on STIs was delivered at primary healthcare (PHC) clinics using one of three training methods: (1) lecture, (2) computer and (3) paper-based. Clinics with training were compared with control clinics.
Methods Ten PHC clinics were randomly assigned to control and 10 to each training method arm. Clinicians participated in on-site training on six modules; two per week for three weeks. Each clinic was visited by three or four unannounced standardised patient (SP) actors pre-training and post-training. Male SPs reported symptoms of male urethritis syndrome and female SPs reported symptoms of vaginal discharge syndrome. Quality of healthcare was measured by whether or not clinicians completed five tasks: HIV test, genital exam, correct medications, condoms and partner notification.
Results An average of 31% of clinicians from each PHC attended each module. Quality of STI care was low. Pre-training (n=128) clinicians completed an average of 1.63 tasks. Post-training (n=114) they completed 1.73. There was no change in the number of STI tasks completed in the control arm and an 11% increase overall in the training arms relative to the control (ratio of relative risk (RRR)=1.11, 95% CI 0.67 to 1.84). Across training arms, there was a 26% increase (RRR=1.26, 95% CI 0.77 to 2.06) associated with lecture, 17% increase (RRR=1.17, 95% CI 0.59 to 2.28) with paper-based and 13% decrease (RRR=0.87, 95% CI 0.40 to 1.90) with computer arm relative to the control.
Conclusions Future interventions should address increasing training attendance and computer-based training effectiveness.
Trial registration number AEARCTR-0000668.
- SYNDROMIC MANAGEMENT
- HIV TESTING
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Syndromic management for sexually transmitted infections (STIs) diagnosis and treatment in primary care settings is recommended and widely used in many countries of Africa.1–9 Syndromic management is a presumptive diagnosis of STIs based on symptoms and easily recognised signs of infection and is typically directed by national treatment protocols.10 Protocols are regularly updated based on microbiological studies, as the prevalence of aetiological agents and treatment resistance profiles for STIs change. The South African National Department of Health (NDOH) updated the national STI guidelines in 2008 to reflect changing gonorrhoea resistance profiles by removing ciprofloxacin for gonorrhoea treatment and to add acyclovir as part of first-line therapy for genital ulcer syndrome.11
The NDOH sought to train health professionals on the revised STI national guidelines. The NDOH supported a decentralised approach to continuing medical education, which is implemented at the clinic level, such as educational outreach.12 ,13 The approach builds capacity and minimises absenteeism due to training. An STI training programme with three clinic-based training methods was developed: lecture, computer and paper-based.
Several articles have reported on the effectiveness of classroom-based STI syndromic management training,9 ,14 ,15 but less is known about the effectiveness of clinic-based programmes. In general, evidence shows that online learning is as effective as classroom methods.16 A pilot study of the STI training programme was conducted prior to national implementation with two objectives: (1) to test the feasibility of using each of the training methods for clinic-based teaching and (2) to conduct preliminary tests of their effectiveness. The pilot study was conducted at 40 primary healthcare (PHCs) clinics in North West Province that were randomly assigned to one of four arms; one for each training method and a delayed implementation arm that served as a control during the pilot.
Feasibility was assessed by the percentage of clinicians who attended training sessions. Effectiveness was tested with two measures pre-training and post-training: (1) STI knowledge test and (2) patient simulation by unannounced standardised patient (SP) actors. The STI knowledge test was completed by participants, but not by clinicians in the control arm. We tested the hypothesis that STI knowledge would increase post-training. SPs visits occurred at intervention and control clinics and measured quality of care. We tested the hypothesis that the pre-training to post training change in STI tasks performed by clinicians during SP encounters would be larger at clinics where STI training occurred relative to control clinics.
Sites and participants
In total, 40 stationary PHC clinics out of 74 in three subdistricts were purposefully selected for the pilot study and randomly assigned to four arms as shown in figure 1. PHC clinic size varied from 2 to 33 assigned clinicians and 40–230 patients per day. Two of the selected PHCs in one subdistrict provide 24 h services, three in the second and four in the third. Most other PHCs operate from 07:30 to 16:00, and a few operate 12 h per day, on weekdays. The district management team selected clinics with wider perceived gaps in STI management. Three exclusion criteria were: national STI clinical surveillance sites, clinics at correctional facilities, and non-governmental organisation clinics. Training programme participants were doctors and nurses assigned to the PHC clinics.
The six modules of the STI training programme and their learning objectives are listed in table 1. The modules were organised around clinical cases based on several recent reviews that concluded that interactive training was more effective than didactic.17 ,18 Each module was designed to be completed in 1 h. Computer-based modules are available at http://edgh.uw.edu/series/sexually-transmitted-infections and on compact disc. All training materials are in English, which is 1 of 11 official languages of South Africa.
Training frequency and time allocated was the same for all methods: two modules per week for 3 weeks for a total of six. Tests were given before and after the allocated time. The 2 h sessions were offered twice each week with the intention of training at least 90% of the clinicians at each facility.
All participants received a printed version of the national STI guidelines. For the lecture arm, a trainer taught two modules and gave participants handouts for those modules. For the computer arm, computers and assistance with navigation were provided. For the paper-based arm, participants received printed modules, which they could keep. No training or materials were provided to staff in the control clinics.
Feasibility was assessed by the number of clinicians who completed the pre-test for each module as a percentage of the number of clinicians at the clinic who consented to participate in the study. STI knowledge was measured with five, case-based, multiple choice questions per module. Module 4 was the exception because one question was withdrawn. Case-based questions began with a description of a case that the participants might encounter, and then posed questions about managing that case.19
We measured whether or not clinicians completed five STI management tasks during an unannounced SP encounter, and the primary outcome was total number completed. SPs are widely used in medical and nursing education in the USA,20 and unannounced SPs are considered the gold standard for measuring quality of care.21 Unannounced SPs have been used to measure outcomes of interventions for syndromic management of STIs in pharmacies22 ,23 and clinics.9 ,22 ,24–26
Professional actors who were fluent in Setswana attended a 3-day SP training and 1-day pilot in September 2013. Actors were trained on a standard script for vaginal discharge syndrome for women and male urethritis syndrome for men. All scripts were adapted to the actor's age, and other personal characteristics. To prevent potential bias due to acting quality, the SPs encounters were balanced, meaning that each SP visited a similar number of clinics in each arm during each time period.
After the encounters, SPs provided information about five STI tasks: correct medication, HIV test, condoms, partner notification slip and genital exam (see table 3 for exact wording). To avoid ingesting or removing drugs from the clinic, SPs disclosed at the end of the encounter. The clinicians recorded the drug name, dose, mode of administration, frequency and duration on a Medication Slip, and all five elements had to be correct.
Forty PHC clinics were randomised to four parallel arms (1:1:1:1 balance): arm 1 was control, arm 2 was lecture, arm 3 was computer and arm 4 was paper-based. Sites were randomised in strata to control for two characteristics of PHC clinics: subdistrict and operating hours, meaning 24 h services versus fewer hours. The randomisation was conducted on 30 September 2013 before the pre-training SP visits and knowledge tests.
A research coordinator met each SP after each encounter to record information about the five STI tasks on the SP Encounter Form and conduct a debriefing. Data from the SP Encounter Form, Medication Slip and SP Debriefing Questionnaire were entered using REDCap Software, V.5.8.2 (Vanderbilt University, Nashville, Tennessee, USA) and verified by comparing entered data against the paper forms. All medication data were checked by a clinician for accuracy. Total STI knowledge test scores for each clinic, module and time period were entered on a Microsoft Excel spreadsheet (Microsoft, Redmond, Washington, USA).
Descriptive statistics were calculated on training attendance and STI tasks performed in each time period and arm. STI knowledge test scores were compared pre/post with a two-sample t test. The hypothesis about changes in STI tasks completed was tested with the post training/pre-training relative risk (RR) of the number of tasks completed during SP encounters in the training arms compared to the RR of the control arm (see below). Analyses were performed with Stata V.11 (Statacorp, 2009 College Station, Texas, USA).
The comparison of STI knowledge test scores was conducted with complete cases, where the unit of analysis was the clinic module. A complete case meant that at least one person at the clinic completed the pre-test for a module and one person completed the post-test. A sensitivity analysis was conducted in which an average score of zero was assigned to clinic modules when no one at a clinic completed a test for the module and all clinic modules were included in the analysis.
Complete cases for SP encounters were analysed as binomial counts of the number of STI tasks performed correctly. We used a generalised linear model with a Poisson family and log link and main effects for arms, time period and their interaction, controlling for gender of the SP. The coefficients for arm and time were RR and the coefficient for Post*Arm was the ratio of RR (RRR) of the post training/pre-training RR in the training arm to the post training/pre-training RR in the control arm. All regression analyses were clustered on the clinic with robust SEs to adjust for using the Poisson instead of the binomial family. Results are presented with 95% CIs.
Two sensitivity analyses were conducted: controlling for clinic operating hours, and incomplete visits were coded as tasks not completed and all SP encounters were included in the analysis.
Figure 1 reports the flow for PHC clinics and training participants. Clinic managers and clinicians were recruited and enrolled from August to October 2013. Total number of SP encounters and number of visits by individual SPs were similar across arms and time periods because of their balanced distribution. There were two exceptions: (1) 32 incomplete visits were not included in the main analysis, and (2) two actors who visited PHC clinics before training were not available after training and were replaced by two actors who used their scripts. Reasons for incomplete visits by arm are reported in figure 1.
As shown in figure 1, an average of 22 of 71 (31%) of clinicians who consented to participate completed the pre-tests in the lecture arm, 15 of 59 (25%) in the computer arm and 25 of 65 (38%) in the paper-based arm. Participation on the pre-test was 17% at clinics with 24 h services and 40% at other clinics.
When all training methods were combined, average test scores increased from 52.5% pre-test to 65.7% post-test, an absolute increase of 13.2% (p<0.001), as shown in table 2. The increases were statistically significant for all participants for each module and for total score for each arm. The increase in total score was largest for the lecture arm. In sensitivity analyses, results were substantially the same as the complete cases analysis, but average test scores were lower especially for the post-test. Scores were missing for 37 of 240 clinic modules for the post-test compared with 10 for the pre-test.
STI tasks completed
The majority of SP encounters were with female clinicians (93%) and all but two clinicians were nurses (99.2%). Before training, the average number of STI tasks completed was 1.63 (median=1, IQR 0–3). Clinicians gave correct medications to 30% of SPs, offered an HIV test to 51%, provided male and/or female condoms to 22%, gave partner notification slips to 29% and offered genital exams to 39%. Despite the random assignment of PHC clinics to arms, there were differences in percentages of tasks completed across arms before training. Table 3 reports results by arm. Consequently, the ratio of RR or difference-in-difference analysis was necessary.
After training, the average number of STI tasks completed was 1.73 (median=2, IQR 0–3). There was no change in the number of STI tasks completed in the control arm. When all training arms were combined, there was an 11% increase in the number of STI tasks completed relative to the control arm (RRR=1.11, 95% CI 0.67 to 1.84). Looking at effects across training arms, there was a 26% increase (RRR=1.26, 95% CI 0.77 to 2.06) in STI tasks completed in the lecture arm relative to the control arm, a 17% increase (RRR=1.17, 95% CI 0.59 to 2.28) in the paper-based arm and a 13% decrease (RRR=0.87, 95% CI 0.40 to 1.90) in the computer arm.
Incorrect medications were prescribed for 88 of 121 (73%) women and 78 of 121 (64%) men. Among incorrect medications, 42 (48%) women and 25 (32%) men did not receive any drugs to treat an STI. The clinician prescribed ciprofloxacin instead of cefixime to 4 (5%) women and 35 (45%) men. Duration of doxycycline prescription was incorrect for 11 (13%) women and 3 (4%) men. Dose and frequency of metronidazole was incorrect for 21 (24%) and 13 (15%) women, respectively, and it was incorrectly prescribed to 10 (13%) men.
In sensitivity analyses, results were the same when controlling for clinic operating hours. Clinics with 24 h services were associated with a 16% decrease in the number of STI tasks completed (RR=0.84, 95% CI 0.58 to 1.21). Results were substantially the same in analysis of all SP encounters in which incomplete visits were coded as tasks that were not completed.
The pilot study identified challenges with attendance at clinic-based sessions across all training methods. Despite working closely with clinic managers to schedule training at convenient times, participation in the training programme overall was only 31%, and in the computer-based method was only 25%. Some clinicians may have been off duty during the trainings, as suggested by the relatively low participation at PHC clinics with 24 h services. Some clinicians were unfamiliar with computers and may have found this method challenging. STI knowledge test scores increased, however, suggesting that the training methods may have been effective among participants.
Lecture and paper-based training methods were associated with improvements in syndromic management of STIs. Computer-based training was associated with a decrease in the number of STI tasks completed. None of the effects were statistically significant. Some lectures were delivered in Setswana, whereas paper-based and computer-based trainings were in English.
The clinic-level design was based on 90% participation in training, with the expectation that most clinicians who treated a SP post-training would reflect the effects of the training programme. In practice, SP encounters were not necessarily with clinicians who participated in training. Although we did not identify the clinicians and link them to training records, future SPs could simply ask the clinicians if they participated in the training when they disclose at the end of the encounter.
The preliminary test of the effect of the training on STI tasks completed could be characterised as “absence of evidence, rather than evidence of absence” 27 because the sample size was too small. The pre-training mean number of STI tasks completed was 1.63 out of 5, with an SD of 1.48. In future tests, at least 251 SP visits would be needed per arm to detect an absolute increase of 0.5 STI tasks performed to 2.13, and 84 visits to detect an absolute increase of 1.0 STI tasks performed. Accounting for clustering of clinicians at facilities would increase the sample size. A difference-in-difference analysis as opposed to a comparison of means would also increase the sample size.
Similarity of the scripts across SPs of the same gender may have contributed to some clinicians suspecting the SPs' identity before she/he disclosed. During debriefings, 11 SPs (4.5%) reported that the clinician suspected his/her identity. In these encounters, clinicians may have performed better than normal, but even their best practice left room for improvement. Also, the clinics in the sample had wider perceived gaps in STI management than other clinics. Effects could be larger at other clinics that more readily adopt new skills or smaller at clinics with less room for improvement.
Comparison with previous studies
The quality of STI care was low. In previous research, a national PHC study reported that STI care was a well-established component of PHC services and available in >90% of PHC clinics in 1998, 2000 and 2003.28 A national survey of the quality of care reported that ciprofloxacin was available in 86% of PHC clinics and condoms were available in 92% and 79% of urban and rural PHC clinics, respectively.29 During SP visits in 2002, the percentage of STI tasks performed correctly was lower than in table 3 for three tasks: (1) physical exam was offered to 10% of women and 21% of men, (2) HIV test was offered to 8% of SPs and (3) partner notification slips were given to 18%.30 Condoms were provided without asking to 69% of SPs in 2002 compared with 32% among all SP post-training encounters.
Implications for future STI training
Clearly additional effort to improve the quality of STI care is warranted. Both paper-based and computer-based training methods could be used to train new staff and clinicians who work on shifts outside normal training hours. During the pilot study, a participant in the paper-based arm could access the content whenever and wherever she/he wanted, even though the pre-tests and post-tests were limited to the 2 h sessions at clinics. If the PHC clinic had routine access to a computer, participants could complete basic computer tutorials to improve their computer skills and then complete computer-based training. The voiceover for the computer-based content could be available in multiple languages.
Although the SP encounter was a data collection method for this pilot study, SPs are primarily a training method20 that supports participants to apply new clinical skills. In the future, when SPs disclose at the end of the encounter, they could give clinicians a handout about STI tasks. Or a training coordinator could provide constructive feedback to clinicians after the SP encounter.
Finally, the national STI training programme could be expanded to address health system challenges, such as stock-outs of cefixime, condoms and partner notification slips. A subsequent, national evaluation of STI service delivery found that oral cefixime was only available in 45.2% of health facilities (Pamela Kohler, personal communication). One solution tested by Colvin et al25 would be to provide ‘syndrome packets’, in addition to training, with correct antibiotics, 10 condoms, a partner notification card, and an information leaflet.
The South African National Department of Health supported clinic-based training for health professionals on revised sexually transmitted infection (STI) national guidelines. Three training methods were compared: lecture, paper and computer-based.
Attendance at clinic-based training was lower than expected, averaging 31% of clinicians.
STI knowledge scores increased on case-based tests across all three training methods, suggesting that the trainings were effective among participants.
The average number of STI tasks completed correctly was 1.63 out of 5 pre-training and increased by 11% post-training.
We are grateful to the clinicians who participated, the (Sub) District and Provincial Departments of Health in North West Province and the South Africa National Department of Health. We express our gratitude to Dr Albert Bakor, I-TECH South Africa country director, Harnik Gulati, I-TECH South Africa country team lead, and Lycia Zembe of the US Centers for Disease Control and Prevention. We also thankful to Elizabeth M Frick, Anya Nartker and Elizabeth Scott, who developed training materials, Robert McLaughlin for TrainSMART data, Gladys Mema, who coordinated the evaluation, Fulufhelo Malamatso and Sipho Mazibuko for leading the field teams, and the talented actors and actresses of Morgeez Casting Agency.
Handling editor Jackie A Cassell
Twitter Follow Erushka Pillay at @erushka1
Contributors Conception or design of the work: JG, SLJ, EM, SM, EN, GO'M, EP, VP and MRW. Design of the intervention: JG, SLJ, SM, GO'M, EP and VP. Data collection: JDK, SLJ, TO, EP and MRW. Analysis or interpretation of data: JDK, JG, SG, SLJ, EN, GO'M, EP and MRW. Drafting the work or revising it critically for important intellectual content: all authors. Final approval of the version to be published: all authors.
Funding This study and the activities detailed were developed and conducted by the University of Washington and I-TECH with funding from Cooperative Agreement U91HA06801-06-00 from the US Department of Health and Human Services, Health Resources and Services Administration (HRSA). The developers of REDCap were supported by grant UL1 RR025014 from National Center for Research Resources of the US Department of Health and Human Services, National Institutes of Health.
Competing interests None declared.
Patient consent Clinic managers participated in an informed consent process on behalf of the clinic and clinicians provided written consent for the training programme and unannounced SP visits.
Ethics approval The protocol and modifications were reviewed and approved by the South African Human Sciences Research Council (REC 1/22/08/12). University of Washington Human Subjects Division determined that this study did not meet the regulatory definition of research under 45 CFR 46.102(d).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The authors will make an anonymous data file of the SP visits available to other researchers upon request.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.