A randomised controlled trial of the effectiveness of combining video role play with traditional methods of delivering undergraduate medical education
- 1Academic Department of Genitourinary Medicine, GKT, St Thomas's Hospital, London SEI 7EH, UK
- 2School of Education, University of Greenwich, London, UK
- Dr Barry
- Accepted 7 June 2001
Objective: To determine the effectiveness of video role play with structured feedback in improving undergraduate communication skills and application of knowledge in genitourinary medicine.
Design: A blind, randomised, controlled trial.
Subjects and setting: Fourth year undergraduates attending a 5 week attachment in genitourinary medicine during 1997 at a London medical school.
Intervention: A randomly selected sample group of undergraduates were filmed in the role of a doctor interviewing a patient (played by an actor) presenting with a genitourinary (GU) problem. Structured feedback by a GU physician and an educational psychologist was given a week later. The control group of undergraduates did not receive this training intervention.
Outcome measures: Student performance in two stations of an objective structured clinical examination (OSCE), administered at the end of their attachment. This tested communication skills and knowledge in GU settings.
Results: 132 undergraduates were assessed in the OSCE. 40 of these were in the sample group who received training using video role play with feedback and 92 were in the control group. The sample group scored significantly higher marks than the control group (p<0.001).
Conclusions: Video role play with structured feedback is effective in improving undergraduate communication skills and application of knowledge in GU medicine settings.
The United Kingdom is in the middle of a major change in undergraduate medical education which will be compounded over the coming years by a large increase in the number of medical students. University medical schools have planned new curricula, underpinned by core knowledge requirements with individual teachers or departments left to decide on their teaching methods. These methods depend on the availability of resources (time, equipment, money, and the support of other professionals) and familiarity with different teaching techniques. It is rare that the teacher has evidence of the comparative effectiveness of different training techniques to inform this selection. Indeed, in most clinical settings the different approaches have not been evaluated. There is a general shift towards delivering these curricula by using approaches such as problem oriented learning. Problem oriented learning using real life clinical situations as a framework for teaching, learning, and assessment is strongly encouraged by the General Medical Council in the training of undergraduates.1 Both the General Medical Council and the King's Fund highlight the need to reduce the factual load in traditional medical curricula, emphasising instead student skills in self directed learning, which may be sustained throughout their professional lives.1,2 An important area of training is communication skills.1–5 This is particularly relevant in genitourinary medicine where sexual history taking and HIV counselling require sensitive application of these skills. Previous studies report poor performance in these areas.6,7 This may be because patients are less willing to have students present in such consultations.8,9 The lack of opportunities are further compounded by a reduced allocation of time for speciality training.10,11
In this study we report a method of problem orientated learning which focuses on the application of knowledge and skills to a simulated “real life” clinical scenario framed in a real time context. We use the technique of video role play with structured feedback specifically to enable the students to view themselves in action and to reflect on their knowledge, communication, and interpersonal skills and their management of the “consultation.” Students are called upon to identify strengths and areas of weakness which then forms the basis for further learning.
The use of video feedback has been shown to enhance performance in communication skills in other disciplines.3,4,12ndash;14 We hypothesised that adding video feedback to traditional methods of undergraduate teaching would achieve a better outcome in terms of acquiring and retaining communication skills and knowledge than traditional methods alone. The objective was to test this hypothesis in as tightly controlled a setting as the medical school system would allow. A secondary objective was to estimate the cost implications of this intervention. This study investigates its value in clinical training, using the setting of genitourinary medicine.
Subjects and methods
The subjects were all fourth year medical students at UMDS during the academic year 1996–7. UMDS had a traditional medical curriculum with 2 years of basic science followed by 3 years of clinical medicine. The attachment in genitourinary medicine took place during their second clinical year and lasted for 5 weeks. Training in sexual history taking and HIV counselling were both delivered by seminars, which included an educational video,15 and clinical sessions. Students were attached at the same time to the departments of dermatology, ENT, and ophthalmology. Students were examined in these clinical disciplines by an objective structured clinical examination which also included stations in general practice (another fourth year attachment).
A pilot study was performed before this investigation to develop and refine the methodology.
The actor was given a case scenario beforehand, which was either a patient presenting with a sexually transmitted disease or a patient requesting an HIV test. Students were expected to take a sexual history or counsel for HIV testing, respectively, for these scenarios. The actor gave feedback immediately after the 10 minute interview. The criteria used by the actor was his/her overall assessment of how they were managed throughout the consultation. This complies with the approach used for scoring by actors in the end of term OSCEs at King's College medical schools. The student completed a self evaluation questionnaire on exit from the consultation (fig 1). The videos were viewed with the student the following week. The student gave his or her assessment by completing the structured questionnaire with the tutor and or educationalist. Then there was a structured assessment (box) and feedback by a GU physician and an educational psychologist on both the clinical content and interview technique. The structured assessment had been developed during the early pilot stages to ensure appropriateness and ease of use. The rigorous instructions given to assess performance in the box were developed in order to ensure interassessor concordance and to aid ease of use. This was confirmed within the pilot study, where concordance over a sample of students approached 100%. Although the study was not designed to further check concordance between the two raters, complete agreement was independently reached on the students performance. Students were encouraged to draw on existing knowledge and direct further learning towards solving the clinical problem presented.
Video role play with structured feedback is particularly effective in improving undergraduate communication skills and application of knowledge.
This approach recreates the clinical scenario one wishes the student to experience.
Students consistently report this approach as the most relevant and stimulating aspect of their training.
Consideration needs to be given to redirecting resources towards more effective training methods such as these.
One week later these students and a control group of students (who did not undergo the intervention) were examined by an objective structured clinical examination (OSCE). This is a well recognised method of testing clinical skills in undergraduates and postgraduates.16–18
The genitourinary medicine stations included sexual history taking and HIV counselling. Stations lasted 6 minutes and professional actors were used as simulated patients with standardised presentations. Test objectives were identified for each station and examiners used a checklist to mark students. The GU medicine stations for the OSCE were designed by the investigators of this study. The criteria, test objectives, and marking criteria conformed to those used in our study.
Examiners were given set grade criteria (box) to standardise their assessment. A difference of 5% between the cumulative scores of the assessment used in the box is taken as a significant performance variation, and a difference of 10% is taken as highly significant. These differences were estimated before the study based on the sample sizes. It was calculated that a sample size of 20–30 would probably be required in each group in order to demonstrate a significant difference between the main outcome measures, which were the results of the OSCE used to assess the students after their course had finished.
Results of the objective structured clinical examination were analysed using the Mann-Whitney (Wilcoxon) test. The analysis was performed on an “intention to teach” basis.
Groups of undergraduates attending the genitourinary medicine course were randomly allocated over a 1 year period to the sample group. The method of assignment was to have every third group of 12–15 students allocated to the intervention group where video was used. We felt it was important to assign by groups to avoid crossover during the course between those having or not having the intervention. Every other two groups out of three acted as a control group.
The students were assigned to their groups at the start of the academic year by the central registry in a random fashion, but with scrutiny to ensure that there was a demographic and sex balance within each group. No individual taking part in this study was associated with the assignment process of students to their groups. They were individually filmed in the role of a doctor interviewing a patient (played by an actor) presenting with a GU problem or requesting an HIV test. The examiners were not aware which students were in the intervention group or the control group.
The record of the students allocated to each group was made by the teachers at the end of each teaching session using a standardised signature and name register system which has been in operation for 3 years and has been designed to accurately capture those students who attended. This register was available for identifying the groups the students were in at the time of analysis of results.
There were 40 students in the sample group and 92 students in the control group. The numbers in the sample group were restricted to 40 because of the resources allocated to the study. The participant flow and follow up is given in figure 2. Eight students who were eligible for the study were not randomised because of electives (six), illness (one), and absence for indeterminable cause (one). One student who was randomised, and was within the non-intervention group, did not enter the study because of a family situation that precluded her receiving teaching. There were no significant sex or age differences between the intervention and non-intervention groups. The results and analyses of the OSCE results of both groups are shown in table 1. The size of both groups were much larger than required to demonstrate significant differences. (Minimum sample size of 20–30 was comfortably exceeded for both groups.) There was a normal distribution of scores in both groups.
An interesting observation made by both the clinical tutor and educationalist, was the ease of use of the assessment form (box).
Interacts with patient in a professional sympathetic manner
Elicits an adequate clinical history
Picks up and develops underlying problems or anxieties as appropriate
Students shake hands with the patient, giving their name and offering them a seat (as appropriate). In inquiring about the presenting complaint they should ascertain the nature of the vaginal discharge (colour, odour), the duration, and whether it has occurred before. Associated symptoms, such as vulvovaginal soreness or ulceration should be noted, and also whether there are any urinary symptoms (dysuria, haematuria, or frequency) or abdominal pain. The sexual history should include the dates of the last two sexual contacts (in past 3 months), with details of whether the partners were male or female, regular or casual (the duration of the relationship), and whether contact occurred in the UK or abroad. Use of condoms should also be inquired about. The date of the last period is important and method of contraception. Inquiry is made about menstrual problems such as heavy, painful, or irregular periods. The date of the last cervical smear is sought. Details of all previous pregnancies are also extremely important. Questions relating to general health should include major illnesses, such as diabetes or blood pressure, and any operations. More specifically any previous GU complaints should be sought. Current medication and any drug allergies (particularly penicillin allergy) are very important.
A polite, professional introduction is always required to score an “adequate “ response. The nature of the discharge and its duration should be noted and details of the most recent sexual contact must be sought, including whether the partner was regular or casual, and if barrier methods were used. The date of the last menstrual period and method of contraception are important. A brief reference to other illnesses and any current medication or drug allergies should be noted.
The introduction is poor or omitted entirely. Insufficient details of the presenting complaint are taken and the sexual history does not elicit partner details or whether condoms were used. Little or no mention is made regarding last menstrual period, general health, or current medications and drug allergies.
The opening question is clear and non-leading allowing the patient to speak generally—for example, “What can I do for you?” Then more specific questions may be asked, following verbal clues, leading the patient in an appropriate way to explaining the nature of the problem. Non-verbal clues such as inappropriate anger or undue distress are sensed and handled sympathetically, allowing the patient to open up—for example, “Is anything else worrying you?” Throughout the interview, a professional, sympathetic manner is maintained and culminates in a simple summing up of the problem and how to treat and deal with it. Appropriate counselling with the health adviser may be offered and information and leaflets may be given.
The student has a professional and sympathetic manner eliciting the main problems, although may not fully develop non-verbal clues. However, the patient is not left feeling frustrated and unhappy and is given time to answer questioning. A simple explanation of further management is given.
No rapport is established between student and patient. The history is led by direct questions allowing insufficient time for the patient to answer or open up, and non-verbal clues are not developed. The consultation ends with the patient unhappy and unclear as to what is happening.
The OSCE results of the sample groups were higher than those of the control group with the Wilcoxon T statistic significant at p <0.001. When the students gave their structured evaluation of our course the sample group highlighted their perceived value of video role play with structured feedback. In particular, they consistently mentioned the relevance of this form of learning to clinical practice, and the stimulation and enjoyment they obtained from it.
This study demonstrates that video role play with structured feedback is an effective method of training medical students to apply knowledge and skills in similar clinical settings. It is a relatively new method of learning which has several advantages over traditional teaching methods. It overcomes limited time in real life clinical situations. This approach also overcomes patient reluctance to be interviewed by medical students. Our study demonstrates an improvement in the students' acquisition of skills and attitudes. Many students commented in their formal course evaluation that this had been the most valuable and relevant aspect of the course.
This broad approach to problem oriented learning presents knowledge in context and directs learning towards solving that clinical problem. It enhances retention of knowledge because it is relevant and develops skills in learning sustainable throughout professional life.19,20 By simulating GU clinical scenarios with actors we have been able to develop the application as well as the acquisition of student knowledge and communication skills within this framework. The individualised structured feedback session allowed for student participation and discussion of the case encountered and for the teacher to give constructive feedback. This training method and study is unique in the field of GU medicine, but there is work from the specialty to support the value of these training methods.21 Other studies from GU medicine have demonstrated the value of role play techniques among students, but have only employed peer feedback, and have not used professional actors or evaluated the effectiveness of this technique with this degree of rigour.22
One of the potential difficulties of a study such as this is the degree to which intervention and control groups will interact, and hence introduce some aspects of the intervention to the control group. This is difficult to prevent within the medical student milieu. However, we attempted to avoid this interaction by randomising the students by groups. Furthermore, this effect would tend to reduce the difference between the two groups rather than exaggerate it. An advantage of this study over many other studies with human volunteers is the very high follow up rate achieved. All students in both teaching groups attended for the examination. This is because students perceive examinations as part of their required progress through medical school. The study was not designed to look at the wider distribution scores of the different groups, but we feel that this would provide valuable further insight into the differences between the two groups, and should be included as an aim of any further studies in this area.
One of our most interesting incidental findings was the relative cost effectiveness of this intervention. We present below a simple cost analysis of the teaching intervention. If this approach were to be introduced in a more widespread fashion for the teaching of medical students, then a more formal economic analysis could be undertaken. The cost of hiring actors to accommodate the application of this method to a yearly intake of 200 students is £1500 (£7.50 per student). To this must be added the cost of 30 days of time each for two trainers for supervision and feedback. The other costs involved include training staff for this form of teaching, and costs which vary between departments and institutions would include the use of the use of rooms and use of video equipment. Offsetting this is a reduction in clinic and ward training time, of approximately 147 hours (22.5 days). In routine practice we only have one assessor and the cost of role play could be reduced by using “in-house” staff.
In view of the relevance, reliability, and effectiveness, including cost effectiveness, of this training method, we had introduced it into our regular undergraduate programme for GU medicine.
All the authors contributed to the trial design and conduct of the study.
websiteextraA slide show presentation which can be downloaded is on the STI websitewww.sextransinf.com
Conflicts of interest: none.