Introduction

The United Kingdom is in the middle of a major change in undergraduate medical education which will be compounded over the coming years by a large increase in the number of medical students. University medical schools have planned new curricula, underpinned by core knowledge requirements with individual teachers or departments left to decide on their teaching methods. These methods depend on the availability of resources (time, equipment, money, and the support of other professionals) and familiarity with different teaching techniques. It is rare that the teacher has evidence of the comparative effectiveness of different training techniques to inform this selection. Indeed, in most clinical settings the different approaches have not been evaluated. There is a general shift towards delivering these curricula by using approaches such as problem oriented learning. Problem oriented learning using real life clinical situations as a framework for teaching, learning, and assessment is strongly encouraged by the General Medical Council in the training of undergraduates.¹ Both the General Medical Council and the King's Fund highlight the need to reduce the factual load in traditional medical curricula, emphasising instead student skills in self directed learning, which may be sustained throughout their professional lives.^1,2 An important area of training is communication skills.^1–5 This is particularly relevant in genitourinary medicine where sexual history taking and HIV counselling require sensitive application of these skills. Previous studies report poor performance in these areas.^6,7 This may be because patients are less willing to have students present in such consultations.^8,9 The lack of opportunities are further compounded by a reduced allocation of time for speciality training.^10,11

In this study we report a method of problem orientated learning which focuses on the application of knowledge and skills to a simulated “real life” clinical scenario framed in a real time context. We use the technique of video role play with structured feedback specifically to enable the students to view themselves in action and to reflect on their knowledge, communication, and interpersonal skills and their management of the “consultation.” Students are called upon to identify strengths and areas of weakness which then forms the basis for further learning.

The use of video feedback has been shown to enhance performance in communication skills in other disciplines.^{3,4,12ndash;14} We hypothesised that adding video feedback to traditional methods of undergraduate teaching would achieve a better outcome in terms of acquiring and retaining communication skills and knowledge than traditional methods alone. The objective was to test this hypothesis in as tightly controlled a setting as the medical school system would allow. A secondary objective was to estimate the cost implications of this intervention. This study investigates its value in clinical training, using the setting of genitourinary medicine.

Subjects and methods

PROTOCOL

The subjects were all fourth year medical students at UMDS during the academic year 1996–7. UMDS had a traditional medical curriculum with 2 years of basic science followed by 3 years of clinical medicine. The attachment in genitourinary medicine took place during their second clinical year and lasted for 5 weeks. Training in sexual history taking and HIV counselling were both delivered by seminars, which included an educational video,¹⁵ and clinical sessions. Students were attached at the same time to the departments of dermatology, ENT, and ophthalmology. Students were examined in these clinical disciplines by an objective structured clinical examination which also included stations in general practice (another fourth year attachment).

A pilot study was performed before this investigation to develop and refine the methodology.

The actor was given a case scenario beforehand, which was either a patient presenting with a sexually transmitted disease or a patient requesting an HIV test. Students were expected to take a sexual history or counsel for HIV testing, respectively, for these scenarios. The actor gave feedback immediately after the 10 minute interview. The criteria used by the actor was his/her overall assessment of how they were managed throughout the consultation. This complies with the approach used for scoring by actors in the end of term OSCEs at King's College medical schools. The student completed a self evaluation questionnaire on exit from the consultation (fig 1). The videos were viewed with the student the following week. The student gave his or her assessment by completing the structured questionnaire with the tutor and or educationalist. Then there was a structured assessment (box) and feedback by a GU physician and an educational psychologist on both the clinical content and interview technique. The structured assessment had been developed during the early pilot stages to ensure appropriateness and ease of use. The rigorous instructions given to assess performance in the box were developed in order to ensure interassessor concordance and to aid ease of use. This was confirmed within the pilot study, where concordance over a sample of students approached 100%. Although the study was not designed to further check concordance between the two raters, complete agreement was independently reached on the students performance. Students were encouraged to draw on existing knowledge and direct further learning towards solving the clinical problem presented.

One week later these students and a control group of students (who did not undergo the intervention) were examined by an objective structured clinical examination (OSCE). This is a well recognised method of testing clinical skills in undergraduates and postgraduates.^16–18

The genitourinary medicine stations included sexual history taking and HIV counselling. Stations lasted 6 minutes and professional actors were used as simulated patients with standardised presentations. Test objectives were identified for each station and examiners used a checklist to mark students. The GU medicine stations for the OSCE were designed by the investigators of this study. The criteria, test objectives, and marking criteria conformed to those used in our study.

Examiners were given set grade criteria (box) to standardise their assessment. A difference of 5% between the cumulative scores of the assessment used in the box is taken as a significant performance variation, and a difference of 10% is taken as highly significant. These differences were estimated before the study based on the sample sizes. It was calculated that a sample size of 20–30 would probably be required in each group in order to demonstrate a significant difference between the main outcome measures, which were the results of the OSCE used to assess the students after their course had finished.

Results of the objective structured clinical examination were analysed using the Mann-Whitney (Wilcoxon) test. The analysis was performed on an “intention to teach” basis.

ASSIGNMENT

Groups of undergraduates attending the genitourinary medicine course were randomly allocated over a 1 year period to the sample group. The method of assignment was to have every third group of 12–15 students allocated to the intervention group where video was used. We felt it was important to assign by groups to avoid crossover during the course between those having or not having the intervention. Every other two groups out of three acted as a control group.

MASKING

The students were assigned to their groups at the start of the academic year by the central registry in a random fashion, but with scrutiny to ensure that there was a demographic and sex balance within each group. No individual taking part in this study was associated with the assignment process of students to their groups. They were individually filmed in the role of a doctor interviewing a patient (played by an actor) presenting with a GU problem or requesting an HIV test. The examiners were not aware which students were in the intervention group or the control group.

The record of the students allocated to each group was made by the teachers at the end of each teaching session using a standardised signature and name register system which has been in operation for 3 years and has been designed to accurately capture those students who attended. This register was available for identifying the groups the students were in at the time of analysis of results.

Results

There were 40 students in the sample group and 92 students in the control group. The numbers in the sample group were restricted to 40 because of the resources allocated to the study. The participant flow and follow up is given in figure 2. Eight students who were eligible for the study were not randomised because of electives (six), illness (one), and absence for indeterminable cause (one). One student who was randomised, and was within the non-intervention group, did not enter the study because of a family situation that precluded her receiving teaching. There were no significant sex or age differences between the intervention and non-intervention groups. The results and analyses of the OSCE results of both groups are shown in table 1. The size of both groups were much larger than required to demonstrate significant differences. (Minimum sample size of 20–30 was comfortably exceeded for both groups.) There was a normal distribution of scores in both groups.

An interesting observation made by both the clinical tutor and educationalist, was the ease of use of the assessment form (box).

The OSCE results of the sample groups were higher than those of the control group with the Wilcoxon T statistic significant at p <0.001. When the students gave their structured evaluation of our course the sample group highlighted their perceived value of video role play with structured feedback. In particular, they consistently mentioned the relevance of this form of learning to clinical practice, and the stimulation and enjoyment they obtained from it.

Discussion

This study demonstrates that video role play with structured feedback is an effective method of training medical students to apply knowledge and skills in similar clinical settings. It is a relatively new method of learning which has several advantages over traditional teaching methods. It overcomes limited time in real life clinical situations. This approach also overcomes patient reluctance to be interviewed by medical students. Our study demonstrates an improvement in the students' acquisition of skills and attitudes. Many students commented in their formal course evaluation that this had been the most valuable and relevant aspect of the course.

This broad approach to problem oriented learning presents knowledge in context and directs learning towards solving that clinical problem. It enhances retention of knowledge because it is relevant and develops skills in learning sustainable throughout professional life.^19,20 By simulating GU clinical scenarios with actors we have been able to develop the application as well as the acquisition of student knowledge and communication skills within this framework. The individualised structured feedback session allowed for student participation and discussion of the case encountered and for the teacher to give constructive feedback. This training method and study is unique in the field of GU medicine, but there is work from the specialty to support the value of these training methods.²¹ Other studies from GU medicine have demonstrated the value of role play techniques among students, but have only employed peer feedback, and have not used professional actors or evaluated the effectiveness of this technique with this degree of rigour.²²

One of the potential difficulties of a study such as this is the degree to which intervention and control groups will interact, and hence introduce some aspects of the intervention to the control group. This is difficult to prevent within the medical student milieu. However, we attempted to avoid this interaction by randomising the students by groups. Furthermore, this effect would tend to reduce the difference between the two groups rather than exaggerate it. An advantage of this study over many other studies with human volunteers is the very high follow up rate achieved. All students in both teaching groups attended for the examination. This is because students perceive examinations as part of their required progress through medical school. The study was not designed to look at the wider distribution scores of the different groups, but we feel that this would provide valuable further insight into the differences between the two groups, and should be included as an aim of any further studies in this area.

One of our most interesting incidental findings was the relative cost effectiveness of this intervention. We present below a simple cost analysis of the teaching intervention. If this approach were to be introduced in a more widespread fashion for the teaching of medical students, then a more formal economic analysis could be undertaken. The cost of hiring actors to accommodate the application of this method to a yearly intake of 200 students is £1500 (£7.50 per student). To this must be added the cost of 30 days of time each for two trainers for supervision and feedback. The other costs involved include training staff for this form of teaching, and costs which vary between departments and institutions would include the use of the use of rooms and use of video equipment. Offsetting this is a reduction in clinic and ward training time, of approximately 147 hours (22.5 days). In routine practice we only have one assessor and the cost of role play could be reduced by using “in-house” staff.

In view of the relevance, reliability, and effectiveness, including cost effectiveness, of this training method, we had introduced it into our regular undergraduate programme for GU medicine.