Statistics from Altmetric.com
In 1815, when barbers were surgeons and quacks peddled their ineffective wares to the trusting sick, the Worshipful Society of Apothecaries became the first official body empowered to license medical practitioners. The society still runs a qualifying examination, through the United Examinations Board, and also offers a number of postgraduate diploma examinations such as the ever popular Diploma in the History of Medicine and the recently introduced Diploma in the Medical Care of Catastrophes. The Diploma in Genitourinary Medicine came about in the early 1970s when several senior venereologists lobbied for a postgraduate qualification to be made available to the specialty and, rebuffed by the Royal College of Physicians, approached the Apothecaries' Society. The first Diplomas in Venereology were awarded in 1974 under the stewardship of Drs Catterall, Dunlop, Fluker, King, and Nicol.
For many years the diploma examination included a clinical section with “long” and “short” cases, a viva voce, an essay paper, and multiple choice questions. This format of examination will be familiar to generations of doctors as the arbitrary nature, and sometimes questionable content, of clinical and viva examinations went unquestioned by the medical profession for many years. As a Liverpool undergraduate in the 1970s, when an appreciable number of syphilitic survivors from a once thriving dockland era were still alive, I remember being drilled on the clinical signs of tabes dorsalis, not because such patients presented frequently for care but because of the likelihood of such a case appearing in medical finals. Later, as a medical registrar tasked with finding suitable cases for final year examinations, I came to appreciate the value of patients with stable health and reliable clinical signs. Later still, after working for some years in genitourinary medicine without seeing a single case of tabes dorsalis present de novo, I was bemused to find myself examining candidates for the Diploma in Genitourinary Medicine on just such a (rather elderly) patient. Plus ça change.
The assessment of medical undergraduates and postgraduates has come under closer scrutiny in recent years, as have the tools available for the task. An assessment should involve comparing a candidate against agreed objective criteria, using methods that are both valid (meaning appropriate and relevant to clinical practice) and reliable (give consistent results regardless of variables such as different examiners).1 The test methods used should be able to cover large parts of the syllabus, assess higher cognitive functions such as clinical problem solving rather than simple factual recall and, particularly at postgraduate level, assess what a candidate actually does rather than simply knows.
Against these criteria the traditional clinical and viva examinations inevitably attract criticism; both methods test knowledge and, to an extent, the ability to detect or interpret clinical findings but both are of low validity as their content, as in the example given above, may not be relevant to everyday clinical practice. Furthermore, since candidates cannot all be examined on the same clinical problems by the same examiner asking the same questions, it is inevitable that some will experience a more difficult examination than others and so their respective performances cannot be compared fairly. Reliability is further reduced by differences between candidates that are unrelated to their clinical competence; for example, it is likely that personable, self confident candidates will score more highly than less confident candidates of the same ability because of the personal bias of examiners.2
Multiple choice questions (MCQs) also have their problems. Although their results are consistent (any given candidate is likely to perform as well at each attempt) MCQs are usually designed to test factual knowledge rather than clinical problem solving ability.3 They are also time consuming to set as it is difficult to design good questions that not only lend themselves to yes/no responses but also avoid “cueing” the correct answer. However, MCQs do have the twin advantages of being able to cover as much of the syllabus as is required and being quick and easy to mark. Essay questions which require lengthy written responses often test only factual recall4 in a narrow range of subjects while their marking is time consuming and can show significant differences between examiners (and even the same examiner re-marking the same paper), thus reducing reliability.5 However, it is possible for essays to test problem solving and the ability to synthesise, or criticise, an argument and the use of a structured marking scheme can reduce interexaminer discrepancy.
In an attempt to improve the reliability and validity of examinations a number of other methods of assessment have been introduced, with variable success, into a number of medical schools and postgraduate colleges in the United Kingdom and elsewhere. These include such techniques as modified essay questions,6 extended matching items,7 patient management problems,8 and objective structured clinical examinations.9 None of these assessment methodologies has proved ideal but examiners now have more tools at their disposal and can, if they first define what it is they wish to assess, select the most appropriate method(s) in order to allow valid and reliable judgments to be made. Often a combination of assessment methods will be preferred and some effort will be required to minimise each of their inherent flaws.
If the Diploma in Genitourinary Medicine is to identify those candidates who have particular expertise in the specialty, what attributes does the examination need to test? A degree of knowledge certainly, not only at the level of factual recall but also an ability to apply that knowledge appropriately in solving clinical problems. The ability to take a sexual history, carry out a competent genital examination, and formulate a management strategy would be minimum requirements. Good communication skills are essential, a high degree of manual dexterity less so. With the relative attributes of a competent clinician in genitourinary medicine in mind, the Diploma in Genitourinary Medicine began a process of change 5 years ago when it was decided to abandon the clinical examination and viva voce for the reasons outlined above. In their place is an objective structured clinical examination (OSCE) which involves all candidates rotating through identical stations, at each of which is a specific task designed to assess a particular clinical skill. The morning section consists of 12 stations, each lasting five minutes and involving such tasks as data interpretation, assessment of x rays or commonly used scans, passing a speculum or doing a pelvic examination on a plastic pelvis, identifying faults in a briefly sketched out clinical or research scenario, explaining the correct use of a condom, or using a microscope to report on a Gram stained slide. Predetermined marking schedules mean that all candidates are examined to the same standard. The afternoon section consists of six stations, each lasting for 10 minutes which allows sufficient time to assess the candidate's ability to perform relevant clinical examinations, take a sexual history and, most importantly, communicate effectively with patients. The latter two categories of stations use standardised simulated patients (trained actors or, if appropriate, health advisers) and candidates may find themselves asked to counsel a “patient” contemplating an HIV test, discuss the concerns of a pregnant woman experiencing recurrent episodes of genital herpes, advise a healthcare worker who has just suffered a sharps injury, or discuss partner notification with a patient who has just been told they have a chlamydial infection. These interactions are marked by an observing examiner using a predetermined marking schedule which allows the “patient” to contribute a portion of the marks. Again, all candidates rotate through the same stations and so are examined on the same scenarios.
The essay paper, still a part of the examination but under active review, provides an opportunity to assess candidates' ability to put knowledge and basic principles to creative use. However, the marking of this section of the examination has been radically revised. Each of the four (compulsory) questions now has a model answer prepared which contains a predetermined marking schedule thereby ensuring that each candidate's response is judged along similar lines and that credit is given for quality rather than quantity. Each question is marked by a different pair of examiners who then average their marks with reference, in cases of significant disparity in scores, to the original paper. Thus, each candidate's paper is marked, anonymously, by eight examiners and interexaminer variability is reduced as much as possible. The MCQ examination (in reality consisting of clustered true/false questions) no longer relies on slide projected clinical material and has received considerable attention to ensure a consistent standard.
The diploma provides neither a qualifying nor exit certificate but does provide a postgraduate qualification for those seeking to demonstrate a degree of knowledge and competence in their chosen field. During the past 5 years a total of 211 candidates have chosen to have their abilities tested by appearing in the examination, 150 (71%) of whom have passed. The striking change in candidate demography in recent years reflects the different needs which the diploma fulfils. As recently as 5 years ago it was unusual for junior doctors training in genitourinary medicine to sit the examination but, since the introduction of a defined curriculum for higher specialist training, special grade registrars are increasingly keen, indeed encouraged, to demonstrate their progress against a recognised benchmark and now account for about half of all candidates. The diploma is also attractive to doctors in non-consultant career grades who wish to have their specialist skills acknowledged; advertisements for NCCG posts have occasionally specified “Dip GUM preferred” suggesting that trusts may be increasingly aware of the importance of being able to demonstrate that their staff have achieved a recognised standard. Fewer overseas candidates now apply to sit the examination, possibly reflecting a widening difference in the practice of genitourinary medicine between developed and developing countries or possibly reflecting the availability of postgraduate qualifications in other countries.
Any examination candidate is likely to show an interest in the form their assessment will take and evidence shows that the method and content of an examination can significantly influence the pattern of students' learning.10 In revising the format of this particular examination the panel of examiners has sought to ensure that the diploma provides a relevant assessment of the clinical skills important in the practice of genitourinary medicine.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.