Should aciclovir prophylaxis be used in late pregnancy in women with recurrent genital herpes infection? How to use a clinical decision analysis

Introduction The purpose of this article is to explore the benefits of using decision analysis in clinical decision making when the published evidence about the effectiveness of an intervention is uncertain. The use of decision analysis will be explored using the example of aciclovir prophylaxis in late pregnancy for women with recurrent genital herpes infection.' The article draws on the guidelines published by Richardson and others which set out a framework for evaluating the usefulness of a clinical decision analysis model (table 1).2 3


Introduction
The purpose of this article is to explore the benefits of using decision analysis in clinical decision making when the published evidence about the effectiveness of an intervention is uncertain.The use of decision analysis will be explored using the example of aciclovir pro- phylaxis in late pregnancy for women with recurrent genital herpes infection.'The article draws on the guidelines published by Richardson and others which set out a frame- work for evaluating the usefulness of a clinical decision analysis model (table 1).2 3 National Perinatal Epidemiology Unit, Radcliffe Infirmary, Oxford OX2 6HE P Brocklehurst T

Roberts
Accepted for publication 7 May 1997 THE CLINICAL PROBLEM Women with genital herpes infection in preg- nancy are at risk of transmitting herpes to their baby at the time of delivery resulting in neona- tal herpes infection.Neonatal herpes is a severe illness with a high mortality and mor- bidity even with prompt antiviral treatment. 4urrent management of women with recurrent genital herpes infection during pregnancy in the UK rests almost entirely on whether the woman experiences a symptomatic recurrence at the time of labour.If there is evidence of an active recurrence then delivery by caesarean section is recommended to avoid the risk of mother to child transmission.5 If there is no evidence of an active recurrence then a vaginal delivery is anticipated.The risk of mother to child transmission is unknown but estimates suggest that it can be no higher than 8%6 although many authorities believe it is significantly lower than this and some have suggested that it is negligible.7As a consequence of the current management policy many women with recurrent genital herpes infection present at the time of labour undergo caesarean section when the risk to the neonate appears to be small.An alternative manage- ment strategy which has been suggested is the use of suppressive aciclovir starting in late pregnancy, usually around 36 weeks, to pre- vent symptomatic recurrences of herpes and therefore allow women to anticipate a normal vaginal delivery.8If breakthrough recurrences do occur at the time of labour there are two management options.Either deliver by caesarean section or allow vaginal delivery and take viral cultures from the neonate and then monitor for signs of herpes infection and treat aggressively if either the cultures are positive or signs occur.

THE EVIDENCE FOR ACICLOVIR SUPPRESSION
A Medline search for randomised trials which evaluate the effectiveness of aciclovir suppres- sion in late pregnancy will reveal only one.9 This was conducted in women experiencing their first episode of genital herpes infection during the index pregnancy.The trial demonstrated a decrease in the incidence of cae- sarean section in the women receiving aciclovir.The generalisability of these results to women with pre-existing recurrent herpes infection is not straightforward.It is possible that subsequent recurrences during the preg- nancy in these women are more frequent and/or more severe than those experienced by women with pre-existing herpes infection.In addition, it appears that these women are at a substantially increased risk of mother to child transmission and other adverse neonatal out- comes than women with pre-existing infection.'0Extrapolating the results of this trial to women with pre-existing recurrent infection, therefore, seems unwise.There has, however, been a recent economic evaluation by Randolph of the use of suppressive aciclovir in this group of women which utilises a clinical decision model.'We will demonstrate how to critically appraise this paper using the guide- lines set out by Richardson and Detsky.3

DECISION ANALYSIS
The process of decision analysis makes explicit the decisions involved in clinical practice.The decision tree in the paper by Randolph et al (fig) expresses the outcomes which can occur for both mother and baby whether the woman is given aciclovir or is not given aciclovir.In either situation there may be symptomatic recurrences at the time of delivery which may be managed by caesarean section or vaginal delivery and culture, there may be asympto- matic shedding which is not detected at the time of delivery, or there may be no evidence of herpes virus at the time of delivery.Probabilities can be attached to each treatment path of the decision tree and if there is little certainty about these probabilities then ranges of probabilities can be incorporated into the model.
In order to evaluate the usefulness of any published decision analysis it is necessary to address (a) whether the results of the study are valid and (b) what the results actually are and whether they will help in caring for your patient.A framework for assessing a decision analysis is included in table 1.This is similar to the framework developed in previous papers in this series."How valid are the results of the study?The validity of the results is largely determined by the methods used.An invalid study design Aciclovir prophylaxis in late pregnancy to prevent neonatal herpes: the decision tree.

Lesions at delivery
Caesarean for HSV + baseline caesarean

Aciclovir
Caesarean for HSV + baseline caesarean No aciclovir means that the results are meaningless and applicable to no one.Therefore, this needs to be the first step in deciding whether this paper will provide useful information for you and your patient.

WERE ALL IMPORTANT STRATEGIES AND OUTCOMES INCLUDED?
This is determined by whether the model which is described accurately mirrors your clinical decision making process.In clinical decision analysis the alternative treatment paths for a clinician are represented by a deci- sion tree.This contains chance nodes (circles) where the probability of pursuing one pathway or another is determined by chance and deci- sion nodes (squares) where the probability of pursuing one pathway or another is deter- mined by the clinical decision made, or in this case by the effectiveness of the intervention.All the relevant pathways which lead to the potential outcomes should be included in the tree.The judgment about whether a decision tree includes all the valid alternatives depends on your interpretation of the various outcomes from both your reading of the literature and your clinical experience.If there are major omissions in important strategies or outcomes then the decision tree is obviously invalid and the results, no matter what they are, will not be applicable to your patients.

WERE ALL OF THE REALISTIC CLINICAL STRATEGIES COMPARED?
Until recently the management of women with recurrent genital herpes rested on whether to use caesarean section for women with recur- rent lesions at term or not.Since the advent of aciclovir and its more widespread use in preg- nancy a decision analysis which did not include aciclovir may be considered to be an incomplete comparison of all the potential clinical strategies.This decision tree describes four possible strategies (table 2).A woman with recurrent herpes infection either receives aciclovir during late pregnancy or she does not.Those who experience a recurrence at the time of labour then either undergo vaginal delivery and the neonate is monitored or they undergo caesarean section.The four strategies Table 1 User's guide for clinical decision analysis Are the results valid?Were all the important strategies and outcomes included?Was an explicit and sensible process used to identify, select, and combine the evidence into probabilities?Were the utilities explained in an explicit and sensible way from credible sources?Was the potential impact of any uncertainty in the evidence determined?
What are the results?In the baseline analysis, does one strategy result in a clinically important gain for patients?If not, is the result a toss up?How strong is the evidence used in the analysis?Could the uncertainty in the evidence change the result?Will the results help me in caring for my patients?Do the probability estimates fit my patients clinical features?Do the utilities reflect how my patients would value the outcomes of the decision?are therefore: (a) no aciclovir and no caesarean section, (b) no aciclovir and caesarean section if there is a recurrence at the time of labour, (c) aciclovir and no caesarean section, and, finally (d) aciclovir and caesarean section if there is a recurrence at the time of labour.An accurate description of the intervention should be included.The effectiveness of aciclovir may vary depending on the dose used and the duration of its use.If aciclovir is to be started at 36 weeks' gestation some women will have delivered before that time and some of these will have had active recurrences at the time of delivery; hence, the model is not entirely complete.The extent to which this alters the management of the majority of women, however, is likely to be small.

WERE ALL CLINICALLY RELEVANT OUTCOMES CONSIDERED?
For the mother various outcomes were included in the model: caesarean section for recurrent herpes infection; caesarean section for other indications regardless of the herpes infection; or vaginal delivery with or without culture.For the baby the health outcomes specified in the decision tree were either herpes infection or no herpes infection.For herpes infected babies there were four eventual outcomes: the baby was normal; the baby had moderate neurological disability; the baby had severe neurological disability; or the baby died.The neonatal states would appear to represent a complete range of health outcomes.The out- comes for the mother, however, do not include the consequences of vaginal or caesarean sec- tion delivery-for instance, post-caesarean sec- tion infectious morbidity or the increased risk of maternal mortality associated with operative delivery when compared with vaginal delivery.This appears to be because the specific aim of the paper was to address the cost of a case of neonatal herpes averted and, as a consequence, only a partial evaluation was undertaken.This does not invalidate the study, but readers need to be aware that not all the potentially relevant health outcomes are included.
In addition, no mention is made of the potential side effects of aciclovir for the mother and the baby.Any side effects could be included in the decision tree so that an explicit weighing up between the risks and benefits can be seen.The failure to mention side effects is probably because of the safety of aciclovir in this setting with no major side effects having been reported for mother or baby.12Nevertheless, mention of this information would have been useful.The chance and decision nodes of the decision tree need to be accompanied by probabilities that either pathway will be followed depending on the preceding event.Probabilities will need to be assembled from a broad range of information in the medical literature.In order to determine the probabilities it is necessary to review critically a large volume of information using the processes already described in this series so that only those studies where the methods are valid are incorporated.The litera- ture review should be systematic and clearly explained if bias is to be avoided.'3The paper by Randolph et al does not specify the search strategy or search terms used to search Medline, nor does it specify the process of selection of the studies.The paper clearly states that "the baseline probability values were the most plausible estimates".This statement could be interpreted as suggesting that the authors only included those articles which reported estimates which agreed with their existing views.A clearer description of the selection process would help to satisfy those worried about this statement and how this may contribute to selection bias.If the analysis is being taken beyond a partial analysis, as in the paper by Randolph et al, to a complete cost utility analysis then there may need to be inter- views with patient groups Once the authors have decided what information to include they must synthesise it into a quantitative estimate of each probability.Depending on the source of this information these probabilities may come with some uncer- tainty, in which case a range of probabilities may be incorporated into the model (see below).It is often useful in these publications to indicate the strength of the evidence which has been used to provide these probability esti- mates.For aciclovir prophylaxis in late pregnancy the authors have devised a scale from A to D which assesses the quality of the evidence resulting in their baseline probability values.Grade A was from data with a high level of confidence-for example, randomised con- trolled trials or multicentre cohort studies with consistent findings, through to grade D which was a best guess with no data available.The authors assembled an impressive list of probabilities with ranges.There are two probability estimates, however, which are particularly cru- cial to the decision tree and both these are the decision nodes.The first is the effectiveness of aciclovir at preventing lesions at the time of labour and hence preventing delivery by cae- sarean section.The second is the effectiveness of caesarean section at preventing mother to child transmission.These probabilities are essential in this decision tree.As has already been discussed, however, there are no ran- domised controlled trials of the effectiveness of aciclovir in this population of women and there are no large multicentre cohort studies addressing this question.Therefore, the probabilities used for the effectiveness of aciclovir could be considered to be weaker than is sug- gested by the paper.Utilities represent a quantitative measure of the value of the various outcomes to the patient or decision makers (whether this be individual clinicians or policy makers).Various methods of measuring utilities are available and if utili- ties measures are included in a decision analysis one of the accepted methods should be described.'4This paper did not present data on the utilities of the various health outcomes.As the authors state, this information can be extremely difficult to obtain and difficult to interpret.Whether utilities have or have not been included does not affect the validity of a study if the study objectives can be achieved without utility measurement.Readers, how- ever, need to be aware that this aspect of the clinical scenario may have been omitted.

WAS THE POTENTIAL IMPACT OF ANY UNCERTAINTY IN THE EVIDENCE DETERMINED?
Uncertainty in probability estimates can be expressed by a range of estimates.A sensitivity analysis can then be carried out by substituting the highest and lowest values for the probabilities which were included in the original model.Which probability estimates should be varied will be a matter of judgment and this will depend to an extent on the level of certainty of the estimate.The sensitivity analyses in the paper by Randolph et al varied three factors.The first of these was the effectiveness of aciclovir which ranged from 45% to 95%.As has already been discussed, however, the estimate of effectiveness of aciclovir has not been determined in this population of women.The range of probabilities given assumes that aciclovir is effective (the lowest estimate used is 45%).It may have been more appropriate to vary these limits between the highest estimate and 0%.The remaining factors in the sensitivity analy- ses include altering the mother to child trans- mission risk at vaginal delivery to 4% (from the original model which included an estimate of 1 %) and altering the effectiveness of caesarean section at preventing mother to child transmis- sion from 80% to 0%.
What are the results?The four strategies investigated were com- pared to explore the effect each had on the number of cases of neonatal herpes infection which could be averted per 10 000 women.The strategy which was most effective in this decision analysis was that of aciclovir with cae- sarean section if breakthrough recurrences occurred at the time of labour (table 2).This prevented 5.5 cases of neonatal herpes for every 10 000 women treated with aciclovir, with 216 of these women undergoing delivery by caesarean section.
This would suggest, therefore, that using aciclovir will prevent more cases of neonatal herpes than relying on caesarean section alone and it will decrease the number of caesarean sections by a factor of five (table 2).These clinical results were then translated into financial costs which supported these findings and suggested that the use of aciclovir could dra- matically reduce the cost per case of neonatal herpes averted by a factor of three when com- pared with caesarean section.HOW STRONG IS THE EVIDENCE USED IN THE ANALYSIS?
As has already been discussed above the strength of a clinical decision analysis depends on the strength of the information used within it.Information from high methodological quality studies is likely to be more reliable than that from poor quality studies.The paper by Randolph et al has addressed this issue by grading all the evidence included although once again the evidence to support the clinical effectiveness of aciclovir is said to come from grade A information and as we have already seen this is not the case.The strength of the remaining evidence is also relatively weak.The effectiveness of caesarean section in reducing transmission is grade D-that is, there are no data to support this estimate.Likewise the risk of mother to child transmission and the sensitivity and specificity of herpes culture are grade C. As a consequence, a more cautious interpretation of the results would seem appropriate.
COULD THE UNCERTAINTY IN THE EVIDENCE CHANGE THE RESULT?Sensitivity analyses will demonstrate whether including the extremes of range in the model will produce a different result and therefore a different interpretation of the relative advan- tages or disadvantages of the various strate- gies.The sensitivity analyses can be very sophisticated and involve two dimensional graphs of the variables with various threshold values for probabilities above which one par- ticular strategy is preferred over another and vice versa.
The more precise the estimates of probabili- ties the more likely a sensitivity analysis is to produce a similar result to the original model.The less robust the evidence, however, the more likely a sensitivity analysis is to produce a model with a differing result and interpreta- tion.The paper by Randolph et al includes the effectiveness of aciclovir, the mother to child transmission rate, and the effectiveness of cae- sarean section in separate sensitivity analyses (table 3).Some of these analyses change the result but not the relation between each of the strategies.For example, decreasing the mother to child transmission risk increased the num- ber of women undergoing caesarean section and needing treatment with aciclovir to pre- vent a case of neonatal herpes infection.However, the various strategies all maintain the same position in terms of their relative effectiveness to each other.Changing the effectiveness of caesarean section in preventing mother to child transmission from 80% to 0%, however, would greatly increase the relative effectiveness of aciclovir as the strategy of caesarean section for women with herpes would prevent no cases but would be at an increased cost to the women and the health services.
Varying the effectiveness of aciclovir to 95% makes the intervention even more effective at a lower cost.If aciclovir is 45% effective, how- ever, the use of aciclovir with culture becomes equally effective with no aciclovir and cae- sarean section (although at a lower cost).As has already been discussed lowering the effec- tiveness of aciclovir to 0% (compatible with an ineffective intervention) may be a more realistic view of our current knowledge and would pro- duce effects the same as for the strategies not including aciclovir but at an increased cost (the extra cost of the aciclovir).
Will the results help me in caring for my patients?

DO THE PROBABILITY ESTIMATES FIT MY PATIENTS' CLINICAL FEATURES?
The answer to this question depends on how well the clinical characteristics described in the decision analysis fit the situation of your patients.Probably the most important consid- eration in this decision analysis is the risk of symptomatic recurrences occurring at deliv- ery.This has been assumed in the Randolph model to be 14%.Therefore, if you are coun- selling a woman in clinic who has a recurrence of her herpes infection once every 2 years it is unlikely that she has a risk of a recurrence at the time of delivery similar to this and this par- ticular decision analysis may not help you in making a decision about the best way forward for this particular patient.If it is likely however that the woman sitting in front of you has a risk of a recurrence at the time of labour similar to the population in the model then this deci- sion analysis may be of some help if you feel that the results are valid.
This baseline risk of 14% has not been incorporated into the sensitivity analyses and therefore all your patients must fall into this category in order for this decision analysis to be of clinical value.It is likely however that the lower the risk of recurrence for a particular woman the more women will need to be treated to avert one case of neonatal herpes and the more caesarean sections will need to be undertaken.The threshold at which it becomes unacceptable to undertake a large number of interventions in order to prevent one extra case of neonatal herpes has not been addressed in this model.DO THE UTILITIES REFLECT HOW MY PATIENT WOULD VALUE THE OUTCOMES OF THE DECISION?Utility ratings can have a strong influence on the choice of strategies as patients may prefer one outcome much more than another.This decision analysis has not been influenced by utilities.However, it is possible to imagine what those utilities may be and to weigh up in one's own mind the relative trade offs between the various outcome including maternal outcomes which were not explicitly considered in the paper by Randolph et al.This may help you in deciding whether this analysis is of use when dealing with your patients.
Resolution of the scenario Without a good and relevant randomised con- trolled trial of the effectiveness of aciclovir in women with recurrent genital herpes infection your ability to decide the most effective man- agement for your patient will be limited.Aciclovir is not licensed for use in pregnancy and although no long term side effects have been reported with its use the lack of strong data supporting its effectiveness has to be taken into consideration when assessing the usefulness of this decision analysis.
The decision analysis however has been useful in defining those parts of the clinical pathway which are important and where more robust information needs to be obtained before firm recommendations can be given.Until that time, however, the decision analysis, by making the process explicit, should have made you much better informed and better able to judge how to manage your patients.
WERE THE UTILITIES OBTAINED IN AN EXPLICIT AND SENSIBLE WAY FROM CREDIBLE SOURCES?
IN THE BASELINE ANALYSIS, DOES ONE STRATEGY RESULT IN A CLINICALLYIMPORTANT GAIN FOR PATIENTS?IF NOT, IS THE RESULT A TOSS UP?

Table 2
Strategies to prevent neonatal herpes infection.(The number ofcases of neonatal herpes infection averted per 10 000 women treated)

Table 3
Aciclovir to prevent neonatal herpes.Results of the sensitivity analyses Number of neonatal HSV cases averted per 10 000 women for each of the treatment strategies