METHODS

The intervention

In 1997 the Tanzanian Ministry of Health and the African Medical and Research Foundation (AMREF) introduced on-site rapid syphilis screening and treatment with a single dose of benzathine penicillin into routine antenatal care at the main antenatal clinic (ANC) in Mwanza. The syphilis screening programme (known as the intervention) aimed to screen all new attenders for syphilis as part of routine antenatal services. Women were screened if they presented for their first antenatal visit in that pregnancy, or if they had previously attended an ANC that did not offer screening. Women gave a blood sample that was tested by the rapid plasma reagin (RPR) test. All RPR positive women were treated on the same day with a single dose of benzathine penicillin 2.4 MU by intramuscular injection. They were individually counselled on the results. RPR positive mothers were encouraged to send their partners for treatment.

From September 1997 to November 1999 an evaluation of the effectiveness of screening and treatment with single dose benzathine penicillin in averting adverse birth outcomes was conducted. The results of this study provided the cost and cost effectiveness measures for this analysis.⁸

Costs

Cost collection was based on the Costing Guidelines for HIV Prevention Strategies.¹⁴ Costs were collected retrospectively using an ingredients based costing methodology. The total costs of implementing the intervention were collected, including relevant activities undertaken by the municipal health services, the research study, and AMREF. Resources used from July 1998 through June 1999 were compiled to determine the annual costs of running the intervention. Both financial and economic costs were estimated from a provider’s perspective. Financial costs represent actual project expenditure. Economic costs represent the estimated market value of all resources used in the intervention, including donated goods and services. As the intervention was added to existing ANC services, incremental costs were collected. The cost data were collected from administrative records in the health clinic, the research project accounts, and from the AMREF office in Mwanza, as well as through discussions with the project staff and observation at the ANC site. Costs for the start up and implementation of the intervention were included in the analysis. This also included treating partners who presented for treatment. Costs associated with pure research related activities were excluded. Although during the evaluation, quality control was carried out by the study team, the cost of this was not included in our analysis because, before the evaluation, the intervention had abandoned routine quality control because of operational difficulties.

The annual costs of capital goods were calculated using straight line depreciation for financial costs and annualisation with a 3% discount rate for economic costs. Capital goods were given an estimated life between 3 and 10 years. Equipment included an RPR shaker, a refrigerator, and a vacutainer centrifuge. Primary personnel involved were laboratory staff, a counsellor, a nurse, and a team leader.

The average annual exchange rate in the year of the expenditure was used to convert Tanzanian shillings into US$, ranging from 619 to 749 Tanzanian shillings per US$. All costs presented in this paper, including costs quoted from other studies, have been adjusted to 2001 US$. Given the retrospective nature of the cost data collection, it was only possible to estimate the proportion of time spent by staff on the various intervention activities. The impact of this estimation was explored in the sensitivity analysis.

Effectiveness

Data to measure the effectiveness of the intervention for this CE study came from two studies in Mwanza Region. The methodology for these studies has been previously published.^8,15 In summary, to measure the impact of untreated maternal syphilis on birth outcome, a retrospective cohort of 382 women was recruited at three hospitals in Mwanza Region.¹⁵ At delivery, consenting women, who had not been screened for syphilis during pregnancy, were screened for syphilis by the RPR test. Maternal sera from enrolled women were also tested by the Treponema pallidum haemagglutination assay (TPHA). If the TPHA test was negative and the RPR test was positive, sera were also tested by the fluorescent treponemal antibody assay (FTA). Syphilis stages were defined as high titre active syphilis (HTAS) when the RPR was reactive at a dilution of ⩾1:8, low titre active syphilis (LTAS) when the RPR titre was reactive at a titre of <1:8, and old or resolved/treated syphilis when the RPR was non-reactive but the TPHA was positive.^8,15,16 This definition was adopted in order to facilitate comparison with other African studies and because clinically based definitions could not be confirmed with available diagnostic tests in Mwanza. Of the 140 RPR positive women, 21% had a titre <1:8, 19% had a titre ⩾1:8 and <1:32, and 34% had titres ⩾1:32. Twenty six per cent of these women had a biological false positive RPR reaction shown by negative TPHA and FTA tests. Data were collected on stillbirth and LBW (which included prematurity) and cord blood serology. The study showed that women with untreated HTAS had a risk of stillbirth 18 times higher than women without syphilis, and a fourfold risk of any adverse outcome.¹⁵ No association was found between other stages of syphilis and adverse birth outcomes.

To measure the effectiveness of syphilis screening and treatment with single dose benzathine penicillin, a prospective cohort of 556 RPR positive and 1132 RPR negative ANC attenders was recruited and followed to delivery. On average, women presented for their first ANC visit in their 25th week of pregnancy. New ANC attenders were screened routinely using the RPR test. Of these women, 7.7% were RPR positive and were treated with a single dose of 2.4 million units of benzathine penicillin intramuscularly at the same visit. Twenty seven per cent had HTAS. Infants were examined at birth for stillbirth and LBW including prematurity. The study showed that women screened and treated at the ANC for HTAS or LTAS had no increased risk of adverse pregnancy outcomes compared with seronegative women.⁸ This provided strong evidence that the intervention was effective in averting adverse outcomes attributable to maternal syphilis, since the first study had shown that approximately 49% of untreated women with HTAS from Mwanza Region experienced an adverse birth outcome.¹⁵

The cohort study did not include an infant follow up study and would not have had untreated infants to follow up to measure infants developing clinical congenital syphilis. Although all infants in the study were examined at birth, it is rare for an infected infant to show evidence of congenital syphilis at birth.⁶ Most symptoms of congenital syphilis appear from the age of 3 weeks and older. Published literature suggests that 14% to 50% of infants born to infected mothers will develop congenital syphilis,^7,9 but there is little evidence about the proportion of infants with positive serology and a normal birth weight who develop clinical congenital syphilis. As it was not clear how much of the burden of disease was already taken into account by DALYs saved by averting low birth weight, this study has assumed that no cases of congenital syphilis were averted. This assumption is explored in the sensitivity analysis.

Cost effectiveness

Based on analysis of data from the 26 month prospective cohort study, effectiveness data for the CE study were matched for the year of the cost study. During this period, 9713 women were screened of which 696 (7.2%) were RPR positive, and 264 partners presented for treatment at the ANC (0.38 partners per woman). The number of adverse outcomes averted was estimated by comparing the results of the two studies described above and are presented in table 1.^8,15 Of women with a positive RPR test at the ANC, 26.8% (187 women) were estimated to have HTAS.¹⁷ During the period of the costing, the intervention prevented 44 stillbirths and 31 LBW infants, averting 75 adverse birth outcomes in total.

DALYs were calculated following the methodology described by Fox-Rushby and Hanson.¹⁸ DALYs are defined as the sum of years of life lost (YLL) due to premature mortality and years lived with disability (YLD) adjusted for severity.¹⁹ DALYs were calculated using adverse outcomes averted from the above studies.^8,15

In the past, stillbirth has not been included in DALY calculations because life expectancy is considered to begin at live birth.²⁰ However, because stillbirth is the most important adverse outcome of untreated maternal syphilis in Mwanza Region,¹⁵ we have included DALYs saved by averting stillbirths, where stillbirth was considered a full life lost. DALYs saved are presented including and excluding stillbirth.

Sequelae of adverse pregnancy outcomes were taken from published literature.^8,15,21–25 Of LBW infants, approximately 13% would have died and 6% would have experienced a lifelong disability with a disability weight of 0.291 (the greater the weight the more severe the disability; a weight of 0 is no disability).¹⁸ Tanzanian life expectancy at birth (45 years), and the life expectancy at birth used in the WDR (81.75 years) were used to facilitate comparison with other studies.^3,26

A univariate sensitivity analysis was performed varying the main assumptions and inputs in the analysis. Personnel time inputs were reduced by 10% and increased by 20%. The study coordinator’s input was varied between 2% and 5% of her time in the recurrent costs, and the discount rate was varied from 1% to 6% in both the cost and DALY calculation. As the distribution of syphilis titres differed between the retrospective and the prospective studies, we explored the impact of 53% and 20% of women who tested RPR positive had HTAS. The mortality rates of LBW infants are highly variable. We used rates ranging from 10% to 40%.^21–25 Although we assumed no cases of congenital syphilis were averted, we explored the effect of averting congenital syphilis if 14% and 50% of infants born to untreated mothers developed congenital syphilis and none of the infants born to treated mothers did. LBW and congenital syphilis were assumed not to be mutually exclusive, therefore a LBW infant could also develop CS. DALYs saved were DALYS saved from LBW cases averted plus DALY saved from CS cases averted.

RESULTS

Incremental costs

Table 2 presents the total annual costs, outcome indicators and average costs. The incremental economic costs were $13 956, 1% higher than financial costs of $13 852. As these are so similar, only detailed economic costs are presented. Personnel and supplies accounted for 91% of total costs. More than 60% of capital costs were attributable to start up activities.

The average costs were calculated by dividing the annual incremental economic costs by the outcome indicators. The cost per woman screened was $1.44, the cost per woman treated was $20.05, and the cost per person (women and partners) treated was $14.53. The cost per averted stillbirth was $318.37. The cost per LBW averted was $451.18. The cost per all adverse outcomes averted was $186.66. The CE of the intervention in preventing LBW, as measured by the cost per DALY saved, was $110.03 and $92.56, using the Tanzanian life expectancy and the WDR life expectancy, respectively. When including stillbirths, the cost per DALY saved was $10.56 or $8.88, depending on the life expectancy.

Sensitivity analysis

Reducing the DALY discount rate to 1% improved the CE by 36%, and increasing the discount rate to 6% worsened the CE by 79% (table 3). The percentage of RPR positive women with HTAS had a large impact on cost effectiveness. An increase in the percentage of HTAS in RPR positives to 53% would decrease the cost per DALY saved including stillbirth to $5.43, a 49% improvement. The mortality rate from LBW was important in the cost per DALY saved due to LBW births averted, worsening CE by 28%, and improving it by 64% with a lowering of the mortality to 10% and an increase to 40%, respectively. The impact on cost per DALY saved was far smaller when stillbirth was included, the divergence from the central estimate ranging from 2% to −15%. If the intervention averted 14% to 50% incidence of congenital syphilis, the CE would improve by 31% to 62%, but only improved CE by 4% to 13% in the CE estimate including stillbirth. The CE estimates were not as sensitive to changes in the cost assumptions, ranging between −6% and 16% divergence from the central estimate.

The role of prevalence

Cost effectiveness of the intervention is influenced by syphilis prevalence. We estimated the effect of different levels of syphilis prevalence (between 2% and 15%) on the cost effectiveness of the intervention. The cost per DALY saved decreases substantially as prevalence increases. However, even at RPR prevalences as low as 2% antenatal syphilis screening and treatment remains cost effective ($33 per DALY saved including stillbirth; fig 1).

DISCUSSION

The cost per DALY saved by the intervention in Mwanza ranged from $5.34 to $196.63. A detailed breakdown of the cost by input type was provided in order to consider the cost of replicating the intervention elsewhere. The effectiveness of the intervention was estimated in the same region over the same period using the Mwanza syphilis study results. This should provide a more reliable estimate of the intervention’s effectiveness in this population than using assumptions of effectiveness based on historical data.

Comparison of cost effectiveness studies

As none of the comparison studies included building costs, these studies were interpreted as providing costs for adding antenatal syphilis screening on to routine antenatal services. Hence these costs were interpreted as incremental costs. Jenniskens et al estimated the recurrent costs of on-site antenatal syphilis screening in Nairobi to be $34 per case treated,¹⁰ including treating 0.57 partners per woman, which is more than double the cost incurred here. However, costs included a yearly refresher course as well as quality control, in contrast with this study where systematic quality assurance mechanisms were not in place. In the Jenniskens et al study, there was no breakdown of the costs, they did not include start up costs, and it is unclear whether capital costs were included. They assumed 40% of babies born to RPR positive women would contract congenital syphilis, while none of the babies born to treated women would. The cost per case of congenital syphilis averted was $62. If we had calculated the numbers of congenital syphilis cases averted in the same way, we would have found a similar cost ($50 per case averted).

Fonck et al examined the CE of this same syphilis screening programme after 6 years.¹¹ More detailed costs were presented, but neither the costs of initial staff training nor clinic staff salaries were included. The source of the effectiveness assumptions was not clear. The sensitivity analysis from our study suggests that the variation in effectiveness is highly significant in determining CE, and so places even greater importance on ensuring appropriate measurement of outcome variables.

Hira et al estimated the incremental CE of a syphilis screening programme that included health education and attempted to screen women twice during pregnancy in Zambia.⁹ The CE estimate assumed 100% of RPR positive women were treated, although only 46% of their cohort actually received treatment. Using the actual numbers treated more than doubled their cost per adverse outcome averted ($44 per adverse outcome averted rather than $19). In addition, it appears that they modelled the costs, rather than costing the actual project as implemented, and did not include any training or labour costs. Including the same portion of labour as used in Mwanza the cost per adverse outcome averted would rise to $70. Key limitations of all the above studies are their assumptions on the number of adverse outcomes averted. Comparison of these three studies showed that as syphilis prevalence rises, the costs effectiveness of the intervention improved.

Our study is compared with the other three existing studies in table 4. The left side of the table presents data from these studies, and uses them to calculate DALYs saved. Cost effectiveness for the three studies is modelled as if it were implemented in Mwanza, using effectiveness parameters from the current study on the right side. A number of critical assumptions are highlighted in table 4. The stage of syphilis assumed to affect pregnancy outcome varied for the different studies. The study of untreated maternal syphilis in Mwanza found no association between LTAS or biological false positive reactions and adverse birth outcomes.¹⁵ We therefore only included women with HTAS in our calculations, 27% of those women treated at the ANC. Although the retrospective cohort had a higher percentage of RPR positive women with HTAS, this group is not representative of antenatal clinic attenders in terms of their treatment seeking behaviour. Hira and Jenniskens assumed all births of untreated RPR positive women would be affected, and Fonck assumed 69% would be affected.^9–11 These studies assumed that 14%, 33% and 40% of children born live to an infected mother develop clinical congenital syphilis.^9–11 We assumed no cases of congenital syphilis were averted.

The CE of the intervention in Tanzania falls between that of the Kenyan and Zambian studies when outcomes are measured by process indicators (number of women screened and numbers of people treated). The RPR prevalence in Mwanza also lies between the Kenyan and Zambian RPR prevalences. The intervention would need to improve its partner presentation rate from 0.38 partners per woman to 0.68 partners per woman to match the cost per person treated found in Zambia. There is a difference in cost per adverse outcome averted because of the other studies modelled adverse outcomes averted, while the Mwanza study measured the proportion of adverse outcomes in untreated mothers and assumed that these were all averted in treated mothers because they were not at increased risk compared to seronegative women. Using the effectiveness parameters from the Mwanza studies, these studies may have underestimated the cost per DALY saved by a factor of 2 to 3, depending on the distribution of syphilis titres in their RPR positive population.

Despite the range of CE estimates (between $4 and $33 per DALY saved depending on RPR prevalences, titre distributions and outcome assumptions, and up to $110 for DALYs saved by averting LBW alone), all fall below the suggested upper bound of CE interventions given by the WDR ($193).³

Comparison with other interventions

The costing of nevirapine treatment for the PMTCT in Uganda with an ANC HIV seroprevalence of 15% estimates the incremental cost per DALY saved to be $11.² The syphilis screening intervention also compares well with other child health interventions. Cost effectiveness estimates for immunisations and oral rehydration therapy range from $17–42 per DALY and $7 and $28 per DALY respectively.²⁷

The CE calculations of this study used recent estimates of infections averted. Only adverse outcomes that were found to be associated with untreated maternal syphilis in Mwanza and averted by syphilis screening and treatment (stillbirth and LBW) were considered in this analysis. Neither congenital syphilis averted nor adult infections treated were taken into account. Syphilis screening and treatment also provides an opportunity to identify and counsel couples who may be at increased risk for HIV infection, thereby possibly preventing HIV infections. These make the current CE study more conservative.

There has been some debate about the construction of the DALY for antenatal interventions, and criticism that the WDR approach is too restrictive.²⁰ The inclusion of stillbirth in constructing DALYs differentiates this study from other ANC syphilis screening and treatment studies as well as other maternal-child health interventions.

The CE of maternal syphilis screening is well within the range considered acceptable for other antenatal and child health interventions. The CE of screening and treatment is highly dependent on the prevalence of syphilis, with the cost per DALY saved decreasing with higher prevalences. If having to make choices between implementing PMTCT and syphilis screening programmes, it is critical to take the prevalence of HIV and syphilis into account. “Competition for resources and attention between high effect interventions for child survival and MTCT will not save children’s lives. Building on the potential synergies of shared service delivery mechanisms for child survival and MTCT will save lives.”²⁸ Within the context of high prevalence of HIV and syphilis, economies of scope can be achieved, rendering both PMTCT and syphilis prevention activities more cost effective. Economies of scope arise when an increase in the range of services provided leads to lower average costs than when the services are provided independently of each other. Although the PMTCT has an important role in antenatal care in high HIV prevalence areas, it is crucial that it does not divert resources away from cost effective, but less high profile, interventions. This study shows that on-site syphilis screening and treatment are extremely good value for money and, given the devastating impact of untreated maternal syphilis in pregnancy, are urgently in need of scaling up to national level in sub-Saharan African countries.