Test evaluation

A preliminary estimate of test performance can be determined using well-characterised archived specimens. If results are promising, field trials can be conducted to determine test performance in which testing is performed by healthcare workers (HCWs) in settings of intended use17 (figure 1).

Cost-effectiveness

The cost per person screened and treated depends largely on the cost of the test and the prevalence of the STI. The cost of rapid syphilis testing in sub-Saharan Africa amounts to approximately US$2 per woman screened.18 The cost per person treated becomes higher as the prevalence drops. In models developed based on costs in Tanzania and Haiti, approximately 50% of the cost of the rapid test programme is the cost of the tests, with personnel and supplies accounting for less than 15% of the remaining costs.18–20

POCTs are slightly more expensive than RPR (approximately US$0.5, compared with $0.2 for RPR). Nevertheless, even when the costs of training, supervision and QA are included, the cost per ANC attender tested is generally less than US$3.

Integrating syphilis screening into Prevention of Mother to Child Transmission programmes for HIV screening, is particularly cost-effective as personnel time and building space can be considered shared costs, especially in countries that have made a strong commitment for the joint elimination of Mother to Child Transmission of HIV and syphilis.

Baseline survey to determine health systems failures

Before making plans for implementation, a baseline survey is critical as policy makers are sometimes not aware of the low implementation rates or the barriers faced by health facilities around the country. This is especially true when a surveillance system for maternal syphilis screening does not exist or does not function well. Information on current practice for syphilis screening and barriers to achieving full coverage, recent records on the number and proportion of women screened and treated for syphilis can be obtained through interviews with healthcare providers and their supervisors.

Acceptability of rapid syphilis tests among healthcare workers and communities

Although POCTs have been well accepted by HCWs and ANC attenders in many studies, laboratory staff is sometimes opposed to the idea of training non-laboratory personnel such as nurses and midwives to do a task that had traditionally been done by trained laboratory technicians, especially in countries with excellent laboratory services. Laboratory staff needs to be involved in the training and supervision of HCWs doing the tests, and in setting up a QA programme to monitor the proficiency of HCWs in performing the test. Giving certificates of competence motivates the HCWs.

Cultural contexts

In central Haiti, many babies are either stillborn or die in the first year of life. A myth has developed in the communities that a werewolf, the Loup Garou, preys on pregnant women as they sleep, sucking the blood out of the babies in the mother's womb.22 In introducing syphilis rapid tests, Groupe Haitien d'Etude du Sarcome de Kaposi et des Infections Opportunistes and the Ministry of Health worked with HCWs, traditional healers and voodoo priests to overcome these traditional beliefs. Knowledge of syphilis has empowered pregnant women to understand the importance of syphilis screening and treatment to keep their babies healthy.

In working with marginalised populations, understanding their perception of risk, what motivates them to have better healthcare seeking behaviours, and how to gain their trust and make services culturally sensitive and user-friendly are important considerations in planning the introduction of new diagnostic technologies. Cultural issues are also encountered in the treatment of STIs. There are cultures in which patients prefer injections to oral therapies. In some countries, health providers are reluctant to give penicillin to syphilis patients because of their fear of allergic reactions.

Training and site preparations

There is a serious shortage of trained staff in the health sector in many countries. Integrated screening for syphilis and HIV in ANCs can reduce the demands on HCWs, but problems remain even in clinics that have a high turnover of staff. Trained HCWs are often transferred to new facilities and do not always train the HCW remaining on site, leaving the facility unable to provide testing. Training in the use of POCTs needs to be rolled out at the national level, and included in the curriculum at schools of nursing and midwifery.

HCWs need to be trained in the use of POCTs, using training materials translated into the local language and adapted according to local needs. Courses for trainers and supervisors should be organised, including practical training on how to perform the tests and interpret the results, and how to train HCWs and assess their competence, with additional training in stock management, record keeping and quality control.

Standard operating procedures can be developed for performing a finger-prick, performing a POCT, preparing proficiency panels for QA, performing quality control testing and managing an out of specification result, ordering supplies and collecting samples for confirmatory retesting. Algorithms should be developed to guide treatment and other patient management decisions based on the test result. All individuals who test positive should be treated with appropriate drug regimens as recommended in the national guidelines on the same day and requested to refer sexual partners for testing and treatment.

Supply chain management

In many countries, supply chain failure is one of the most common reasons for not testing. Healthcare workers and their supervisors should receive training in stock management, but problems may remain at national, regional or district levels, especially when manufacturers are unable to keep up with the demand or there are long delays in clearance through customs. In recent years, simple optical readers or scanners for POCTs have made it possible to transmit test results to a central database. It is anticipated that supply chain problems may be alleviated if test results can be linked to supply chain software, allowing more accurate forecasts based on real time data on consumption of POCTs.

Quality assurance

For accurate results, it is important to ensure the quality of the tests as well as the testing. A quality management system should be developed that includes the development and distribution of standard operating procedures, training of HCWs and laboratory staff to ensure the quality of the tests and that testing is performed correctly.

A simple, low cost method for proficiency testing has been developed by the US Centres for Disease Control and Prevention to assess the ability of a healthcare worker to correctly perform rapid HIV testing and interpret the results.23 Dry tube specimen panels are prepared by allowing aliquots of sera with high titre antibodies, and negative control sera, to evaporate. These dry tubes can then be stored and transported at ambient temperature, and reconstituted with buffered saline on arrival at the site of HIV testing. This approach can be adapted for use to ensure accurate POC testing for syphilis.

The proficiency panel allows supervisors to identify HCWs in need of refresher training, motivates HCWs, and reinforces good practice. The continuing use of proficiency panels to monitor the quality of testing is particularly important where there is a rapid turnover of staff, but the importance of QA programmes in settings outside of the laboratory is not widely recognised.

Monitoring and evaluation

Programmes should develop a checklist for monitoring visits and train proficient staff members to conduct monitoring visits. After the POC testing programme has been introduced, an internal monitor should be appointed to collect data on the number and proportion of people screened, the proportion that tested positive for syphilis, and the proportion of those testing positive who were treated, and to monitor problems in a timely manner. Problems should be immediately brought to the attention of the programme manager, and solutions tried until a sustainable one is found. External monitors should also visit the sites and provide suggestions for improvement. The opinions of HCWs on the simplicity and acceptability of the POCTs can be sought through interviews and focus group discussions.

Impact on other services

Integrating HIV and syphilis screening in prevention of mother to child transmission programmes avoids the tragedy of preventing babies from HIV infection only to have them die from syphilis, and is cost-effective.24 Vertical disease control programmes and silo healthcare infrastructures often do not result in cost-effective programmes and patient- important outcomes. An essential diagnostic package for prenatal care which can include diagnostics for infectious diseases and conditions such as anaemia and pre-eclampsia, should be considered, based on disease prevalence and epidemiological context, to improve the quality of prenatal care in all settings.

Impact of POCT introduction on health systems

In addition to improving pregnancy outcomes by ensuring higher coverage and earlier treatment of syphilis in pregnancy, the STAT strategy using POCTs can strengthen health systems by motivating HCWs and increasing capacity and efficiency as it reduces the number of return visits required.25

We have conducted a multi-country project to determine feasibility and cost-effectiveness of introducing rapid syphilis screening in prenatal and high-risk populations.25 Using the STAT strategy, we developed a generic protocol with standardised outcome measures. We encouraged all country teams to develop country-specific protocols within the health system infrastructure and the political, cultural and socioeconomic context of their setting. In all countries, policy has changed as a result of this demonstration project. POC testing, using the new rapid syphilis tests to provide a STAT service is now recommended policy in all countries in which the study was conducted, and national guidelines have been amended accordingly. What lessons can be learnt from this? Clearly, it is important to involve policy makers from the outset of a demonstration project such as this, and to keep them informed at all stages. In all six countries, regular meetings were held between project staff and Ministry of Health officials at national, regional and district levels, focussing on those responsible for maternal and child health and HIV/AIDS programmes. Through this project, we learnt that, unlike the situation for the adoption of new drugs and vaccines, leadership and infrastructure for critical decision making on the adoption of new diagnostic technologies is fragmented or absent in many countries in the developing world. The following are critical to the success of test introduction:

1. Engagement of policy makers and stakeholders as peers, throughout the entire study period

2. Sharing and communication of results

3. Using clear compelling messages

4. Identifying champions to promote syphilis screening

5. Training and technical assistance

6. Positive feedback to HCWs

7. Involving other partners

8. Making cost-effectiveness analysis user friendly

9. Keeping motivation high