You are here

Article Text

Article menu
Original research
Developing a symptoms-based risk score for infectious syphilis among men who have sex with men
  1. Silvia Achia Nieuwenburg1,
  2. Elske Hoornenborg1,
  3. Udi Davidovich1,
  4. Henry John Christiaan de Vries1,2,3,
  5. Maarten Schim van der Loeff1,2,4
  1. 1 Department of Infectious Diseases, Public Health Service Amsterdam, Amsterdam, The Netherlands
  2. 2 Amsterdam Institute for Infection and Immunity (AII), Amsterdam UMC location University of Amsterdam, Amsterdam, The Netherlands
  3. 3 Department of Dermatology, Amsterdam UMC location University of Amsterdam, Amsterdam, The Netherlands
  4. 4 Department of Internal Medicine, Division of Infectious Diseases, Amsterdam UMC location University of Amsterdam, Amsterdam, the Netherlands
  1. Correspondence to Silvia Achia Nieuwenburg, GGD Amsterdam, Amsterdam 1018 WT, The Netherlands; snieuwenburg{at}ggd.amsterdam.nl

Abstract

Background Syphilis incidence is rising among men who have sex with men (MSM). An online tool based on a risk score identifying men with higher risk of infectious syphilis could motivate MSM to seek care. We aimed therefore to develop a symptoms-based risk score for infectious syphilis.

Methods We included data from all consultations by MSM attending the Amsterdam Centre for Sexual Health in 2018–2019. Infectious syphilis (ie, primary, secondary or early latent syphilis) was diagnosed according to the centre’s routine protocol. Associations between symptoms and infectious syphilis were expressed as odds ratios (OR), with 95% confidence intervals (CI). Based on multivariable logistic regression models, we created risk scores, combining various symptoms. We assessed the area under the curve (AUC) and cut-off based on the Youden Index. We estimated which percentage of MSM should be tested based on a positive risk score and which percentage of infectious syphilis cases would then be missed.

Results We included 21,646 consultations with 11,594 unique persons. The median age was 34 years (IQR 27–45), and 14% were HIV positive (93% on antiretroviral treatment). We diagnosed 538 cases of infectious syphilis. Associations with syphilis symptoms/signs were strong and highly significant, for example, OR for a painless penile ulcer was 35.0 (CI 24.9 to 49.2) and OR for non-itching rash 57.8 (CI 36.8 to 90.9). Yet, none of the individual symptoms or signs had an AUC >0.55. The AUC of risk scores combining various symptoms varied from 0.68 to 0.69. For all risk scores using cut-offs based on Youden Index, syphilis screening would be recommended in 6% of MSM, and 59% of infectious syphilis cases would be missed.

Conclusion Symptoms-based risk scores for infectious syphilis perform poorly and cannot be recommended to select MSM for syphilis screening. All MSM with relevant sexual exposure should be regularly tested for syphilis.

  • syphilis
  • diagnosis
  • homosexuality, male

Data availability statement

All data relevant to the study are included in the article or uploaded as supplemental information.

https://creativecommons.org/licenses/by/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

https://doi.org/10.1136/sextrans-2022-055550

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THIS TOPIC

    • A symptoms-based risk score for infectious syphilis motivating men who have sex with men (MSM) to seek care has not previously been developed.

    WHAT THIS STUDY ADDS

    • We developed a risk score based on symptoms and notification for syphilis. Individual symptoms included in the risk score were strongly associated with infectious syphilis, but the risk score had poor sensitivity and specificity.

    HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

    • Symptoms-based risk scores for infectious syphilis cannot be recommended to select MSM for syphilis screening; all MSM at risk of syphilis should be regularly screened.

    Introduction

    Syphilis, caused by the bacterium Treponema pallidum, remains a public health concern. In many countries, the incidence of syphilis is rising, especially among populations of men who have sex with men (MSM).1 If left untreated, it can lead to serious cardiovascular and neurological complications.2 Syphilis can also facilitate the transmission of HIV.3 It is highly contagious through sexual contact during the primary and secondary stage, as well as in the early latent stage. Long-acting penicillin G, an effective and inexpensive antibiotic, remains the recommended treatment for syphilis.4

    The signs and symptoms of syphilis vary depending on the stage, and for this reason syphilis is also known as ‘the great imitator’.5 This makes it challenging for patients to recognise the disease and seek a health provider when they are symptomatic. The primary stage is characterised by a usually painless ulcer at the site of infection, usually appearing within 3 months after acquisition and disappearing within 6 weeks. If left untreated, the disease may progress to the secondary stage characterised by a non-itching rash.6 Early latent stage syphilis is asymptomatic and can be diagnosed with serological tests in patients without a history of primary or secondary symptoms.

    The increasing incidence of syphilis calls for preventive efforts to reduce exposure to the infection. To reduce risk of ongoing transmission (especially in the primary and secondary stage) and of sequelae, early diagnosis and treatment are key, and it would be helpful if patients would be able to recognise the disease and be motivated to seek care.

    Therefore, the Centre for Sexual Health of the Public Health Service of Amsterdam and the national non-governmental organisation STI-AIDS Netherlands launched a campaign to create awareness of syphilis in Amsterdam, the Netherlands. The awareness campaign, which ran from 2018 to 2019, included an online tool to enable individuals to self-identify symptoms that could be related to syphilis, and if indicated motivate them to seek care (testing and immediate treatment in case a diagnosis was made). This online tool included an expert-based algorithm of various symptoms; we also included a partner notification for syphilis in the risk score.7 The aim of the current study was to assess whether an evidence-based symptoms-based risk score could be developed, which could help individuals to self-identify possible infectious syphilis.

    Methods

    There are few data from the peer-reviewed literature on the associations of symptoms with a current syphilis infection. We therefore developed a syphilis assessment algorithm in consultation with experts (dermatologists and internist-infectiologist). Based on the expert opinions, the following signs and symptoms were included in the risk algorithm: a painful or painless ulcer on the penis, in the mouth, at the anus or on the skin, with or without palpable regional lymph nodes, and an itching or non-itching rash with or without influenza-like symptoms. In addition, we also added partner notification for syphilis. A complete overview of the included items is provided in online supplemental table 1.

    Supplemental material

    To assess the validity of the expert-based algorithm, we used data from all consultations by MSM attending the Centre for Sexual Health of the Amsterdam Public Health Service, the Netherlands, from July 2018 to July 2019, regardless of reason of visit. Consultations were at the client’s own initiative and free of charge.

    For this study, the outcome of interest was infectious syphilis, which comprises primary, secondary and early latent syphilis. A syphilis diagnosis was made according to the centre’s routine protocol using a combination of clinical signs and symptoms and laboratory testing based on the European guidelines for syphilis management.8 Serological screening for syphilis consisted of a treponemal test chemiluminescence immunoassay (CLIA, DiaSorin, Saluggia, Italy) confirmed by an immunoblot (INNO-LIA Syphilis, Innogenetics, Ghent, Belgium) and a non-treponemal test, the rapid plasma reagin (RPR, RPR NOSTICON II, bioMérieux, Marcy l’Etoile, France). In persons with suspected primary syphilis, we additionally performed dark-field microscopy of ulcer exudate and a PCR of ulcer scraping.9 Secondary syphilis was diagnosed if a client had a positive serology with characteristic skin or mucocutaneous lesions. Early latent syphilis was defined as confirmed positive serology without clinical manifestations, according to the European guidelines.8

    We described person characteristics by medians and interquartile ranges (IQRs), and numbers and percentages. Univariable logistic regression was done using Generalized Estimating Equations (GEE), because of correlated data (multiple visits by the same person). Logistic regression was done to obtain ORs with 95% CIs for infectious syphilis diagnosis for each symptom and risk factor. Various multivariable models were constructed to enable creation of a series of prediction models and prediction scores. Our initial risk score A1 was based on the symptoms of the expert-based algorithm: ulcer in the mouth, ulcer on the skin, painless ulcer at the anus with palpable regional lymph nodes, painless ulcer on the penis with palpable regional lymph nodes, painless ulcer at the anus without palpable regional lymph nodes, painless ulcer on the penis without palpable regional lymph nodes, painful ulcer at the anus with palpable regional lymph nodes, painful ulcer on the penis with palpable regional lymph nodes, painful ulcer at the anus without palpable regional lymph nodes, painful ulcer on the penis without palpable regional lymph nodes, rash on the palms of the hands or soles of the feet, itching rash, non-itching rash and a partner notification for syphilis (online supplemental table 1).

    In risk score A1, each factor was awarded 1 point if present and 0 if not, that is, without weighing. Risk score A2 included the same factors as risk score A1, but here each symptom/factor was given a weight equal to the rounded regression coefficient of the multivariable analysis. Risk score B1 was a simplified risk score. Here we only included the symptoms and risk factor (a partner notification for syphilis) that were significantly associated with infectious syphilis in model A1, without weighing. Risk score B2 included the same factors as risk score B1, but with weighing based on the regression coefficients. To optimise the risk scores, we also calculated risk scores C1 and C2. Penile ulcers were associated with infectious syphilis, independently of lymph nodes being palpable or not. Therefore, risk score C1 included the same factors as risk score B1, but rather than including painless and painful ulcers with and without regional palpable lymph nodes, we included painless and painful ulcers, both regardless of lymph nodes. Risk score C2 included the same factors as risk score C1, but with weighing based on the regression coefficients.

    For each risk score, we assessed the area under the curve (AUC). To evaluate the clinical relevance of the risk scores A, B and C, the post-test probability of disease was evaluated. The positive predictive value of having infectious syphilis when having a score above the cut-off was estimated; the cut-off was based on the Youden Index. The Youden Index measures the effectiveness of a diagnostic marker as follows: sensitivity+specificity–1.10 As there is no generally agreed optimum of sensitivity and specificity combination, we selected the highest value for the Youden Index as optimal cut-off. Furthermore, we estimated which percentage of infectious syphilis cases would be missed, if patients would only be tested for syphilis if their score exceeded the cut-off of the risk score. A p value of <0.05 was considered statistically significant. We carried out analyses using Stata (V.15.1, StataCorp, College Station, Texas, USA).

    Results

    In total, 21,646 consultations of 11,594 persons were included in this study (see table 1 which provides details of persons at their first visit in the study period). The median age was 34 years. A total of 1,586 (13.7%) were living with HIV; of the 1,501 with information on viral load, 1,379 (91.9%) reported an undetectable viral load. In total, 421 (3.6%) persons were notified for syphilis.

    View this table:
    Table 1

    Characteristics of all MSM at first consultation in study period; Amsterdam Centre for Sexual Health, July 2018–July 2019

    At the 21,646 consultations, 538 cases of infectious syphilis were diagnosed (2.5%). A total of 1270 men (5.9%) reported at least one of the symptoms included in the expert-based risk score. In 260 consultations (1.2%), a penile ulcer was present, in 114 (0.5%) an anal ulcer and in 148 (0.7%) a skin rash. The characteristic rash on hand palms or feet soles was rare, present in 23 (0.1%) consultations.

    In univariable analysis, all symptoms and the risk factor (a partner notification for syphilis) used were associated with a diagnosis of infectious syphilis (online supplemental table 2). The self-reported symptoms: non-itching rash with influenza-like symptoms (OR 86.8, 95% CI 35.2 to 214.1), rash on the soles of the feet (OR 78.6, 95% CI 7.1 to 867.6), painless anal ulcer with palpable regional lymph nodes (OR 78.6, 95% CI 7.1 to 867.6) and painless penile ulcer with palpable regional lymph nodes (OR 76.6, 95% CI 34.0 to 172.6) were most strongly associated with an infectious syphilis diagnosis. The AUCs of the individual symptoms or signs were between 0.50 and 0.55.

    In a multivariable analysis including all symptoms (and a partner notification for syphilis) included in the expert-based risk score (model A), the following symptoms were strongly associated with an infectious syphilis diagnosis: painless penile ulcer with palpable regional lymph nodes (adjusted OR (aOR) 85.8, 95% CI 36.6 to 201.0), painless anal ulcer with palpable regional lymph nodes (aOR 63.8, 95% CI 4.1 to 991.8) and non-itching rash (aOR 54.3, 95% CI 32.7 to 90.2) (table 2). In model B, which is similar to model A, but with non-significant symptoms omitted, the same symptoms were strongly associated with an infectious syphilis diagnosis: painless penile ulcer with palpable regional lymph nodes (aOR 85.7, 95% CI 36.6 to 200.6), non-itching rash (aOR 64.8, 95% CI 40.4 to 103.7) and painless anal ulcer with palpable regional lymph nodes (aOR 63.6, 95% CI 4.1 to 989.3). Because a painless penile ulcer was strongly associated with infectious syphilis regardless of palpable regional lymph nodes, and also a painful penile ulcer was strongly associated with infectious syphilis regardless of palpable regional lymph nodes, we made model C in which ‘painless penile ulcer’ and ‘painful penile ulcer’ were included rather than the four separate items. In this optimised model C, a painless anal ulcer with inguinal lymph nodes (aOR 65.4, 95% CI 4.4 to 981.1), non-itching rash (aOR 64.8, 95% CI 40.5 to 103.7) and painless penile ulcer (aOR 36.1, 95% CI 25.2 to 51.6) were most strongly associated with an infectious syphilis diagnosis.

    View this table:
    Table 2

    Multivariable analysis of possible syphilis symptoms and their association with a diagnosis of infectious syphilis among MSM; Amsterdam Centre for Sexual Health, 2018–2019

    Risk score A1, based on the expert-based algorithm, had a sensitivity of 40.9% and a specificity of 95.0%. Based on the cut-off of 1.0 (based on Youden Index), 5.9% of men would have to be tested and 59% of syphilis cases would remain undetected. The risk score using weighing (A2) performed similarly. The other four risk scores (B1, B2, C1, C2) performed almost identical to the risk scores A1 and A2. The AUC was 0.68 for risk score A1 based on the expert-opinion algorithm (figure 1). The AUCs of the other risk scores, including our optimised risk score C2, were all also 0.68. Table 3 shows the sensitivity and specificity of the various risk scores for infectious syphilis.

    Figure 1
    Figure 1

    ROC curve of score C2 among MSM with infectious syphilis; Amsterdam Centre for Sexual Health. Risk score C2 is comprised of the following symptoms (between brackets the weights of each symptom): painless anal ulcer with palpable regional lymph nodes (4.2)+painful anal ulcer with no palpable regional lymph nodes (2.3)+painless penile ulcer (3.5)+painful penile ulcer (3.1)+itching rash (2.1)+non-itching rash (4.2)+a partner notification for syphilis (1.5). MSM, men who have sex with men; ROC, receiver operating characteristics.

    View this table:
    Table 3

    Performance of several risk scores for infectious syphilis among MSM; Amsterdam Centre for Sexual Health, July 2018–July 2019

    Discussion

    We aimed to assess whether an evidence-based symptoms-based risk score could be developed to identify individuals with infectious syphilis. The optimised risk score C2, including six symptoms and a partner notification for syphilis, had a sensitivity of 40.5% (95% CI 36.3% to 44.8%), a specificity of 95.3% (95% CI 95.0% to 95.6%) and an AUC of 0.68. Based on this score, 5.6% of the study population would be eligible for syphilis testing and 41% of the cases would be identified.

    We had aimed that a symptoms-based risk score would enable MSM to self-identify possible infectious syphilis to increase early testing and diagnosis, and avoid further transmission and complications. To our knowledge, there are no studies on the development of a symptoms-based risk score for infectious syphilis. One study developed a simple-to-use nomogram to predict the risk of syphilis, and several other studies described the risk factors for syphilis based on sociodemographic, clinical or behavioural data.11–14 The main objective of the current study was to develop a symptoms-based risk score so that individuals would seek help when symptomatic, analogous to similar algorithms for acute HIV infection. Several symptom and behavioural risk scores for acute HIV have been developed and validated successfully.15–18 The symptoms-based risk scores may have the advantage above behavioural scores as they may be generalisable among different populations and be unaffected by frequency of high-risk behaviour. Unfortunately, our risk scores did not have promising characteristics and infections would be missed by using the tool. Various iterations to increase sensitivity or specificity did not lead to improved performance; this is probably due to the fact that many infectious syphilis cases are asymptomatic.

    This study has several strengths. The data of the Centre for Sexual Health of Amsterdam provided us with a large sample size. Secondly, self-reported symptoms were systematically noted in all the consultations. Furthermore, all clients were tested systematically based on their symptoms and risk behaviour. All assessed symptoms in this study were highly predictive for infectious syphilis. The study also has several limitations. We chose not to include risk behaviour in the risk score. Including risk behaviour might improve the algorithm, yet this would not benefit our objective of self-reported symptoms-based algorithms leading to seeking care. Furthermore, the cut-offs of our risk scores were based on the Youden Index, but this is a rather arbitrary cut-off; the optimal cut-off may depend on the prevalence of infectious syphilis in the population, financial resources and acceptability of low sensitivity. Lastly, we used data of MSM visiting the Centre for Sexual Health; possible results may not be generalisable to all MSM including those who do not yet seek sexual healthcare.

    For the prevention of syphilis, multiple approaches are necessary. An awareness campaign with a symptoms-based risk score will likely not prevent the ongoing transmission of infectious syphilis. Early identification of the infection and timely treatment are helpful to reduce the duration of infection and onwards sequela. An online symptoms tool might in some cases be harmful as it may warrant incorrect assessment of no risk. Thus, all MSM who engage in sexual behaviour that places them at higher risk of acquiring STIs should be regularly tested for syphilis.8 Yet, an online tool might help to motivate MSM to seek help timely in case of suspected symptoms.

    In conclusion, our proposed symptoms-based risk scores performed poorly to diagnose infectious syphilis. Thus, we cannot recommend a symptoms-based risk score to select MSM for syphilis screening. All MSM with relevant sexual exposure should be regularly tested for syphilis.

    Data availability statement

    All data relevant to the study are included in the article or uploaded as supplemental information.

    Ethics statements

    Patient consent for publication

    Ethics approval

    Ethical approval was not required for this study, as it made use of routinely collected data and all data were de-identified prior to analysis.

    References

    1. ↵
      1. World Health Organization
      . Report on global sexually transmitted infection surveillance. Geneva: WHO, 2018.
    2. ↵
      2. Hook EW
      . Syphilis. Lancet 2017;389:1550–7.doi:10.1016/S0140-6736(16)32411-4 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27993382
      OpenUrlCrossRefPubMed
    3. ↵
      2. Wu MY ,
      3. Gong HZ ,
      4. Hu KR , et al
      . Effect of syphilis infection on HIV acquisition: a systematic review and meta-analysis. Sex Transm Infect 2021;97:525–33.doi:10.1136/sextrans-2020-054706 pmid:http://www.ncbi.nlm.nih.gov/pubmed/33219164
      OpenUrlAbstract/FREE Full Text
    4. ↵
      2. Clement ME ,
      3. Okeke NL ,
      4. Hicks CB
      . Treatment of syphilis: a systematic review. JAMA 2014;312:1905–17.doi:10.1001/jama.2014.13259 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25387188
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      2. Çakmak SK ,
      3. Tamer E ,
      4. Karadağ AS , et al
      . Syphilis: a great imitator. Clin Dermatol 2019;37:182–91.doi:10.1016/j.clindermatol.2019.01.007 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31178101
      OpenUrlCrossRefPubMed
    6. ↵
      2. Ghanem KG ,
      3. Ram S ,
      4. Rice PA
      . The modern epidemic of syphilis. N Engl J Med 2020;382:845–54.doi:10.1056/NEJMra1901593 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32101666
      OpenUrlCrossRefPubMed
    7. ↵
      Syfilis. Available: www.syfilistest.nl
    8. ↵
      2. Janier M ,
      3. Unemo M ,
      4. Dupin N , et al
      . 2020 European guideline on the management of syphilis. J Eur Acad Dermatol Venereol 2021;35:574–88.doi:10.1111/jdv.16946 pmid:http://www.ncbi.nlm.nih.gov/pubmed/33094521
      OpenUrlCrossRefPubMed
    9. ↵
      2. Heymans R ,
      3. van der Helm JJ ,
      4. de Vries HJC , et al
      . Clinical value of Treponema pallidum real-time PCR for diagnosis of syphilis. J Clin Microbiol 2010;48:497–502.doi:10.1128/JCM.00720-09 pmid:http://www.ncbi.nlm.nih.gov/pubmed/20007388
      OpenUrlAbstract/FREE Full Text
    10. ↵
      2. Fluss R ,
      3. Faraggi D ,
      4. Reiser B
      . Estimation of the Youden index and its associated cutoff point. Biom J 2005;47:458–72.doi:10.1002/bimj.200410135 pmid:http://www.ncbi.nlm.nih.gov/pubmed/16161804
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      2. Zhao P ,
      3. Yang Z ,
      4. Li B , et al
      . Simple-to-use nomogram for predicting the risk of syphilis among MSM in Guangdong Province: results from a serial cross-sectional study. BMC Infect Dis 2021;21:1199.doi:10.1186/s12879-021-06912-z pmid:http://www.ncbi.nlm.nih.gov/pubmed/34844553
      OpenUrlCrossRefPubMed
    12. ↵
      2. Arando M ,
      3. Fernandez-Naval C ,
      4. Mota-Foix M , et al
      . Early syphilis: risk factors and clinical manifestations focusing on HIV-positive patients. BMC Infect Dis 2019;19:727.doi:10.1186/s12879-019-4269-8 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31420018
      OpenUrlCrossRefPubMed
    13. ↵
      2. Roth JA ,
      3. Franzeck FC ,
      4. Balakrishna S , et al
      . Repeated syphilis episodes in HIV-infected men who have sex with men: a multicenter prospective cohort study on risk factors and the potential role of syphilis immunity. Open Forum Infect Dis 2020;7:ofaa019.doi:10.1093/ofid/ofaa019 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32016128
      OpenUrlCrossRefPubMed
    14. ↵
      2. Towns JM ,
      3. Huffam S ,
      4. Chow EPF , et al
      . Clinical factors associated with syphilis concordance in men in sexual partnerships: a cross-sectional couples study. Sex Transm Infect 2018;94:571–3.doi:10.1136/sextrans-2017-053297 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29191816
      OpenUrlAbstract/FREE Full Text
    15. ↵
      2. Dijkstra M ,
      3. de Bree GJ ,
      4. Stolte IG , et al
      . Development and validation of a risk score to assist screening for acute HIV-1 infection among men who have sex with men. BMC Infect Dis 2017;17:425.doi:10.1186/s12879-017-2508-4 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28615005
      OpenUrlCrossRefPubMed
    16. ↵
      2. Lin TC ,
      3. Gianella S ,
      4. Tenenbaum T , et al
      . A simple symptom score for acute human immunodeficiency virus infection in a San Diego community-based screening program. Clin Infect Dis 2018;67:105–11.doi:10.1093/cid/cix1130 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29293891
      OpenUrlCrossRefPubMed
    17. ↵
      2. Sanders EJ ,
      3. Wahome E ,
      4. Powers KA , et al
      . Targeted screening of at-risk adults for acute HIV-1 infection in sub-Saharan Africa. AIDS 2015;29 Suppl 3:S221–30.doi:10.1097/QAD.0000000000000924 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26562811
      OpenUrlPubMed
    18. ↵
      2. Tordoff DM ,
      3. Barbee LA ,
      4. Khosropour CM , et al
      . Derivation and validation of an HIV risk prediction score among gay, bisexual, and other men who have sex with men to inform PreP initiation in an STD clinic setting. J Acquir Immune Defic Syndr 2020;85:263–71.doi:10.1097/QAI.0000000000002438 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32658131
      OpenUrlCrossRefPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Handling editor Nigel Field

    • Twitter @henryjdevries

    • Correction notice This article has been corrected since it was first published. The open access licence has been updated to CC BY.

    • Contributors MSvdL, HJCdV and SN conceptualised and designed the study. SN performed the data analysis and interpretation of the data. MSvdL reviewed all analyses. SN drafted the manuscript. All authors critically revised the manuscript. MSvdL had full responsibility for the work, the conduct of the study, had access to the data, and controlled the decision to publish.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.