Article Text

PDF

A meta-analysis of the effect of new-media interventions on sexual-health behaviours
  1. Rosie Swanton1,
  2. Vanessa Allom2,
  3. Barbara Mullan1,2
  1. 1School of Psychology, University of Sydney, Sydney, Australia
  2. 2School of Psychology and Speech Pathology, Curtin University, Perth, Australia
  1. Correspondence to Professor Barbara Mullan, Health Psychology & Behavioural Medicine Research Group, School of Psychology & Speech Pathology, Curtin University, GPO Box U1987, Perth, WA 6845, Australia; barbara.mullan{at}curtin.edu.au

Abstract

Background and objectives Direct access to individuals in non-intrusive ways, as well as the technical abilities of new-media to provide tailored information in relatively inexpensive ways, creates a unique opportunity for the delivery of health-related information. The aim of the present research was to examine the effect that new-media-based sexual-health interventions have on sexual-health behaviours in non-clinical populations and to determine the factors that moderate the effect of technology-based sexual-health interventions on sexual-health behaviours.

Data sources A systematic literature search of the following databases was conducted: MEDLINE, psycINFO, Global health, and EMBASE, using terms that captured three subject areas—‘Sexual-health’, ‘New Technology’ and ‘Intervention’.

Study eligibility criteria Randomised controlled trial, or a quasi-experiment; delivered exclusively via new-media; included sexual risk behaviour change as an outcome measure and delivered to non-clinical groups.

Results Twelve studies tested the effect of new-media interventions on condom use, whereas nine tested the effect on sexually transmitted disease testing. Results indicated that new-media interventions led to significant increases in both outcomes; however, these effects were not homogeneous. Moderation analyses revealed that interactivity of the intervention, target population and study design influenced the efficacy of interventions on both sexual-health outcomes, whereas intervention duration influenced sexually transmitted infection testing.

Conclusions and implications Interventions aiming to improve condom use are more successful when an interactive component is used. Further research needs to be conducted to reach specific at-risk populations.

Statistics from Altmetric.com

Introduction

New-media technologies have resulted in increased communication and access to information and have significantly changed the way individuals interact.1 New-media refers to interactive forms of communication that allow individuals to create, connect and collaborate. Examples of new-media include websites, social networking sites (such as Facebook), email, text messaging, smart phone apps and chat-rooms.2 ,3 New-media does not include television programmes, feature films or paper-based publications, unless these contain technologies that allow digital interactivity.4 Over 95% of young people in Australia use the internet, and more than half have access to the internet on their phone.5 Direct access to individuals in non-intrusive ways,6 along with the ability of new-media to provide tailored information in relatively inexpensive ways,7 ,8 creates a unique opportunity for health interventions.

This may be particularly useful in the area of sexual health. Regular use of new-media technology is high in groups with higher rates of both risky sexual-health behaviour and sexually transmitted infections (STI), such as men who have sex with men (MSM) and young people.9 ,10 Access to traditional sexual-health education and treatment centres is also disproportionately low in some high-risk groups.11 There are many examples of new-media being used for sexual health, particularly in the area of health promotion12 and personalised health information.13

The use of new-media in health-related interventions14 ,15 and clinical populations16 has been reviewed previously. Moderate effects were reported, particularly in interventions in which behaviour change techniques were incorporated.14 Guse et al2 systematically reviewed new-media based interventions and adolescent sexual health and concluded that more research was necessary to better understand the effect of new-media interventions on sexual health.2 While this review was informative, it neither meta-analysed the data nor focussed specifically on behavioural outcomes, such as condom use and STI screening. In addition, since publication, there have been a number of new studies.

Aims

The current meta-analysis had two main purposes: to examine the size and consistency of the effect of new-media based sexual-health interventions on sexual-health behaviours in non-clinical populations and to determine the factors that moderate this effect.

Method

Search strategy

A systematic literature search was conducted in MEDLINE, psycINFO, Global health and EMBASE in September 2013 and updated in September 2014. A list of subject headings and keywords used is presented in online supplementary table S1. The reference lists of relevant reviews were examined,15 ,17 and any papers that were not captured in the literature search were subsequently included. The preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines were followed.18

Inclusion and exclusion criteria

Studies were eligible for inclusion if they were (a) a randomised control trial (RCT), or a quasi-experiment; (b) delivered exclusively via new-media; (c) included sexual risk behaviour change as an outcome measure; (d) delivered to non-clinical groups and (e) reported in English.

Selection of studies

Studies were assessed at the title and abstract for full text reading. Assessment was completed by the first author, and quality assured by the third. Agreement about inclusion was high (99.3%, κ=0.897) and disagreement was resolved through discussion.

Data extraction

Selected studies were grouped according to type of behaviour. Data extracted included type of new-media used, interactivity of intervention, duration of intervention, target population, study design (RCT vs cohort) and attrition rate.

Analytical strategy

Effect sizes were calculated from proportions of individuals demonstrating positive sexual-health behaviours, or mean number of occasions engaging in sexual-health behaviour, in intervention and comparison groups. If no control group was included, preintervention scores were used as the comparison group. Behavioural outcomes were analysed separately. Authors were contacted to obtain additional data; however, if data were not provided the study was excluded from the meta-analysis.

Comprehensive Meta-Analysis V.2.019 was used for calculating effect sizes, conducting moderation analyses and examining publication bias using the Fail-safe N20 and Duval and Tweedie's Trim and Fill method.21 A random effects model was used in all analyses to account for the possibility that some studies may have been missed in the search22 and because some sample groups were reported as heterogeneous.23 The effect size metric employed was the OR, which represents the likelihood of performance of sexual-health behaviours given participation in the intervention where values >1 indicate that the intervention resulted in greater likelihood of performing the behaviour. For all effect sizes, 95% CIs were also calculated. Both Q and I2 statistics were used to explore heterogeneity. Q, when statistically significant suggested heterogeneity. An I2 of up to 25% indicated low heterogeneity, up to 50% suggested moderate heterogeneity and up to 75% or higher indicated high heterogeneity.24

A moderator analysis was conducted in a mixed-effects model25 (see online supplementary material for a detailed description of this technique). The first and second author coded all moderators. Studies were coded according to whether the intervention included an interactive component, type of population targeted, the duration of the intervention and design of the intervention. If an interactive component was included in the intervention, such as personalised emails,26 this was coded as interactive, whereas if the material was static, such as watching an online video27 this was coded as static. Target populations were coded as MSM, young people or female teenagers. Duration of intervention was coded as single session, ≤6 months or >6 months. Design of the study was coded as RCT, in which a control group was used and the effect size reflects differences between conditions at follow-up, controlling for baseline data; or cohort, in which no control group was used and effect sizes reflect differences from baseline to follow-up.

Quality assessment

All included studies were assessed against the Effective Public Health Practice Project (EPHPP)28 assessment tool for quantitative studies.

Results

Study selection

In total, 3228 articles were identified for screening. A full text search was conducted on 67 papers. In total, 18 papers met the selection criteria; however, data from three papers could not be obtained from the corresponding authors,7 ,29 ,30 and thus only 15 papers were included in the meta-analysis. The selection process, including reasons for exclusion at each screening point, is depicted in figure 1.

Figure 1

PRISMA study selection flow chart.

Study characteristics

The behavioural outcomes targeted were condom use7 ,29–36 and STI testing37–40 with five studies reporting both.41–45 There was a diverse range of new-media types implemented, including websites,7 ,30 ,31 ,33 ,34 ,37 ,45 mobile text messages, social media (ie, Facebook),32 ,35 chat room and online video,38 and one study using several conditions including an online video and a website.43 To maintain independence of effect sizes, only the comparison between the website and control condition was included in the meta-analysis. In addition, some studies included two techniques: email and website, and email and mobile text messaging. As there was little consistency in techniques, we were unable to examine type of new-media as a moderator. Further details about study characteristics can be seen in table 1.

Table 1

Characteristics of included studies

Quality assessment

Study design and quality ranged across the studies. Nearly all studies received a ‘weak’ score from the EPHPP assessment, and this was largely due to attrition rates, lack of blinding/allocation concealment in RCT studies or non-validated data collection tools. Non-representative samples were also a concern, most likely because of online recruitment.

Meta-analysis

Condom use and new-media interventions

The effect of new-media interventions on condom use was tested in 12 studies. The overall OR for condom use and new-media-based sexual-health interventions indicated that participants in the intervention groups had a significantly higher rate of condom use than in the comparison groups (OR=1.39, 95% CI 1.06 to 1.83, p=0.02). However, the effect exhibited significant heterogeneity, Q(11)=48.29, p<0.01, I2=77.22%, indicating that further factors may account for the variability in the effect sizes.

The Fail-safe N for the included studies was 58, indicating that 58 non-significant studies would need to be located and included for the overall effect of new-media interventions on condom use to be non-significant. Using Duval and Tweedie's Trim and Fill method to account for small study bias, it was estimated that four studies were missing to the left of the mean, indicating that potentially four studies with null effects exist that were not included in the meta-analysis. Accounting for the potentially missing studies, the new estimate was no longer statistically significant (OR=1.07, 95% CI 0.79 to 1.46).

Moderation analyses revealed that the effect of new-media interventions on condom use differed according to the interactivity of the new-media technique used. It appeared that interventions using interactive components yielded significant effects (OR=1.79, 95% CI 1.15 to 2.77, p=0.01); however, this effect was significantly heterogeneous, Q(6)=28.03, p<0.01, I2=78.59%, suggesting that further factors may influence the size of the effect. Interventions using static content did not yield significant effects, and the effect size was homogeneous, suggesting that this effect truly does not exist (see table 2).

Table 2

Moderator analysis of the effect of new-media interventions on condom use

Interventions that targeted teenage women yielded significant effect sizes (OR=2.04, 95% CI 1.04 to 4.00, p=0.04), and this effect was homogeneous. Interventions targeting young people or MSM did not yield significant effect sizes; however, both effect sizes were heterogeneous, Q(6)=17.29, p<0.01, I2=65.30%; Q(2)=25.72, p<0.01, I2=92.22%, respectively.

Interventions in which an RCT design was used did not yield significant effect sizes, and this effect was homogeneous, whereas interventions that used a cohort design did yield significant, but heterogeneous effects (OR=1.98, 95% CI 1.15 to 3.43, p=0.01); Q(2)=16.02, p<0.01, I2=87.51%.

Intervention duration and attrition rate did not influence the size of the effect on condom use.

STI testing and new-media interventions

Nine studies explored the impact of new-media interventions on STI testing. Overall, rates of STI testing in the intervention groups were significantly higher than in the comparison groups (OR=1.51, 95% CI 1.18 to 1.95, p<0.01); however, there was significant heterogeneity in effect sizes, Q(8)=25.87, p<0.01, I2=69.07%. The Fail-safe N for the included studies was 71, indicating that 71 ‘null’ studies would need to be located and included for the overall effect of new-media interventions on condom use to be non-significant. Using Duval and Tweedie's Trim and Fill method to account for small study bias, it was estimated that no studies were missing to the left of the mean, indicating no small study bias.

Moderation analyses revealed that the effect of new-media intervention on STI testing differed according to the interactivity of the new-media used. It appeared that interventions that used interactive content yielded significant homogenous effects (OR=1.52, 95% CI 1.27 to 1.83, p<0.01), whereas those using static content yielded non-significant but heterogeneous effects, Q(4)=20.94, p<0.01, I2=85.68% (see table 3).

Table 3

Moderator analysis of the effect of new-media interventions on STI testing

Interventions targeting young adults yielded significant but heterogeneous effect sizes (OR=1.54, 95% CI 1.09 to 2.17, p=0.02); Q(5)=21.13, p<0.01, I2=76.33%. Interventions targeting MSM produced non-significant and homogenous effect sizes.

Interventions using a single session were not significant (OR=1.06, 95% CI 0.87 to 1.30, p=0.58), and this effect was homogeneous. Interventions that were up to 6 months in length or more than 6 months both yielded significant homogeneous effects (OR=1.64, 95% CI 1.36 to 1.98, p<0.01; OR=2.17, 95% CI 1.36 to 3.47, p<0.01, respectively). Both longer interventions were significantly more effective than a single session as the CIs of the longer interventions did not overlap with that of the single session.

Interventions in which an RCT design was used yielded marginally significant effect sizes; however, this effect was heterogeneous (OR=1.56, 95% CI 1.01 to 2.40, p=0.05); Q(5)=14.33, p=0.01, I2=65.12%. Interventions that used a cohort design did yield significant, but heterogeneous effects (OR=1.46, 95% CI 1.04 to 2.06, p=0.03); Q(2)=11.41, p<0.01, I2=82.47%.

The effect of new-media interventions on STI testing did not differ according to the attrition rate.

Discussion

This meta-analysis investigated the effectiveness of new-media-based interventions on condom use and STI testing. New-media interventions appeared to improve sexual-health behaviours; however, the size of these effects varied considerably across several dimensions including interactivity of new-media used, target population and study design.

Interventions administered using interactive components were shown to be more effective in improving both STI testing and condom use. Considering the highly interactive and engaging nature of new-media platforms,46 the direction of these results is not surprising, and a recommendation arising from these results is that all future new-media-based interventions need to include a level of interactivity or personalisation of content for maximum engagement and results. However, further research is needed to establish the level of interactivity needed for effectiveness and to determine which platforms (if any) may be more appropriate for static, broadcast content by implementing an intervention that systematically varies the level of interactivity.

The size and consistency of the effect of new-media on sexual-health outcomes differed according to the population targeted. Specifically, the results revealed that interventions targeting condom use were more successful in women and less successful in other groups including MSM and young people in general. In terms of STI testing, interventions targeting young people were more effective than those targeting MSM. It appears that new-media interventions are not successful at changing the behaviour of MSM. This is perhaps reflective of male populations being less likely to engage with health messages or take proactive steps for their healthcare.47 Alternatively, it may be the case that MSM groups were not effectively engaged by the intervention strategy employed. This indicates a need for future research to develop and implement interventions that are effective within this high-risk population.

New-media trends change at a rapid rate,46 and to ensure that the most effective communication approaches are being used, end-users of the platforms being implemented need to be engaged and consulted during the design and delivery of new-media-based interventions. It is important for intervention designers to understand that new-media is an entirely new method for engaging with population groups. Algorithms for content delivery on social media platforms favour popular content with which users connect and engage.48 Researchers must, therefore, design their intervention in ways that reflect the use of these platforms, rather than attempting to adjust the use of platforms for delivery of their predetermined health message. Establishing practice guidelines for new-media could help with consistency around recruitment and intervention delivery, as well as provide a helpful space for more immediate ‘work in progress learning’ about the practical implementation and execution of new-media-based interventions.

Study design also influenced the size and consistency of new-media interventions on both outcomes. For condom use and STI testing, effects appeared to be larger when a cohort study design was used. This design does not use a control group, and as such, effects drawn from this literature may be overestimated. Control group statistics were difficult to capture in many of the analysed interventions, as it was difficult to manage exposure versus non-exposure in new-media environments. Recent developments in new-media analytics, such as A/B testing options within social media platforms like Facebook and Twitter,49 would make it easier to categorically target some groups and exclude others, making management of exposure and follow-up with control and intervention groups easier. Thus, it is suggested that future research uses these tools rather than using independent tools to track participation.

For STI use, the length of the intervention appeared to influence the effectiveness of new-media interventions. Those using a single session were found to be less effective than those using longer interventions, and this effect was homogeneous. Therefore, interventions using only a single session are not recommended to improve rates of STI testing. The effectiveness of interventions targeting condom use did not appear to be influenced by study duration. However, these effects were heterogeneous, suggesting that other factors may be unaccounted for. Therefore, future studies should systematically investigate how study duration influences condom use to more effectively change this behaviour.

Attrition was generally very high across the 15 studies reviewed. Although no significant differences between studies according to attrition rate were found, one study reported much higher rates of STI testing in the intervention group when engagement with the intervention was controlled for.39 It is possible that the intervention itself was not ineffective and that it is more effective in specific groups or that the engagement strategy was unsuccessful. Mevissen et al45 also found that tailored interventions were more effective than generic interventions, and three studies reported a difficulty engaging, or had a high drop-out rate, of minority groups.32–34 This reflects previous results regarding the population that was targeted and reinforces the need for formative research to be conducted to ensure that the intervention reaches, and suitably engages, the target population.50 Thus, future studies should examine how a more targeted approach could improve retention and outcomes for new-media-based sexual-health interventions.

Strengths and limitations

The current meta-analysis is the first to look explicitly at sexual risk behaviours and whether new-media-based interventions can improve engagement in these behaviours. PRISMA guidelines and a random effects model were used to control for study selection and inclusion bias, but it is possible that not all relevant literature has been included. It was also not possible to include some studies in the meta-analysis due to incomplete data, and this could have changed the outcome of the analysis. Furthermore, the effect of new-media on condom use appeared to be influenced by small study bias. As such these results need to be taken with caution.

Conclusion

New-media-based sexual-health interventions are a potentially effective way to increase condom use or STI testing. Specifically, new-media interventions appeared to be effective when interactive formats were used and less effective when MSM populations were targeted. Future studies should look at developing more targeted interventions, based on formative research to effectively change behaviour by reaching the desired population and minimising attrition.

Key messages

  • New-media interventions appear to be effective at increasing engagement in sexual-health behaviours including condom use and sexually transmitted infection testing.

  • New-media interventions were more effective when an interactive component was included.

  • Interventions targeting condom use appear to be more effective within women compared with those in young adults generally or in men who have sex with men.

  • Interventions targeting sexually transmitted infection testing appear to be more effective in young people generally compared with those in men who have sex with men and are not effective if only a single session is used.

References

View Abstract
  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

Footnotes

  • Handling editor Jackie A Cassell

  • Contributors RS and BM conceptualised the review and conducted the first search. RM extracted the data; VA conducted the analysis and interpretation of results. All authors contributed to writing the manuscript.

  • Competing interests None.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.