Health authorities often respond to emerging issues with public awareness campaigns. However, the impact of these campaigns can be difficult to assess, unless appropriate indicators are defined and collected in advance.

Australia observed a marked increase in reports of HIV in the early 2000s. Victoria, the second most populous state in Australia, was the first state to show increased rates, from 2.9 per 100 000 in 1999 to 4.5 per 100 000 in 2002, with more than 80% of diagnoses associated with male to male sex.1 Between 1995 and 2003, there was also a steady increase in the number of reports of sexually transmissible infections (STIs) among men who have sex with men (MSM) in Victoria. Gonorrhoeae notifications among males (majority being MSM) increased threefold, from 341 to 1165, chlamydia notifications among males increased fivefold, from 13 172 to 6473 and infectious syphilis notifications among MSM increased by 13-fold, from 3 to 39.2 3

STI and HIV testing are important secondary prevention strategies. STI testing enables effective and timely treatment, thus reducing the development of complications and sequelae.4 HIV testing enables earlier diagnosis and timely intervention with antiretroviral treatments. People who test late in the course of HIV infection are not able to benefit fully from antiretroviral therapy and prophylaxis to prevent opportunistic infections and, thus, are more likely to progress to AIDS.5 Earlier HIV diagnosis also presents an opportunity to provide people with HIV with information and tools to prevent HIV transmission to others. A recent meta analysis showed the prevalence of high-risk sexual behaviour is reduced substantially after people become aware they are HIV positive.6

Despite HIV and STI testing being recommended at least annually for MSM, surveys have shown that only approximately 60% of MSM report having a HIV test within the previous 12 months and half having a STI test.7 In this context, health authorities recommended a social marketing campaign to increase testing. Media campaigns to promote HIV testing have been previously shown to have a significant initial impact on the uptake of HIV testing8 and social marketing has been shown recently to be an effective strategy at changing sexual or testing behaviour among MSM.9 10

Social marketing differs from other health education strategies in that it is based on commercial marketing techniques.11 Social marketing has a number of important key components which contribute to its success: branding (message appeals to the target population), segmentation (message takes into account the target population’s attitudes and beliefs), price (the social, psychological or physical cost the target population associates with the suggested health-changing behaviour), placement (ensuring the suggested health-changing behaviour can be undertaken at a time desirable to the target population) and promotion (media, print, radio, television, person-to-person methods, informed by the social behaviour of the target population).9 11

In order to evaluate the social marketing campaign, we assessed its impact by measuring the extent of any change in the uptake of testing for HIV and STIs during and subsequent to the campaign. This article reports the findings of the evaluation.

METHODS

The campaign

In March 2004, the Victorian AIDS Council launched a social marketing campaign entitled ‘Check-It-Out’ targeting a broad range of MSM including: young, gay community attached (GCA), non-gay community attached (NGCA) and culturally and linguistically diverse (CALD). The campaign’s aims were to increase HIV and STI testing, to increase regular HIV and STI testing and promote sexual health.

The Victorian Department of Human Services commissioned the Victorian AIDS Council (VAC) to manage the campaign and Convenience Advertising (www.conads.com) to develop focus group tests and publicise the messages during the campaign. The overall campaign budget for design and media placement was $A130 230.

Separate campaigns, with distinct strategies and budgetary allocations, were conducted for each subpopulation being targeted. Optimal use of the available total funds meant that campaign durations differed among the subpopulations. The campaign component targeting community attached MSM ran over 5 months with a total budget of $A51 300. There were 220 display points in six hotels, four nightclubs, two gyms, five cafes and 13 sex on premises venues (SOPVs) regularly attended by the gay community. These consisted of A4-sized posters (fig 1A,B) and takeaway cards (miniature versions of the poster). The key messages of the campaign posters were ‘Another excuse to drop your pants’ and ‘Let’s talk man to man’. One of the messages contained in the posters was ‘No matter where you’re from and what you’re into, if you’re having sex, you should have a regular sexual health check. Some STIs (sexually transmitted infections) don’t show symptoms so you could have one and not know it. To find out more, visit a doctor for a private and confidential check-up. For more information and referral about sexual health checks call AIDSLINE on 1800133392 or go to www.checkitout.net.au’ Advertisements were also placed in gay and lesbian publications and radio programmes targeting the gay community. In addition, a banner containing a link to the ‘Check-It-Out’ website was posted on a chat room (gaydar.com).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

(A) and (B) ‘Check-It-Out’ campaign posters.

The campaign component targeting non-community attached MSM and young MSM ran over 6 weeks with a total budget of $A49 680. There were a total of 203 display points in 21 educational venues across Victoria consisting of A4-sized posters (these were specific NGCA messages featuring a young man). One-hundred and sixty posters were also placed in 60 tram interiors for 4 weeks and 561×30-second advertisements were placed on regional radio for 2 weeks and in seven local newspapers for 2 weeks.

The campaign component targeting non-community MSM from CALD backgrounds ran over 6 weeks with a total budget of $A19 050. Eighty-two advertisements were placed on radio and in Chinese, Arabic, Greek and Italian publications.

A website (www.checkitout.net.au) was developed specifically for the campaign and provided information about HIV and a range of STIs, risks for HIV and STIs, indications for testing, the consequences of co-infection with STIs and HIV and the address and contact details of metropolitan and rural locations for a sexual health check-up. The website also cited the primary telephone counselling service AIDSLINE.

Evaluation of changes in testing uptake

Three different strategies were used to measure changes in testing uptake.

(i) The pilot programme of HIV sentinel surveillance

The sentinel system was implemented at five primary care clinics in Victoria in April 2004. Of the five clinics, three had patient populations that were primarily MSM and had been recommended as referral sites for testing in the ‘Check-It-Out’ campaign. Another of the sites was a regional sexual health centre, which was also recommended as referral site in the campaign. The methodology for the sentinel system has already been described in detail elsewhere.12 The primary indicator used to assess the impact of the campaign was the number of HIV tests recorded as being carried out at the five clinics. Linear regression was used to estimate whether there was an increase or decrease in the number of HIV tests conducted per month. Comparisons were made between the number of tests recorded during the ‘campaign period’ (1 April 2004, when the sentinel surveillance programme began, to 31 December 2004) and the ‘post-campaign period’ (1 January 2005 to 31 August 2005). The χ-square test was used to assess if there was a difference in distribution of specific characteristics of those tested (demographic, sexual behaviour and testing behaviour between the two periods.

(ii) Retrospective analysis of laboratory records

We analysed laboratory records of HIV and STI testing at four medical clinics, including the Melbourne-based, MSM-focused clinics that made up three of the five involved in the pilot phase of the sentinel clinic project. The fourth was a public sexual health clinic in Melbourne, also with a large MSM caseload. These four clinics are served by two pathology laboratories for HIV and STI testing. The laboratory data were analysed according to the ‘pre-campaign’ period (January to February 2004), the ‘campaign period’ (March to September 2004) and the ‘post-campaign period’ (October 2004 to December 2004). Testing data were analysed for men only, as sexual orientation was not available from the laboratory records. All analyses focused on the number of patients tested rather than the number of tests conducted.

The analysis had two components. The Kruskal–Wallis equality of populations rank test was used to compare the average number of monthly tests conducted during the ‘campaign period’ with the ‘pre-campaign’ and ‘post-campaign’ periods. Linear regression was used to estimate the number of HIV and STI tests conducted per month between 1 July 2002 and 31 December 2004 and assess whether there was a change in the monthly rate during the campaign.

(iii) Behavioural survey

The third source of information was the Melbourne Gay Community Periodic Survey (MGCPS),13 14 a component of a national survey that has been described in detail elsewhere.13 14 The MGCPS is conducted in February each year and recruits MSM cross-sectionally, predominantly though the main gay carnival in Victoria ‘Midsumma’ and a lower proportion from SOPVs and medical clinics.13 14 The short, self-administered survey captures information about sexual risk behaviour, drug use and testing patterns. The 2004 survey served as a campaign baseline with subsequent surveys in 2005 and 2006. The response rates were 58%, 66% and 69%, respectively. Only information from Victorian residents was included in the analysis (approximately 90% of the study participants each year). Changes in testing patterns over time among survey participants were explored using the χ-square test for trend.

All data were stored in an Access database and analyses were conducted in STATA version 9.15 A significance level of 0.05 was used for all statistical tests.

RESULTS

HIV sentinel surveillance

At the five sentinel clinics involved in HIV sentinel surveillance, a total of 4147 HIV tests were conducted among men between April 2004 and August 2005, 3435 (83%) among MSM. In the 17-month period an average of 202 HIV tests were conducted per month among MSM (range 173–238). There was no statistically significant increase or decrease in the number of HIV tests conducted per month between April 2004 and August 2005 or during or following the campaign.

The median age of MSM tested for HIV was 36 years (range 16–80), 38% of tests were conducted among MSM aged 30–39 years. The majority of MSM (77%) were born in Australia and 8% were from culturally and linguistic diverse backgrounds (born in Asia or Africa). About 40% of MSM reported having had HIV testing in the year prior to current testing. There was no statistical difference between time periods in the proportion of HIV tests conducted according to age group, region of birth of Africa and Asia and HIV testing history (table 1).

Laboratory data

Between 1 January 2004 and 31 December 2004, the number of HIV and STI tests conducted among males attending the four clinics was as follows: HIV (10 130), syphilis (10 768), gonorrhoea (11 583) and chlamydia (11 519). There was no significant difference in average monthly tests for HIV, syphilis, chlamydia or gonorrhoea tests in the campaign period, pre-campaign or post-campaign periods (table 2). From 2002 to 2004, there was no change in the monthly number of HIV tests in males (p = 0.48) but there was a statistically significant increase in the monthly number of tests for chlamydia (average increment of 6.4 per month, p<0.01), gonorrhoea (6.5 per month, p⩽0.01) and syphilis (4.0 per month, p⩽0.01). However, the rate of increase did not change during the campaign period (figure 2).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Monthly HIV, chlamydia, syphilis and gonorrhoea tests and among males attending four medical clinics 2002–4.

HIV, syphilis, chlamydia and gonorrhoea tests were most frequently conducted in males aged 30–39 years. There was a significant difference in the proportion of HIV and STI tests conducted according to age group, with a general decrease in the proportion of those aged 20–29 and 30–39 but an increase in the proportion of those aged 40 years and above, particularly among those tested for STIs, rather than HIV (table 3).

Behavioural survey: trends in HIV and STI testing between 2004 and 2006

In 2004, 2005 and 2006, approximately 60% of MSM reported having a HIV test in the last 12 months. There was no significant increase in this proportion over time (table 4). A breakdown of the frequency of reporting a previous HIV test in the last 12 months according to specific demographic groups and community attachment questions also showed no significant change over time.

In 2004 (pre-campaign), 2005 and 2006, there was a significant increasing trend in the proportion of men who reported having specific STI tests (anal swab, penile swab, throat swab or urine) in the past 12 months (table 4).

DISCUSSION

Based on data collected from a sentinel surveillance network of primary health clinics, analyses of testing data from laboratories and the results of repeat surveys of MSM conducted in Melbourne, there was no evidence that HIV testing rates increased following a health promotion campaign conducted in 2004. There was evidence from the laboratory reports of a steady increase in the levels of testing for STIs but the increase was already taking place prior to the campaign and could not be attributed to it.

In addition to showing no increase in the overall extent of HIV testing, the sentinel surveillance network showed no difference in the proportion of MSM reporting regular annual HIV testing during the campaign and post campaign and no differences in HIV testing levels within specific subgroups of clients attending the sentinel network sites, as defined by age group, country of birth and other factors. The annual behavioural surveys also showed that between 2004 and 2006 there was no significant increase in this overall proportion of MSM reporting having a HIV test in the last 12 months and no change within subgroups defined by age, community attachment or country of birth.

The steady increase in the amount of STI testing at the clinics detected through the laboratory reports began at least 2 years before the campaign and was not accelerated during the campaign. Over the time period, there was a steady increase in the amount of testing in all age groups, but the increase was greater in those age 40 years and above. Like the laboratory results, the behavioural survey did however show an increasing trend in STI testing over time, but although not shown here, survey reports show it began a few years before the campaign.7

The increasing trend in STI testing is likely to reflect the ongoing response to an outbreak of syphilis in MSM that was first detected around 2002 and led to a recommendations that clinicians provide more frequent opportunities for STI testing for men who were HIV negative at last test and in conjunction with viral load testing for those who were positive.16

The increase in STI tests conducted among those aged 40 years and above, seen in the laboratory data analysis, probably reflects the ageing HIV-positive MSM population, combined with the increased testing in this group. Two behavioural surveys among HIV-positive men in Melbourne show the median age has increased over time; from 36 to 40 years between 1998 and 2007 (R Thorpe, personal communication, 2007) and from 39 to 44 years between 1997 and 2006 (I Zablotska, personal communication, 2007).

The campaign’s failure to increase HIV or STI testing may have been due to its coverage. The campaign tried to target a diverse audience and had a limited budget for design and media placement; only 40% of the overall budget for design and media placement was spent on media targeting community-attached MSM who make up the majority (about 80%) of MSM diagnosed with HIV infection in Victoria. Alternatively the method may not have adequately resonated with the target population for reasons that would require qualitative methods to explore. For example, the messages may not have caught the attention of those who saw them, or if they were received, the messages may not have been interpreted by those targeted as sufficiently motivating to lead to a higher likelihood of undertaking testing for HIV and STIs.

It is important to consider whether there are any methodological reasons why it may have failed to detect a real increase. First, there may have been small increases in testing, which was beyond the sensitivity of system detection. We were unable to assess the extent of campaign recognition in the community and it is possible that the campaign did in fact increase testing in those who actually saw it. Second, the pilot phase of the sentinel system had only just been established in the period when the campaign was already under way, so it could not provide baseline data to determine if there had been any changes in testing patterns during the campaign compared with before the campaign. However the laboratory data which covered the period 2002–4 showed this was not the case. Third, the evaluation of laboratory testing data focused only on males and did not separately distinguish MSM. However, the participating clinics are recognised as having very high proportions of MSM in their caseloads, so that it is reasonable to assume that the analysis largely reflected testing patterns among this group. Finally, the behavioural surveys were based on self-report and may therefore be biased in ways that cannot be ascertained.

To provide a mechanism to evaluate future behavioural and testing interventions, a linked HIV, STI and hepatitis C sentinel surveillance was implemented in March 2006 at 17 primary care sites, five specific for MSM in Victoria.17 This system will provide demographic, risk behaviour and testing data for specific risk groups including MSM.

Testing is an important prevention strategy, and the evaluation showed the ‘Check it Out’ campaign did not result in an increase in HIV and STI testing among MSM. If social marketing methods are employed in future strategies, the design may need to be modified in consideration of the target group. For varied or large target groups, the campaign may require a larger budget to enable sustained and wider coverage or if funds are limited or a specific target group is the focus then a narrower campaign may be preferable. The results from this evaluation highlight the importance of using all available data sources to evaluate public health campaigns to assess their impact and to inform design and modifications to future campaigns in the event that no impact is identified.