External evaluation carried out by:
This is an evaluation of the May 2022 digital campaign supporting the 5th Global Conference on the Elimination of Child Labour, centring on six days in Durban.
The conference boasted a number of ‘firsts’ - the first hybrid conference (i.e. face-to-face plus virtual), the first in Africa (of the previous four, two had been in Latin America and two in Europe) and the first to feature the active participation of children.
This campaign was undertaken for partners - ILO Fundamentals, Alliance 8.7, and South Africa’s Department of Labour. The ITCILO was in charge of the campaign design and implementation as well as overseeing the digital aspect of the physical conference.
The purpose of the evaluation was to provide the Centre with evidence of the relevance, validity, coherence, effectiveness, efficiency, impact and sustainability of its Communication and Advocacy activities, to assess which modalities offered by the Centre are most effective and efficient, and to extrapolate good practices, lessons learned and recommendations for the improvement or scale-up of Communication and Advocacy activities of the Centre.
The evaluation focused on the six main digital elements of the campaign: Website content and search, Newsletters, Media Relations, Video content, Social media, and Live Streaming.
The analysis of the campaign reviews the ISO standard for Project Management and the OECD DAC Network on Development Evaluation Criteria, and sets out a simplified assessment structure. A number of alignment checks are added of which the most important are Campaign alignment check and Best practices review.
To simplify this evaluation, these approaches were folded into a timeline of pre-project, project, and post-project activities and the utilized Quality Assurance approaches have been mapped against an uber-framework for digital campaigns to identify their different emphases and key questions on alignment added.
The Questions Grid
The central idea behind this evaluation is campaign alignment with ‘best practice’. A second core part of this evaluation is to look at the priority calls that were made and to assess whether different calls might have generated better results.
At a very basic level, a campaign’s success is determined by its sponsors: did they get what they paid for? In formal interviews the three sponsors gave these responses:
The ITCILO set out its main tactics in its Content Strategy document:
"Our efforts are therefore focused on awareness raising, converting commitments into action, and providing stakeholders with clear guidance on how to keep on implementing their pledges. In parallel, we are implementing a comprehensive content strategy that will engage the global community and wider audience"
The ITCILO committed to create, share and package content via:
3.2.1. The Website
ITCILO built a self-contained conference site, and devised the content. The design was eye-catching and friendly, and content was laid out so that it was easy for users to read. The traffic numbers underline that the site focused on the needs of conference delegates.
Newsletters were central to engagement with delegates in Durban and online. Segmentation was very strong with separate lists from the Alliance 8.7 distribution list for in-person delegates, online delegates, pledge-makers, workers group, government group, and employers group.
Pre-event research suggested a short and very clear presentation would be optimal.
Engagement was strong. There were 17 newsletters that went to in-person and online delegates and which were largely a blend of service messaging and event highlights. These generated an average opening rate of 43.7% and a click-through rate of 13.5%. Opening rates were about double the average for Non-profits and the Education & Training sector, click-through rates about 4 times.
The campaign boosted the subscriber list. By the time of the June 12 World Day newsletter, there were 9678 subscribers - nearly 7% more than just before the 5th Global Conference.
3.2.3. Media Relations Strategy
ITCILO had an ambitious media outreach strategy to target all types of media outlets however, budget cuts led to the media strategy being cut right back.
The media relations agency was present at the conference and created nine press releases. But only African media were targeted directly. In addition, only the final press release detailing the Durban Call for Action, was translated.
Despite this, there was extensive coverage - 3,000 mentions of the conference online from more than 300 news outlets. National coverage included the Guardian Nigeria, and regional coverage included CNBC Africa. In addition, SABC, the South African broadcaster carried live streams of the conference on its site.
3.2.4. Video Production
The campaign invested considerable effort into packaged video with more than 20 created and posted on social media with a wide variety of approaches.
There was selective use of Twitter, Facebook, Instagram, LinkedIn and YouTube accounts from ILO, Alliance 8.7 and ITCILO, and total views are estimated at about 20,000.
Video content was the second most highly rated digital output in the survey of users.
3.2.5. Live Streaming
Almost all the Durban sessions were live-streamed via Zoom for those who registered, and on ITCILO’s YouTube channel for everyone else. The live streams were also aggregated on the event website.
Sponsors were delighted with this core part of the hybrid event, with one remarking on how the streaming had been near flawless.
The sessions were high-level discussions, targeted at a specialist audience, and viewing figures of 2750 Zoom sessions and 4349 YouTube views suggest this content did not reach far beyond the hard-core Alliance 8.7 audience.
Of 200 online users surveyed, a high proportion - 76% - said the live-streams had been useful or extremely useful. The data suggests that the average virtual delegate viewed relatively few sessions.
3.2.6. Social Media
The primary tactic implemented was to post on Alliance 8.7 accounts on Twitter, Facebook, LinkedIn and Instagram, with some posting on ITCILO’s own account.
The biggest category of posts was ‘service messages’ - usually referring to live-streams. Quote cards were the next most common, and then the #RaiseYourHandForKids campaign.
The only paid promotion of the campaign was enormously significant and accounted for 70% of #RaiseYourHandForKids posts.
There are some clear patterns in the responses to the online survey that was launched in July 2023:
Were the Digital Campaign Goals Met? In terms of campaign goals, the answer to the question is only a qualified ‘yes’:
The tactics that worked best - live-streams, newsletters and #RaiseYourHandForKIds - tended to be the cheapest because they made very good use of what already exists. Meanwhile, the least effective at engagement took most of the resources.
Users and sponsors liked what they saw and ITCILO more than fulfilled the expectations. However, ITCILO is aiming higher in digital advocacy and wants to model best practices in all its work. On this measure, there is much to be done.
Campaign design was complicated by a tripartite stakeholder system and communications delays due to COVID-19. ITCILO saw its primary roles as facilitating the hybrid conference and making noise about the event but it did not receive well-defined KPIs.
All successful advocacy campaigns require superb mobilisation of networks, and this element got lost in the planning stage - one of several consequences of campaign sponsors not agreeing on a ‘theory of change’.
This was an enormously complex digital event, managed by a tight Turin team. The fact that everything that was promised got produced is a testament to the teamwork and esprit de corps created. However, some of the collective decision-making looks at odds with the campaign goals, and there were some departures from established best practice in digital advocacy.
These misalignments are mostly to do with limited resources. But some could have been avoided by a pre-project alignment check to ensure that audience goals line up with engagement tactics and costs.
The smartness of campaign design largely dictates how successful it will be. This one had many successes but there were some gaps.
While stakeholders and users expressed satisfaction with the outcomes, some modest tweaks to budget allocations would have magnified campaign reach.
It is essential to get campaign goals, target audiences, engagement tactics, and KPIs fully articulated and well aligned. The way to do this is to insist that sponsors discuss and agree on a theory of change.
No campaign ever has the resources it wants, and all successful ones show huge resourcefulness. The cost-benefit review reveals that building event-specific digital infrastructure was expensive and that the biggest engagement came from harnessing existing platforms.
If it’s worth investing in content, then it is worth optimising that content for users. Sessions were rich with insights, case studies and perspectives. People don’t have time to watch long sessions - make it easy by packaging up highlights for all audiences.
Change-seekers need to muster all the support they can. Such campaigns would need to feature greater promotion by the ILO, the involvement of adjacent organisations like UNICEF and FAO, and the possible support of the major social platforms.
The investment in Facebook promotion had a huge impact on #RaiseYourHandForKids. This underlines the harsh reality that social and search is becoming more ‘pay to play’. Commercial campaigns typically spend 20% on the production of content and 80% on promotion/advertising. That’s too rich for a purpose-driven campaign, but the 5-15% recommended by Getting attention is a useful benchmark.