In the 2020-21 biennium, the Centre has seen a massive shift from face-to-face training towards fully online learning, driven by the outbreak of the COVID-19 pandemic. Within two years, the number of enrolments per year in online training activities has increased five-fold.
To provide evidence of the relevance, validity of design, effectiveness, efficiency, impact and sustainability of its fully online training activities, and to derive recommendations for the improvement and further development of online training courses, the Centre has in mid 2021 commissioned an external evaluation. The evaluation criteria were based on the OECD DAC evaluation principles and involved desk research, an online survey of more than 9000 participants, interviews with participants, institutional clients and with staff of the Centre, and the elaboration of case studies.
The evaluation reviewed a sample of twenty training activities offered in 2020 and reaching more than 9000 participants. The sample included a variety of paid and free, open and tailor-made, tutor-supported and self-guided courses that took place using a diverse set of tools and platforms, including eCampus, Solicomm, webinars, and virtual reality.
The evaluation criteria were based on the OECD DAC evaluation principles, and the used methodology combined both quantitative and qualitative analysis of findings.
A significant number of 1.284 responses were collected from an online participants’ survey (Quantitative analysis) that also evaluated the validity of the training design to support a meaningful online learning experience using the Community of Inquiry (CoI) framework.
The model assumes that effective learning and engagement in online learning activities occurs within an online learning community through the interaction of social presence, cognitive presence, and teaching presence.
In-depth interviews were conducted with ITCILO’s staff members (27), institutional clients (2), and participants (7), in addition to a desk-based review of relevant documents, reports, courses design, evaluation, and knowledge acquisition results. Over and above, eCampus was visited to review participants’ engagement (Qualitative analysis).
Based on the interviews, three case studies that capture the positive impacts created by the Centre's online training activities were presented. Each case includes information about how participants or institutional beneficiaries made positive changes in their work experiences or institutional culture through learning new knowledge and skills from the Center's online training activities.
We are definitely staying with ITCILO, and the main reason is that they have a lot of in-house knowledge on how an employer organization works. Anybody can give a training on marketing, but ITCILO has this wealth of knowledge and their trainers are also people who are still working or who have recently worked at employers' organizations.
Attending the course, you gain the experience of knowing that you can do things remotely, you don't have to come to the office to work. The beauty about that particular course that you were able to exchange that information [with] others in other countries.When speaking with others, we're able to practice the knowledge of what they are currently doing, how they are coping, they have lots more than us
The Centre reached a wider and more diversified audience with online distance learning activities, especially, participants from middle-income countries who took advantage of digital learning solutions avoiding costs for travel and accommodation.
The report furnished the recommendations based on the three dimensions of the ITCILO’s strategy framework.
1. It is recommended that the Centre develop an operational plan on how to best reach their target groups in different regions with appropriate educational technologies and media to get the right mix of online training activities.
2. It is recommended that the Centre focus on the development of tutor-based distance learning that facilitates interaction between tutors and learners as well as among participants. Self-guided course content can be used in combination with tutor-guided instruction.
3. It is recommended that the Centre review and improve its technical support provisions, both processes and information, to help training participants smoothly join and navigate their online courses.
4. It is recommended that the Centre consider more student-centered evaluation methods such as a self-rating scale of knowledge application, participant panels, or self-reflective learning journals.
5. It is recommended that the Centre consider the development and use of Open Educational Resources and publish its training materials under a Creative Commons license that allows its users to retain, reuse, revise, remix, and redistribute.
6. It is recommended that the Centre expand its role to provide educational ‘consultation’ and online training packaging services, helping its partners to build their online training capacity as a knowledge hub.
7. It is recommended that the Centre develop a dual online training provision model—i) specialized long-term training courses and ii) general short-term training activities. The Center can consider re-structuring or re-packaging their online training activities with a programme or a degree perspective.
8. It is recommended that the Centre invest in its outreach strategies, thinking more about its future competitiveness after the COVID-19 pandemic when online training becomes more mainstream, and learners have more choices.
9. It is recommended that the Centre recognizes and rewards its staff’s hard work and dedication during the COVID-19. The Centre also needs to provide its staff with reflective learning opportunities, creating and nurturing a supportive learning culture across the units.
10. It is recommended that the Centre develop a systematic course design framework and an effective operational model, taking into account the full spectrum of target groups, content areas, technological tools, pedagogical methods—including corresponding instructional design templates.