Reflect is an approach to adult learning that aims to link literacy, empowerment and the achievement of rights. Reflect has won 4 UN Literacy Prizes in 2003, 2005, 2007 and 2008. It is presently used by 500 organisations in around 70 countries. The huge diversity of the approach has led to the need of systematising evaluation efforts. Together with this need, there was a strategic intention of pioneering participatory evaluation in the field of adult education.
In response, Reflect practitioners internationally have participated in the development of a new evaluation framework. To initiate the process, ActionAid developed a draft framework that was piloted in South Africa in May 2007. In October 2007, a workshop was held with UK-based research students to critique the early framework. This was followed by a one-week workshop co-hosted by DVV International, ActionAid and SARN (South Africa Reflect Network) in Cape Town in November 2007, where 39 Reflect practitioners from 20 countries convened to engage with and critique the initial framework. In April 2008 SARN took up coordination of the initiative and appointed a Reflect Evaluation Framework Coordinator.
As part of this evolving process, for six weeks during June and July 2008, 88 Reflect practitioners from 42 countries joined an online network and collaborated across four languages (English, French, Portuguese and Spanish) to discuss the evaluation of Reflect and to contribute ideas. A summary document of the online dialogue was then shared in English, French and Spanish with Reflect practitioners who were not available to take part.
The Cape Town workshop, 2007 Source: Cascant i Sempere
The framework was also informed by analysis of Reflect evaluations implemented in 2008 as well as two reviews of past Reflect evaluations carried out by Riddell (2001) and Duffy & Fransman (2008) on behalf of ActionAid. A “zero edition” of the framework (to be piloted and shared at CONFINTEA VI) is the latest output of this two-year collaborative process.
The framework provides a broad toolkit for designing and conducting an evaluation which might play a summative (assessing the programme), formative (informing better practice) and pedagogic role (since literacy and communication practices are often developed through the very process of participating in the evaluation process). It is also rooted in the principles of participation which suggest that meaningful and democratic involvement in an evaluation can enhance the ownership of programmes by participants, promoting sustainability and transparency. Individuals are able to reflect on their own learning experience at the same time as the circle as a whole reflects on the collective learning experience. The expectations of participants are therefore just as significant as the programme objectives of implementers and the broader social goals of civil society organisations, governments and donors; all of which should be taken into account by the evaluation.
After carefully explaining the concepts of literacy and evaluation and outlining the principles in which the framework is grounded, the framework then takes the form of a toolkit organised into 16 sections with 80 tools:
Crucially, the framework is not intended to be used as a blueprint but rather a set of guidelines and suggestions which should be adapted to suit a particular concept. A workshop to help pilot the framework in three countries is planned for later in the year.
The process of creating an evaluation framework can be considered to be a good practice for the following:
The on-line discussion on “Evaluating Reflect”, 2008
In the same line, the content of the framework can be considered to be a good practice for the following:
The Reflect Evaluation framework can be considered innovative for the following:
The Cape Town workshop, 2007 Source: Cascant i Sempere
1. It represents a pioneering effort to work in four languages simultaneously rather than working first in English and then translating it into other languages for non-English speaking participants to comment. This allowed, for instance for Anglophone, Francophone and Lusophone African countries to work together in the same effort as well as country representatives from Africa, Europe, Latin America and Asia, all at the same time.
2. It represents a pioneering effort to keep an international literacy community together (CIRAC) with an on-line resource center and constant information flows amongst them as well as its aim to link with other networks such as the African Platform, ICAE, etc…
The next steps of the framework project are as follows:
1. Dissemination, mainly through a workshop in the official programme at CONFINTEA VI in Dec. 2009
2. Training on the evaluation framework
3. Piloting and refinement of the framework
The partnership is presently seeking for more funding in order to achieve these three objectives.
The fish tool, South Africa. Source: SARN
Recommendation 1: To foster and finance a culture of evaluation learning within youth and adult education and literacy organisations.
In particular the following two aspects: the correct gathering of baseline data at the beginning of programmes and the sharing of evaluation results and learnings at the end of them. This includes the devolution of results from external consultants.
Recommendation 2: To support programmes that make an effort to go beyond external-based, donor-led evaluations as the only way of evaluating and that integrate a mixed / multiple evaluation format, through the inclusion of other evaluation methods such as peer reviews, on-going evaluations, internal evaluations and self-evaluations.
Reflect Evaluation in El Salvador, 2008
Recommendation 3: To promote innovative research on new and/or improved evaluation methods tailored for literacy and adult education. Methods could range from evaluating literacy levels as a continuum rather than a line to be passed, or for better evaluating concepts such as the literacy environment and the empowerment side of literacy and education.
The motivation tree tool, Angola. Source: dvv / AAEA
Previous recommendations are crucial because it is not a matter of financing more literacy and education programmes regardless their character istics but rather of financing those programmes that have evident developmental impact and those entities with the will to learn from their past actions, to develop the ownership of their evaluation process and that use evaluations as a literacy/education method in itself.
The self evaluation tool, DRC. Source: CEREBA/GOAL
You can search for articles in our article index (sorted by authors, issues, year, regions and countries). It also provides a full text search.
The journal Adult Education and Development is distributed free of charge in English, French and Spanish. If you wish to receive the journal, please subscribe here.