NGOs around the world are conducting a variety of interventions to improve the well-being of vulnerable women and girls. However, without assessing and analyzing the impact on the target population there is no way of proving program effectiveness.
Recently, impact measuring has been gaining traction amongst researchers, NGOs, and funders, as a way to gauge whether or not programs are producing positive results by changing the status quo, and if so, by how much. In an interview with Forbes, Marc J. Epstein, co-author of the book Measuring and Improving Social Impacts: A Guide for Nonprofits, Companies, and Impact Investors noted that there is an increased interest among donors for greater accountability for their grants. Coupled with a need for internal monitoring and evaluation, impact measuring is critical to the success of any organization.
Girl Up Initiative Uganda (GUIU) understands the value of developing impact measuring tools, and is working on collecting accurate data on how its holistic educational program has impacted the lives of adolescent girls with both quantitative and qualitative data. The organization seeks information that reaches beyond basic data — such as how many schools are participating in the program — and assess how much progress the girls have made in terms of their increased (knowledge) on their human rights, how they perceive certain issues (attitude) and how they have translated this knowledge into changed (behavior).
In January 2015, the Girl Up team began to collect data on the lives, attitudes, and knowledge of the girls through a pre-training questionnaire. GUIU will follow-up the findings with an informal discussion as a mid-term evaluation, and a post-training questionnaire at the end of the year.
Here are some of the initial findings from GUIU’s pre-training questionnaire:
The girls spend an average of 4 hours a day on housework.
41% of girls think it is ok for a man to beat his wife if she misbehaves.
18% of the girls' mothers reached university education level.
The average age of the girls in the program is 12 years old.
54% of the girls were born in Kampala. The others were born in other districts or in Rwanda.
”It is vital that we begin a process of impact evaluation so we can prove the visible change we see everyday with concrete numbers and data.” Kimberly Wolf, Deputy Executive Director
These findings have provided GUIU with a baseline, while the mid-term and post-training evaluation will provide insight on the strengths and weaknesses of the program.
One of the many advantages of simple questionnaires is that they are low cost, and do not call for as many resources as randomized surveys— which is ideal for a small NGO like GUIU. However, they can also present a series of challenges, including language barriers and lying due to social desirability. Creating a network of organizations that develops a set or framework of sharable best practices may be the best way to get meaningful results and not replicate failed methods.
Several opinion pieces have argued that perhaps some NGOs should not conduct their own evaluation, as it is unlikely they have the technical skills and expertise to conduct a robust assessment. In a 2013 review, the Paul Hamlyn Foundation stated that 70% of evidence presented by charities ranked below what the foundation called “good.” Another solution is to partner with think tanks and organizations like the International Center for Research on Women (ICRW) who specialize in impact and evaluation research, and can guide NGOs in the right direction. However, an under-resourced NGO may not have the funds to contract an external consultant, and must therefore do their best with limited resources.
That is the approach GUIU is taking. With hundreds of free, downloadable manuals and guides on impact measurement now available online, it is more possible than ever to begin to take steps to measure impact. By taking this step forward, GUIU will better understand the effect of its work with girls so that it can prove the programs impact to its funders while also learning how to improve its approach to create more transformational change.