Appendix 1: Methodology and Guidelines for the Impact Evaluation of Gender Projects

1This paper explains the different methodologies which will be implemented and the different guidelines set up depending on the targeted groups for carrying out the impact evaluation on gender projects.

2As mentioned in the ToR, the targeted groups will be diverse to guaranty a better understanding of the project’s impact within its environment. For each targeted group the methodology is defined, such as the number of people interviewed. The number of people to interview is the minimum required to carry out the impact evaluation.

3The following guidelines presented are only an indication of the different issues which will be raised during the interviews. As result of which, depending on the specific context new issues could be added to the predefined ones or some issues modified (in agreement with the project financier if this occurs before the field work). As mentioned in the ToR, the questions do not build up a questionnaire but a guideline. In addition to this, the guidelines are made up of open questions. The flexibility of this methodology will allow us to raise different issues—which maybe could not be raised if a structured interview had been the methodology used.

4Finally, the questions do not address awareness activities. Other subjects should be raised if this were the case and another methodology used too.

5Carrying out the interviews requires strong support from the local association: to select the female and male beneficiaries who should be interviewed as well as key local community representatives, as well as help in terms of logistics (accommodation and transportation).

I- Methodology and Guidelines for (partner) Associations

6The following questions are addressed to, at least, two association members or partners—the director and the active militant. The targeted person will be interviewed individually.

7The length of one interview should not exceed one hour and a half.

A- Setting up the Current Project

81. Explain the objective of the project, evaluation criteria… what do they think now about the project as it was designed?

92. How did they set up the project (assessment study before implementation) and why this way?

103. Did they face problems in implementing the activities (yes/no and why)? All the implementers were qualified (gender expertise, assessment…)?

114. Were some activities more successful than others? Why?

125. How to select the beneficiaries? How did they know about the project?

136. Women beneficiaries’ figures (age, marital status, level of education, number of children, urban/rural)? Changes in term of beneficiaries’ figures since the beginning of the activities?

B- Project Analysis

141. What are the signs of a successful/failed project?

152. Do they see their project as successful/failed? (to what extent?)

163. If they had to do the project in a different way, what would it be?

174. How to guarantee an improvement in the implementation of a project? (link with the qualification of the implementer?)

C- Future Project

181. If they had a budget for implementing a project what project would it be?

192. How to implement a new project? (How to deal with the local community, who would be the targeted group, how to select them…)

203. What do you think the targeted population needs most? What do you think, how could the need be met?

214. What is your analysis of gender relations in your region?

225. What are the strengths/threats to women’s situation in their area? And how to improve the current situation?

236. Do they see a change in the women and their position within the local community?

II- Methodology and Guidelines for Beneficiaries

24As mentioned in the ToR priority will be given to women beneficiaries, who are currently following up on the activities of the project or benefiting from the project or who have benefited from the project and who currently do not benefit from it. The first group of women should represent the large majority of interviewed women. In addition to which, men — if their intervention is relevant — will be interviewed and they will represent around 25% of the beneficiaries.

25Two methodologies have been defined for leading the interviews: individual interviews or group discussion. The same questions will be asked during the individual and group discussion. For each project 15 individual interviews will be carried out and 6 focus groups (made up of at least three participants). In total, 33 beneficiaries will be interviewed during the week.

26Interviews will be either carried out in the association/partner office or in the interviewees’ homes (villages). It will be envisioned to pick among the women attending the activities, in order to guaranty a certain neutrality in the choice of the interviewees.

27The length of one interview should not exceed one hour for individual interviews and two hours for group discussions.

A- Social Figures of the Interviewees (men/women)

28First the interviewed women (and men) figures will be set up: age, level of education, marital status, number of children, urban/rural.

B-Guidelines

29Women

30The Choice of the Association/Organization

311. How did she know/hear about the association and its activities?

322. Did she know another association working in this field before being involved in this association?

333. Why did she choose the association?

34Frequency of Activities

351. What are the activities she takes part in? Frequencies and changes (why she changed the frequencies)

362. Why did she choose this activity? And what did she expect to get out of the activities?

37Evaluation of the Activities

381. If she had to give a grade to the activities (from 0 to 5)?

392. What does she think about the activities (timetable, price, atmosphere—friends that follow the same activities—, location of the association…)? What are the positive/negative points?

403. How to improve the activities?

414. What would she like to learn (what kind of activities would she like to attend)?

42Impact of the Activities

431. Does she have problems attending the activities (family restrictions or responsibilities)?

442. How does she organize herself to attend the activities?

453. What does her family think about the activities? Does her family benefit from her involvement in the activities?

464. Did she benefit from the activities (precise—position within their family, in the social sphere, economy)?

475. Does she teach others what she learned during the activities? (how/reaction of the other?)

48Needs Assessment

491. If she had money to set up a project to improve her everyday situation what would she do? (What are the projects you would like to see implemented?

502. What does she think about women’s position within her community (treat/strength)? How would it be possible to improve the situation?

51Men

52It is impossible to set up guideline without having the designed project. The guideline will be set up according to each project and discussed with the financier.

III- Methodology and Guidelines for Key Representatives of the Local Community

535 key representatives of the local communities will be interviewed: Local Council members, Imams, Sheikhs, major association/cooperative/organization, key women. Individual or group discussion will be organized — depending on the context.

54The length of an interview should not exceed one hour.

55Interview guidelines

56Does he/she know about the association’s activities? Yes/No if yes how did he/she know about them and what does he/she know about the association? And what does he/she think about the implemented activities?

571. What is his/her analysis of the women’s situation in the local areas? What are the problems women and men face (women/men complain about) (strength/threat)?

582. Does he/she see a change in gender relations (new trend)?

593. What could the solution be to improve the women’s situation?

604. If he/she were in charge of a project what kind of a project would he/she implement? Why and how?

Appendix 2: Female Students Interviewed

Term of reference. Impact Evaluation of Gender Projects

I. Background

61The French Social Fund for Development (FSFD) is a French Embassy fund which notably supports women participation in development field. The FSFD I supported 18 projects and 5 new programs have been selected by the committee for the current program SFD II on the 24^th of February 2010. In order to improve the success of the FSFD projects an impact assessment is needed.

62The French Embassy has therefore decided to evaluate the impact of the two following projects:

• Support program to improve the education of the girls in Yemeni rural areas (Hodeidah Governorates), 2 phases/2005–2009, NGO Al-SADA-Women Society.
• Support program to promote the education of the girls (Hajjah Governorate), 3 phases/2005–2009, NGO SOUL.

63The ambition of the impact evaluation of gender projects is to learn from successful or failure factors of already implemented projects. The impact evaluation of gender projects will teach about the lessons learned. Sharing the different experiences in the implementation of the projects will participate to the improvement of the project implementation—on gender issue in Yemen.

II. Impact Evaluation of Projects

A- Impact Evaluation

64The main objective of the study is aiming at evaluating the impact of projects/services implemented:

• Evaluate the project impact on women beneficiaries (Knowledge acquired and use of knowledge, spreading knowledge, personnel impact, quality of the services provided by the project, frequency/quality of the services used…)
• Identify women needs (opinion expressed by themselves and by representatives of the local community) — in different fields as economical, social, legal, politics… — and order them into priorities.
• Assess the approaches, methodology of the local partners to implement the project in order to understand the factor of success or failure.

B- Methodology

a- Targeted projects/groups

• The projects selected for the impact study should offer a diversity of approaches, fields of development and sociopolitical community organizations.
• The targeted groups will intend to be comprehensive as encompassing: the associations/other implementers (local partners), key persons representative of the local community, women beneficiaries (current beneficiaries of the project or former) and men (according to the projects — as in health).
• The targeted groups could be extended to some members of the women family’s beneficiaries (husbands, parents…) according to the time dedicated to the field work. (Annexe 1: Methodology and Guidelines of Impact Evaluation of Gender Projects)

b- Questions

• The qualitative study will be carried out following the anthropological methodology — with guidelines (semi structured interviews) and not a questionnaire. The guidelines will be composed by open questions in addition to a rapid sociological figure (age, level of education, marital status… of the women beneficiaries). The guidelines will be adapted according to the projects assessed and the three main targeted groups.
• The guidelines addressed to the women beneficiaries (current or former) and men (according to the projects, as in health) will encompass the following main themes: acknowledgement and the choice of the association (or services provided), the reasons for choosing the project (or services), the women evaluation of the association activities, the impact of the activities/services they benefited from, the women willing to see other projects to be implemented (in each field).
• The guidelines addressed to the representatives of the community will encompass the following main themes: the analysis of the projects’ approach (methodology) implemented by the association, the projects the association would like to implement,
• The guidelines addressed to the associations will raise the following issues: the approach used to implement the project, their evaluation of the project, the factors of success or failure and the ways to improve the project.

C- Recommendations

65The comparative impact studies will give an overview about different factors leading to the failure or success of the project. The recommendations will shed a light on the lessons learned.

D- Estimated Time

66The estimated time to carry out the assessment for one project: 7 days of field work, an equal time for writing the report (from 7).

• The guidelines addressed to the representatives of the community will encompass the following main themes: the analysis of the projects’ approach (methodology) implemented by the association, the projects the association would like to implement,
• The guidelines addressed to the associations will raise the following issues: the approach used to implement the project, their evaluation of the project, the factors of success or failure and the ways to improve the project.

C- Recommendations

67The comparative impact studies will give an overview about different factors leading to the failure or success of the project. The recommendations will shed a light on the lessons learned.

D- Estimated Time

68The estimated time to carry out the assessment for one project: 7 days of field work, an equal time for writing the report (from 7).