Evaluating Teach For Pakistan's Fellowship Program

Photo by Teach For Pakistan

On its website, Teach For All describes itself as a "global network of more than 60 independent, locally led organizations united by a commitment to developing collective leadership to ensure all children can fulfill their potential. Each network partner recruits and supports promising leaders to teach in their nations' under-resourced schools and communities. With this foundation, they work with others inside and outside of education towards a world where all children have the education, support, and opportunity to shape a better future."^[1] To assist Teach For All network partners in understanding their successes and challenge areas, RAND researchers conducted evaluations of Teach For All partner programs in two developing countries. The first evaluation examined Teach For Nigeria.^[2] In the second evaluation (the focus of this brief), RAND researchers conducted a one-year mixed-methods evaluation of the fellowship program of Teach For Pakistan (TFP). The goal of this quantitative and qualitative evaluation was to examine the effect of the TFP program on whole-child development, perceptions of teaching quality, and perceptions of the contributions of TFP Fellows to the school community.

How the Study Was Conducted

The study took place between October 2023 and May 2024. The quantitative study sample included 80 principals, 162 teachers, and approximately 4,899 students from 80 government or public schools in the Islamabad Capital Territory. Approximately one-half of these schools employed at least two TFP Fellows, and one-half were comparison schools with no TFP Fellows. The comparison schools were selected because of their similarities to the schools with TFP Fellows in terms of the student demographic composition and school size and also because of their geographic proximity to the treatment schools. Data included mathematics, science, and English assessment scores and student, teacher, and principal survey responses. By comparing the data from TFP Fellow with that of non-TFP teacher classrooms, the team was able to see what effects the TFP program had on students' academic achievement, social and emotional learning, teaching quality, the quality of the learning environment, and more.

The RAND team conducted qualitative data collection in 16 schools with TFP Fellows, including interviews with the fellows, interviews with school principals, focus groups with non-TFP teachers, and focus groups with a sample of parents whose children were taught by TFP Fellows. The RAND team also conducted brief interviews with a sample of students who were taught by TFP Fellows. The qualitative data gathered from these interviews and focus groups provided insights into how TFP Fellows were perceived to have (1) contributed to students' development, (2) provided classroom instruction and developed their classroom environment, and (3) contributed to the school community at large.

Readers should note several limitations while reviewing the findings. First, the quantitative sample is not fully representative of all TFP schools because it does not include schools in which TFP teachers teach grades other than four, six, and seven. Moreover, the schools in which the researchers conducted the qualitative study constitute a purposeful sample identified in collaboration with TFP; therefore, qualitative study findings may not generalize to all TFP schools. Second, both the quantitative and qualitative study components rely on students' self-reporting many outcomes, including teaching quality, perceptions of the school environment, and their own social and emotional learning; prior research has found various concerns with relying on student and teacher self-reports of these types of constructs. Finally, while the quantitative and qualitative studies are complementary in that they provide different stakeholders' perspectives of outcomes, the researchers expected there to be differences among these findings, and they were limited in the extent to which they could compare findings across the two sets of analyses. This is because the two studies rely on different samples and use different tools.

Findings

This section presents a summary of the effects of the TFP program on the focus areas named previously, and it features excerpts from students who described their experiences in TFP Fellows' classrooms.

Whole-Child Development

Student academic achievement improved more for students of TFP Fellows compared with students taught by non-TFP teachers. However, other measures of whole-child development, such as social and emotional learning, changed at similar rates for both TFP Fellows and non-TFP teachers. A comparison of TFP Fellows' and non-TFP teachers' student assessment scores showed that students taught by TFP Fellows performed better in science, mathematics, and English than students in the comparison group (Figure 1). The results for all three subjects were positive and statistically significant (p < 0.05). Specifically, students working with TFP Fellows scored higher than peers who were taught by non-TFP teachers by about 0.41 standard deviations in science, 0.26 standard deviations in mathematics, and 0.15 standard deviations in English. The effects on science and mathematics achievement are large-sized effects, and the effects on English are medium-sized effects.^[3]

Lab	Effect Size	Lower Bound	Upper Bound	P-Value	Significance	Type
Mathematics (n = 1,625)	0.261	0.162	0.361	0.000	***	Academic outcomes
English (n = 1,487)	0.147	0.007	0.287	0.040	*	Academic outcomes
Science (n = 1,008)	0.414	0.226	0.602	0.000	***	Academic outcomes
Empathy (n = 1,664)	-0.064	-0.137	0.010	0.089	none	SEL outcomes
Growth mindset (n = 1,546)	0.084	-0.006	0.174	0.068	none	SEL outcomes
Self-management (n = 1,645)	-0.038	-0.149	0.073	0.493	none	SEL outcomes
Self-efficacy (n = 1,565)	0.101	-0.005	0.207	0.062	none	SEL outcomes

NOTE: Each dot and whisker represent a single regression. Effect sizes (dots) represent standardized mean differences between treatment and comparison groups. Whiskers represent 95-percent confidence intervals for each estimate. All regressions are weighted by balancing weights. *** = p < 0.001, * = p < 0.050.

Interview and focus group participants viewed TFP as having a positive effect on student achievement. Specifically, educators, parents of children in schools with TFP Fellows, and students themselves noted an improved ability among students to speak English with greater fluency and confidence. They also noted improvements in vocabulary, spelling, and grammar skills that enabled students to read stories and write responses in English.

Interview and focus group participants noticed social and emotional growth in students of TFP Fellows. Although the quantitative data analysis indicated no meaningful differences in the development of students taught by TFP Fellows and non-TFP teachers in terms of empathy, growth mindset, self-management, and self-efficacy, in the qualitative analysis, principals, parents, and non-TFP teachers in schools with TFP Fellows discussed how TFP Fellows had contributed to students' social and emotional development. They perceived that students who used to give up easily were now choosing to struggle or persist through difficulty or set a goal to achieve next time. Multiple stakeholders perceived that TFP Fellows' students had developed a stronger sense of confidence, responsibility, and self-efficacy toward learning. Stakeholders saw these qualities manifesting in students who had become more confident communicators and courageous advocates who ask for support.

Teaching Approach and Classroom Environment

Survey data suggested that there was no difference in how students of TFP Fellows and non-TFP teachers perceived teaching quality or their learning environments. The research team first used student survey data to understand individual students' perceptions of teaching quality and the quality of their learning environments. The team then aggregated the student-level survey scores to assess classroom-level outcomes. This part of the study found no major differences in how students perceived the teaching quality or classrooms of TFP Fellows and non-TFP teachers.

Interview and focus group participants found that TFP Fellows taught for mastery and conceptual understanding. Principals, other teachers, students, and TFP Fellows highlighted how teaching at students' existing levels of fluency and teaching for mastery were key characteristics of the TFP Fellows' instructional approach. Fellows required students to demonstrate clear understanding of the learning content — beyond rote memorization — before moving forward, and the Fellows provided additional support, as needed, to reach this goal.

Interview and focus group participants also reported that TFP Fellows cultivated a classroom community characterized by care and collective responsibility. Stakeholders reported that TFP Fellows built a foundation of mutual trust and respect with their students by consistently showing warmth and affection. TFP Fellows also frequently used collaborative learning approaches to foster teamwork and a sense of responsibility for one another's learning.

Stakeholders also noted that TFP Fellows empowered students to be agents of their own learning and future. Fellows have supported students to become more critically self-aware about their areas of strength and improvement. Furthermore, Fellows have helped students develop a vision for themselves and work toward those goals. Fellows also empowered students by providing them opportunities to voice opinions and make decisions in the classroom.

School Community

Interview and focus group participants stated that TFP Fellows cultivated supportive relationships with parents. Principals, other teachers, and parents themselves reported that TFP Fellows actively engaged parents via in-school meetings and home visits. TFP Fellows' efforts to involve parents had a perceived impact beyond the direct relationships that they built with parents. By connecting with parents and inviting them into the school building, TFP Fellows helped to instill a sense of responsibility in parents and show them how they can engage with educators to support their children's academic learning and overall development.

Interviewees noted that TFP Fellows championed student-centered approaches to classroom management with other school staff. In their respective schools, TFP Fellows built collegial and collaborative relationships with most other teachers and their principals, and they leveraged such relationships in their efforts to shift other educators' beliefs and practices. This was especially true for the practice of using corporal punishment as the default way to manage student behavior. While some principals and non-TFP teachers were resistant to changing their practices, stakeholders reported that TFP Fellows explained the importance and impact of teaching with love and encouragement rather than harsh words and actions.

Recommendations

The findings from this study suggest how TFP can continue to grow to support the success of TFP Fellows.

TFP should continue to encourage its teachers to engage in key practices, such as fostering students' sense of self (e.g., grit and confidence), teaching students to collaborate, empowering students to lead, and providing supports to ensure that every student learns.

TFP should consider implementing targeted social and emotional learning interventions and curricula with explicit instruction on social and emotional learning to improve these skills. TFP can use recent research that reviews social and emotional learning interventions and curricula to find rigorous evidence about which programs improve specific skills.

TFP might consider providing supports to help TFP Fellows overcome resistance from other school staff to their teaching practices. The program might consider providing similar training to principals and non-TFP teachers in schools where they operate to ensure that the successful approaches are not hampered by the school administration.

TFP should consider using multiple modes to measure student outcomes to obtain a comprehensive understanding of students' progress. Surveys are a good tool but have limitations, especially when it comes to students self-reporting on their social and emotional learning. TFP leaders might wish to consider alternative ways to collect data, such as performance-based social and emotional measures or third-party independent observations.

Notes

• [1] Teach For All, "Learning & Insights Hub," webpage, undated. As of May 5, 2025: https://learning.teachforall.org
• [2] RAND Corporation, "Teach For Nigeria: Evaluating and Building on Success," RB-A1870-1, 2024. As of April 25, 2025: https://www.rand.org/pubs/research_briefs/RBA1870-1.html
• [3] The researchers used Matthew A. Kraft's benchmarks for effect sizes: Medium-sized effects are those that are between 0.05 and 0.20 standard deviations, and large-sized effects are those that are larger than 0.20 standard deviations. See Matthew A. Kraft, "Interpreting Effect Sizes of Education Interventions," Educational Researcher, Vol. 49, No. 4, May 2020.

Available for Download

Download PDF

Topics

• Pakistan
• Social and Emotional Learning
• Teacher Training