The TNTP Core Teaching Rubric uses assessments of student behavior, rather than teacher actions, to rate a teacher's instructional practices. In this report, RAND researchers assess whether the TNTP Core Teaching Rubric produces scores that are representative of teachers' overall instructional practices and whether raters' content expertise influence scores on TNTP Core. The report includes recommendations to improve the use of TNTP Core.

Validation Study of the TNTP Core Teaching Rubric

Andrew McEachin, Jonathan Schweig, Rachel Perera, Isaac M. Opper

ResearchPublished Nov 7, 2018

Nearly every public school teacher in the United States receives some form of feedback based on classroom observations, and these assessments are used by policymakers and administrators as indicators of teacher quality and input for professional development purposes. The TNTP Core Teaching Rubric, introduced in 2014, is unlike others in that it is scored based on student behavior rather than teacher actions. In this report, RAND researchers assess whether the TNTP Core Teaching Rubric produces scores that are representative of teachers' overall instructional practices and whether raters' content expertise influence scores on TNTP Core.

For this study, volunteer teachers of mathematics and English language arts (ELA) from more than 20 districts or charter school networks across eight states had their classroom instruction videotaped three times. This sample was complemented with a sample of approximately 100 4th-and 5th-grade teachers drawn from the Measures of Effective Teaching study for whom videotaped instruction was available. Raters trained in the TNTP Core Teaching Rubric then scored the videos from both samples. Analyzing the raters' scores, RAND researchers found a modest relationship between teachers' TNTP Core scores and student achievement gains in ELA, and no statistically or practically significant relationships between TNTP Core scores and math achievement gains. Further, raters struggled to agree on their judgments about the quality of a lesson, suggesting a considerable amount of uncertainty about the extent to which TNTP Core scores represent teachers' overall instructional practices. The report includes a number of recommendations to improve the use of TNTP Core.

Key Findings

  • Relationships between teachers' TNTP Core scores and student achievement gains are modest and vary by subject area.
  • Observers often disagree in their ratings of instructional practice based on the TNTP Core rubric, and these disagreements may be related to their content expertise.
  • There is a considerable amount of uncertainty about the extent to which TNTP Core scores represent teachers' overall instructional practices.

Recommendations

  • Consider setting higher standards for rater certification (specifically by requiring raters in training to have their scores align with those of master raters at a higher frequency).
  • Research has shown that raters tend to change their approach to scoring over time — a phenomenon known as rater drift. To mitigate this, consider using more frequent post-certification calibration and validation exercises during a rating period.
  • Keep in mind that high-quality evaluation and feedback require many observers with different backgrounds to rate many lessons.
  • Consider collecting additional sources of evidence that support claims about the quality of teacher practice, i.e., using TNTP Core as one measure in a multiple-measure system.

Topics

Document Details

Citation

Chicago Manual of Style

McEachin, Andrew, Jonathan Schweig, Rachel Perera, and Isaac M. Opper, Validation Study of the TNTP Core Teaching Rubric. Santa Monica, CA: RAND Corporation, 2018. https://www.rand.org/pubs/research_reports/RR2623.html.
BibTeX RIS

The research described in this report was prepared for TNTP and conducted by RAND Education.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.