In 2013, the Delaware Office of Early Learning contracted with the RAND Corporation to conduct an independent evaluation of Delaware Stars for Early Success, the state's quality rating and improvement system for early learning and care programs. The research evaluated the extent to which rating tiers reflected differences in the quality of home- and center-based programs. This final report summarizes all findings of the evaluation.

Evaluation of Delaware Stars for Early Success

Final Report

Lynn A. Karoly, Heather L. Schwartz, Claude Messan Setodji, Ann C. Haas

ResearchPublished Nov 7, 2016

In 2013, the Delaware Office of Early Learning contracted with the RAND Corporation to conduct an independent evaluation of Delaware Stars for Early Success, the state's quality rating and improvement system (QRIS) for early learning and care programs. The project was designed to provide objective and rigorous empirical evidence of the extent to which rating tiers reflected differences in the quality of home- and center-based programs and whether the system was operating effectively in terms of technical assistance, financial support, and other features. This final report summarizes all findings of the evaluation. The primary focus is on two questions: Do early care and education programs with higher ratings in the QRIS deliver higher-quality care and early learning than those with lower ratings? Do children in programs with higher ratings in the QRIS have better learning and developmental outcomes than children in programs with lower ratings? These questions are examined using data collected in 2014–2015 on program quality from a sample of Delaware center- and home-based providers, along with measures of learning for children enrolled in the sampled programs. The authors also report on results from a survey of the directors of the sampled providers. Other components of the evaluation are also addressed in this report, including findings regarding quality improvement supports, financial incentives, and other aspects of system performance, some of which were analyzed in more detail in two earlier reports. Based on the findings, recommendations are offered for improving the effectiveness of the QRIS.

Key Findings

Delaware Stars Ratings and Program Quality

  • Alternative measures of program quality increased modestly as Delaware Stars rating levels rose.
  • For the most part, alternative measures of program quality were not positively related to scores on the components that make up the Delaware Stars ratings.

Delaware Stars Ratings and Child Developmental Outcomes

  • Differences in children's development across rating tiers were generally small and statistically insignificant.
  • Children from low-income families did not experience differential learning in higher-rated programs when compared with lower-rated programs.
  • Two quality domains of Delaware Stars were predictive of modest differences in selected children's outcomes.
  • These findings for Delaware Stars are consistent with those from QRIS validation studies in other states.

Delaware Stars System Performance

  • Delaware Stars makes a substantial investment in technical assistance (TA), and TA is highly valued, although there may be room for improvement.
  • Financial incentives represent another area of significant investment for Delaware Stars, and directors view the funds as valuable for improving quality.
  • Low parental awareness of Delaware Stars limits value to providers from participating in Delaware Stars.
  • Programs face challenges in advancing through the rating tiers, but they are motivated to improve quality.

Recommendations

  • Learn from other QRIS validation studies. With nearly two dozen states involved in QRIS evaluations, the Delaware Office of Early Learning (OEL) should look beyond the findings of any one study for any particular QRIS and discern the broader findings with relevance for QRIS design and implementation.
  • Consider further refinements to the Delaware Stars rating structure. OEL should consider a simplified, streamlined rating system based on measures of the dimensions of quality that appear to matter most for achieving the goals of the QRIS, such as improvements in child developmental outcomes.
  • Strengthen quality improvement supports in Delaware Stars. With respect to TA in particular, there is scope for lowering TA caseloads, increasing TA training, and improving the guidance that TAs provide.
  • Strengthen the marketing of Delaware Stars to families, drawing on best-practice guidance in the field. Any marketing activities and their timing would need to account for any further planned modifications to Delaware Stars.
  • Enhance administrative data systems to support ongoing system monitoring and quality improvement. OEL should invest in a data manager to revise the Delaware Stars database both by reducing the number of data elements collected and by improving the accuracy of the data elements that remain.

Topics

Document Details

Citation

Chicago Manual of Style

Karoly, Lynn A., Heather L. Schwartz, Claude Messan Setodji, and Ann C. Haas, Evaluation of Delaware Stars for Early Success: Final Report. Santa Monica, CA: RAND Corporation, 2016. https://www.rand.org/pubs/research_reports/RR1426.html.
BibTeX RIS

The research described in this report was sponsored by the Delaware Office of Early Learning and conducted jointly in RAND Education and RAND Labor and Population.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.