Leading Indicators for Quality Programs
Afterschool programs are often awash with data, but access to that information seldom creates action that promotes program success and improvement. An overarching framework to streamline data collection and structure the information in a meaningful way provides programs a frame of reference for important decisions about program implementation.
The Leading Indicators are a comprehensive data package designed to provide out-of-school time stakeholders with powerful information that inspires action.
Imagine a data system that provides information that can be used equally by youth workers, program administrators, funders, and other stakeholders. The Leading Indicators are a compilation of composite measures consisting of program quality data derived from multiple sources, and organized into overarching contexts. The framework presented here summarizes the contexts, indicators, and data sources for the Leading Indicators.
With the Leading Indicators, policymakers are enabled to make research-based decisions for allocation of supports and additional resources. At the organizational level, the Leading Indicators facilitate a process to monitor the progress of afterschool programs and demonstrate their effectiveness in serving youth. Managers of afterschool programs use the Leading Indicators to set and modify objectives and goals to promote positive outcomes for youth. The Leading Indicators can be used for evaluation purposes[i] and have been implemented in statewide 21st Century Community Learning Center systems.
The Leading Indicators are an extension of the Youth Program Quality Intervention.
The Leading Indicators were specifically designed to be used in collaboration with Coaching for Continuous Improvement[ii] as an extension of the Youth Program Quality Assessment (YPQI)[iii]. The YPQI is a data-driven continuous improvement model for afterschool systems, consisting of four continuous improvement practices that have been found to lead to improved program quality. These site practices are: quality assessment, improvement planning, coaching by site managers during staff instruction, and staff attendance at targeted trainings for instructional skill building.
The YPQI was designed to produce the maximum impact for the minimum investment by individual sites; system leaders that wish to deepen the level of support for continuous improvement and to expand the purpose of the intervention from site-level improvement to system-level evaluation, may wish to engage in Coaching for Continuous Improvement or the Leading Indicators as an extension of the YPQI. These extensions come in the form of data, technology, and targeted supports.
Creating a sustainable evaluation system using the Leading Indicators requires thoughtful consideration, planning, and professional development.
The Weikart Center has developed a series of supports to guide the implementation of such a system:
- Consultation for system design. Work directly with experienced Weikart Center staff members to design a performance management and evaluation system that will have the greatest impact in your organization. Examine cost and budget implications. Clarify your evaluation goals and data collection interests, so that the system can be created to support your efforts.
- Data Collection Support and Creation of Leading Indicator Reports. The Weikart Center will handle all of the data collection through online and hardcopy surveys. Survey data is collected once annually. Youth Program Quality Assessment data is collected from participants through the Online Scores Reporter. Additional demographic or enrollment data is collected via already existing data management systems whenever possible (e.g., PPICS or EZReports). The Weikart Center will create and deliver reports to participating programs.
- Training on interpretation of the Leading Indicator Reports. A guided overview of the report and how to use it is provided in webinar format, with follow-up technical assistance available for those with additional questions.
- Planning with Data Training. A full-day training provides an in-depth look at individual reports, and guides program managers and staff in the creation Program Improvement Plans based on the data in the Leading Indicator reports.
- Technical Assistance and support. Weikart Center staff work individually with leaders following training, to provide technical assistance tailored to meet the needs of individual organizations. Weikart Center staff support the development of new structures and strategies that help create a culture of continuous improvement within the organization. Technical Assistance may take many forms – from providing additional workshops for staff to supporting leaders in making informed decisions.
[i] It is important to keep in mind that the Leading Indicators are not designed to provide network leaders with information about individual programs that will be used to evaluate compliance, assess impact, or rank programs in terms of performance. The purpose of the Leading Indicators is to support program improvement efforts at the program and site-level.
[ii] Coaches provide individual support to program managers to strengthen the implementation of YPQI practices in their programs. Coaches can also provide Technical Assistance to a large audience of sites, to support the Assess-Plan-Improve sequence.
[iii] Smith, C., Akiva, T., Sugar, S., Lo, Y. J., Frank, K. A., Peck, S. C., Cortina, K. S., & Devaney, T. (2012). Continuous quality improvement in afterschool settings: Impact findings from the Youth Program Quality Intervention study Executive Summary. Washington, DC: The Forum for Youth Investment. For more information, visit www.cypq.org/ypqi.
|Sample Leading Indicators Report.pdf||856.73 KB|