SUBJECT: Local Evaluation
Federal Statutory Reference:
No Child Left Behind, TITLE I, PART B, SUBPART 3, SECTION 1235(15)
Each program assisted under this subpart shall –
(15) provide for an independent evaluation of the program, to be used for program improvement.
Federal Guidance Reference:
The paragraph numbered 15 in the Program Elements section of the federal guidance reads as follows:
"Local Independent Evaluation [Section 1235(15)]. Each project must provide for an independent evaluation of the program that it will use for program improvement. The independent evaluation generally is conducted annually and provides critical data and information to the local program on the performance of families as well as on the quality of the implementation of the core components and program elements. The project must use the data and information for participant progress monitoring and continuous program improvement to lead to better participant outcomes."
The below paragraph is excerpted from the section of the federal guidance titled National Evaluation, Local Evaluation, and Indicators of Program Quality.
Local Evaluation
"Each Even Start project is required to conduct an independent evaluation of the program to be used for program improvement. These evaluations provide local projects with critical information on areas of strength and weakness and are used to improve program services and participant outcomes. These evaluations also provide local projects, States, the Department, and the Congress with objective data about the activities and services provided by the project, the participants served, the retention rates of those participants and the achievement of the families in the project."
New York State Policy:
Even Start Family Literacy Partnerships must provide for an annual, independent local evaluation of their program and must use the results of the local evaluation to improve the quality of the program and services provided to families, leading to better participant outcomes. An independent evaluator may not be a member of the program implementation staff, or otherwise employed by the section of the partnership agencies with direct responsibility for the Even Start program. The local evaluation is objective and provides information about the performance and effectiveness of the Even Start program. The local evaluator is a critical member of the Even Start team and supports the local program in the process of continuous improvement through evaluation activities. The local evaluator should have a background in at least one of the component areas of family literacy and expertise in evaluation skills, e.g., measurement, data analysis, data interpretation. To support this process, local evaluators are required to participate in annual professional development coordinated and scheduled by the Department.
The roles of the evaluator are to:
- Support the required data collection
- Provide coherent description of program operation and interpret information on program indicators
- Interpret information about participant outcomes
- Conduct a locally-specific systematic inquiry
- Engage in objective, data-based dialogue about program improvement, linking directly to the program’s Action Plan
The local evaluation report must include:
- Program description as context for understanding the program
- Analysis of participant outcome results (including but not limited to performance indicator results), with special focus on participant goals from the program’s Action Plan
- Focused, systematic inquiry (for programs beyond the first year of implementation) to include methods/questions and results
- Discussion and interpretation of all data and findings
- Recommendations for program improvement based on data
All Even Start programs should develop a contract with their local evaluator that includes at least the following:
- NYSED allows up to $12,000 per year for the annual local evaluation, which under normal circumstances will be inclusive of the evaluator’s labor, travel, and participation in the required annual professional development presented by the Department. There may be some circumstances, e.g., unusually large number of sites, for which the contract would include additional travel.
- Specifically identified tasks, including the requirement that evaluators observe all four instructional components as part of evaluative activities
- Products and deliverables
- Timelines for completion
- Schedule for payment
Reporting:
- Mid-year memo by January 15. Each year, the evaluator should provide to the program an informal memo about progress on the previous year’s recommendations, mid-course program corrections or evaluation design changes that are needed, and/or any major issues that have been observed during the first part of the year. The memo might also include results from the current year’s inquiry depending on the time schedule for data collection as well as information that will inform progress on the program’s Action Plan goals and objectives.
- Final evaluation report by September 15. Each year, the program coordinator must submit a final copy of the evaluation report to the New York State Education Department by September 15.
Role of Local Evaluation with Programs in Need of Improvement:
- For programs in need of improvement, the evaluation must focus the local inquiry on exploring the issues underlying the program’s performance problems as identified in the official designation and evaluating the solutions for those problems.