YEAR FIVE EVALUATION REPORT

NATIONAL COALITION FOR EQUALITY IN LEARNING

EXECUTIVE SUMMARY


This evaluation report analyzes progress toward the objectives of the National Coalition for Equality in Learning, and makes recommendations for the conclusion of the project. Located in seven school districts and three affiliate school districts across the United States, the National Coalition is completing a five-year effort to help all children of all families realize their academic and personal promise. Evaluation reports for the first four years have established that a Ònational laboratoryÓ for studying and implementing promising approaches for helping all children learn has been created in the National Coalition. Coalition schools Òlook like America,Ó for the mix of urban, suburban, and rural schools in the Coalition mirrors the various locations, grade levels, school sizes, economic levels, and ethnic diversity of our nationÕs public schools. This accomplishment makes it possible to examine meaningfully the challenges of creating school environments where children of many backgrounds can succeed in their learning. The fifth year of the project was one of extension and refinement, as schools and school systems deepened their understanding of and commitment to the goals and processes advocated by the Coalition. This year, the number of member schools increased, from sixty-one to sixty-five schools. 100% of the schools reported setting priorities for student learning, and 92% of these schools reported study teams at work to accomplish these priorities. For sixty-five schools in member districts, the National Coalition has served as a means to strengthen the cooperation among educators, parents and community leaders; as a vehicle for leadership development; and as a source of ideas and inspiration for the creative process of renewing schools. The generous five-and-a-half year cycle of support by The Danforth Foundation ends in December, 1995; although project activities will continue through the school year. Thus, this report marks the last school year of the original period of commitment asked of the member school districts.

The charge adopted by member school districts for the period September, 1994, through June, 1995, consisted of seven objectives. These were derived from the long-range goals of the project, the recommendations in the Year Four Evaluation, and the directions proposed by the Year Five funding proposal. For each objective, target accomplishments were set, including criteria that would indicate achievement of the objective. A summary of progress will follow the statement of each objective. A full analysis is contained in the Year Five Evaluation Report.

Objective 1. Help principals, teachers, and members of the school communities set and accomplish priorities for improving the learning of students who are not succeeding in school.

Anticipated Accomplishments

At least 95% of the member schools will identify one or more priorities for improving student learning.

For at least 80% of the priorities, member schools will report qualitative and/or quantitative evidence showing the extent to which the priorities have been accomplished.

At least 75% of the member schools will identify a group of students having difficulty accomplishing the priorities set in school, and will prepare case studies or other data that show the learning progress of these students.

Year Five was the third year that schools reported evidence describing their progress on priorities. The target set for this part of the objective was that quantitative and/or qualitative evidence would be reported for at least 80% of the priorities. In Year Five, evidence of progress was described for 88% of the priorities actually reported by schools, indicating this part of the objective was clearly accomplished. In addition, forty-two schools (65%) reported the use of case studies or other evaluations for target groups of students. This is approximately the same percent as reported last year that they had identified specific students, developed interventions, and tracked results. So, this part of the objective was partially accomplished and use of case studies may have reached a numerical plateau in the extent of implementation.

Table 2 from the Year Five Report (included below) summarizes the reporting of evidence. Quantitative evidence was reported for 49% of the priorities; qualitative evidence was reported for 28% of the priorities; and both types of evidence were provided for 10% of the priorities. Trend data included in Table 2 show overall progress toward developing quantitative evidence for the accomplishment of priorities, but no change from last year. Over three years, the percent of priorities where progress is described in quantitative terms has nearly doubled, while the percent of priorities reported without evidence of accomplishment has dropped from 26% in Year Three to 12% in Year 5. However, the percent of priorities for which no evidence was reported increased from 7% last year to 12% this year. Member schools are learning the discipline of stating priorities in terms of outcomes, but their rate of improvement in their ability to monitor their own progress has slowed. These changes are mainly attributed to five schools who reported priorities but did not provide school reports.

Table 3 from the Year Five Report (included below) summarizes progress reported by member schools in areas where they have set common priorities. In interpreting this table, it is necessary to recall that not every school is working on the outcome reported, since individual schools set priorities based on the special problems students face in their local setting.

It is heartening to note that forty-one of sixty-five schools explicitly commented on improved test scores or evaluation results, with several of the schools noting progress in more than one subject area. While comparable data are not available in each category, there is a clear trend showing an increase in the number of schools reporting achievement gains across the Coalition from Year Four to Year Five. That 63% of Coalition schools reported evidence of improved test scores is a clear sign of improving momentum in learning.

One of the best examples of a schoolÕs tracking progress using test scores came from the Plymouth Elementary School report (Plymouth Meeting, Pennsylvania). By focusing many resources to accomplish the one priority of improved mathematics learning, Plymouth saw major gains in studentsÕ problem-solving abilities. Multiple approaches to evaluation (pre/post testing, case studies, computer-based skill monitoring, and portfolios) added to the richness of the information used for improvement. Many other examples of test score gains are noted in Appendix B of the Year Five Report.

The goal of increasing parent involvement in schools was treated as a national objective for the second consecutive year in 1994-95. In brief, twenty-eight schools reported increased parent involvement, up from seventeen schools last year. Again this year, the member schools of Montgomery, Alabama, placed a consistent and effective emphasis on this strategy. More specific evaluation of progress will be discussed later (cf. Appendix B).

Another twenty-seven schools reported notable progress in curriculum planning. This was an increase from twenty-two schools in Year Four. Progress described in an excellent report from Juan Linn Science and Math Magnet School in Victoria, Texas, illustrates the power of curriculum planning in improving student achievement. A Coalition school for five years, Juan Linn has moved to a year-round calendar, re-aligned its math curriculum, and enhanced its use of technology, while simultaneously pushing for increased parent involvement. Longitudinal examination of test data shows that the schoolÕs goal of increasing overall student achievement in reading, writing and math is clearly being met. Moreover, these improvements hold true across ethnic and economic groups. Nearly every group for which data is dis-aggregated (Hispanic, White, and Economically Disadvantaged students) has shown significant improvement in all subjects since 1990. Other schools with notable successes worth reviewing in Appendix B include the Hirsch and Twin Creeks schools in Spring, Texas, and Sheridan School in San Francisco.

This year, seventeen schools reported progress on priorities related to reducing the number of disciplinary incidents and their consequences. This was an increase from nine schools in Year Four. Discipline progress was consistently reported in San Francisco schools: Commodore Sloat, Hillcrest, Lafayette, Miraloma, Sheridan, and Ulloa. Middle Schools (Valley Middle in Apple Valley, Minnesota, and Colonial Middle in Plymouth Meeting, Pennsylvania) were also very successful in their interventions to improve discipline.

Dropout reduction/grade retention successes were reported in thirteen schools, up from eight schools last year. The San Francisco schools concentrated efforts on this objective, as did several schools in Spring, Texas, and Apple Valley, Minnesota. A program in Apple Valley deserves special notice for starting with kindergarten age children who lived in a trailer park. The Cedar Park Elementary School has taken the lead in promoting the establishment of a community council in the nearby trailer park, while holding parent meetings and other school functions there. Through patient and persistent outreach, the kindergarten readiness program had 90% participation from eligible students, most of whom registered for a bridge summer program before first grade. Finally, progress was reported in three other important categories by more than five schools. Improvements in attendance were noted in eleven schools last year; while reduced referrals and improved inclusion for special education students were listed in eight schools. Gains in human relations were reported in five schools. In short, hard evidence of quantifiable progress was reported in many different forms across the Coalition, indicating accomplishment of Objective 1.

Objective 2. Help school systems create a way to evaluate their effectiveness in improving student learning.

Anticipated Accomplishments

At least 80% of the member school systems will conduct an annual evaluation of the progress local schools are making toward improving student learning and strengthening teacher and principal leadership, according to the specifications listed in the evaluation design of the National Coalition.

At least 60% of the member schools will present evidence describing their efforts to increase the internal capacity for evaluation in the local school.

One of the major findings of the National Coalition is that schools need the capacity to evaluate themselves so they can continuously use data to increase the effectiveness of their work to improve learning. In Year Five, evaluation reports were received from every member school except for five (92%). Evaluation reports were also received from every member school district, although a partial report was received from San Francisco due to a summer change of facilitator. With the exception of San Francisco, the district reports were judged to meet the specifications of the evaluation design by providing nearly all of the data requested in the design. The criteria set for this part of the objective were clearly met. This year, schools were directed to reflect in paragraph form on their progress toward developing an improved internal capacity for evaluation. The objective was for at least 60% of the schools to reflect briefly on their successes and setbacks in learning to evaluate themselves more effectively. Appendix C reports, on a school-by-school basis, whether member schools attended to this task. Forty-four schools (68%) did take the time to inquire into the effectiveness of their own evaluation. For the first year of Òevaluating oneÕs own evaluation,Ó this is a respectable result that clearly exceeds the level set as a goal.

In sum, by accomplishing its second national objective of helping schools create a way to evaluate student learning, the National Coalition has also uncovered a significant need in our profession. Intensive training and support appear to be necessary to assist principals and school staffs as they begin to take back into their own hands the responsibility for evaluation. Coaching and modeling on a school by school basis would probably be necessary to raise schools to the level of sophistication in evaluation needed to take full advantage of the power of evaluation to improve learning.

Objective 3. Provide leadership development for teachers and principals that centers on crucial themes in improving student learning and on the problem-solving process for National Coalition Schools.

Anticipated Accomplishments

At least 80% of member schools will report they have used local and/or district study teams as an approach to accomplishing their schoolÕs priorities.

At least 80% of the member school systems will send a representative to participate in each of the national study teams.

At least 80% of the participants in national study teams will judge the effectiveness of their experience to be excellent.

At least 75% of the member school systems will send one representative to participate in at least one exchange with another National Coalition school system.

At least 80% of the participants in exchanges will rate the value of their experience to be excellent.

At least 80% of the member school systems will send at least one representative to a summer institute to improve leadership skills and develop plans for improving student health and learning.

At least 80% of those participating in the summer leadership institute will judge the effectiveness of their experience to be excellent.

In Year Five, sixty schools (92%) reported the use of local study teams to accomplish school priorities. This level of implementation comfortably exceeds the criterion level of 80% implementation. The exceptions to the norm are restricted to five schools. Three national study teams were conducted this year: ÒLearning in School and Non-School Settings,Ó in New Orleans; ÒCreating Conditions for Effective Learning,Ó in San Francisco; and, ÒExcellence with Decency,Ó in Boulder, Colorado. Two major district study teams were opened on an invitational basis to other member districts: ÒMeeting the Needs of All Learners,Ó in Plymouth Meeting, Pennsylvania; and, ÒAssessing Portraits in Learning,Ó in Spring, Texas. Four districts (57%) sent representatives to all three national study teams; two districts sent representatives to two national study teams; one district only sent representatives to their own study team due to restrictions on travel out of state. Thus, the Coalition fell somewhat short in encouraging attendance from each district at every national study team, but overall participation in national study teams should be considered high. In the evaluation summaries received from the New Orleans, Plymouth Meeting, and Spring study teams, the average rating was 4.6 on a 5.0 scale, with 63% of the participants granting the highest rating of excellent. These results fall short of the exceedingly high standard of 80% excellence set by the Coalition, but they do reflect a high level of workshop satisfaction overall. Six of seven member districts (86%) report participating in exchanges, exceeding the 80% participation criterion. Unfortunately, it is harder to collect evaluation data from these often informal exchanges, and incomplete evaluation data make it impossible to estimate the overall success of these ventures in a quantitative way. Representatives from every member district have attended the last two summer institutes. Results from the summer of 1994 indicate that 23 of 26 participants rated the institute as excellent. Seventeen principals and teachers attended the 1995 Summer Leadership Institute in Boulder, and all seventeen considered the experience to be excellent. In many ways, leadership development continues to be one of the CoalitionÕs greatest strengths.

Objective 4. Recognize selected schools that have made significant progress in building a broad-based commitment to providing all students with a quality education on equal terms.

Anticipated Accomplishments

At least five schools will achieve Equality School status in the National Coalition.

In Year Five, eleven schools applied to be recognized as Equality Schools, exceeding the criterion of five schools. The Equality Schools included:

EQUALITY SCHOOLS, 1995
When recognition ceremonies are completed, twenty of sixty-five member schools will have been recognized as Equality Schools (31% of member schools). The CoalitionÕs record indicates accomplishment of this objective, and that is a sign of substantive progress toward its vision of equity.

Objective 5. Increase cooperation among schools and community agencies in the accomplishment of school priorities.

Anticipated Accomplishments

At least half of member schools will report community involvement in the accomplishment of learning priorities for which this cooperation is deemed appropriate.

At least 80% of the school districts will report the involvement of at least one university and one health agency in support of local school priorities.

Thirty-nine schools (60%) reported community involvement. Every district reported involvement by at least one university, and all but one district reported involvement by a health agency in the pursuit of their priorities. The number of institutions of higher education, health agencies, and other community organizations involved in Coalition schools all increased in Year 5, with twenty-two colleges and universities, eleven health agencies, and ninety-three other organizations helping Coalition schools. The Coalition clearly accomplished its community involvement objective this year.

Objective 6. Strengthen local school efforts for involving family members in the learning of their children, particularly those family members who live in poverty or have children who are failing in school.

Anticipated Accomplishments

At least 80% of member schools will report efforts to promote family involvement initiatives that center on improving the learning of children who are having difficulty succeeding in school.

In Year Five, fifty schools (77%) reported family involvement as a strategy to accomplish school priorities. This effectively meets the criterion level of 80% set for this objective. Of the fifty schools who referred to family involvement in their reports, twenty-eight set increasing family involvement as one of their selected priorities (the other schools mentioned it as a means to their ends). Twenty-two schools set a priority on involving parents as partners in their learning; six schools emphasized increasing communication between home, school, and community. All twenty-eight schools setting a family involvement priority provided data that indicated an increase in family involvement this year.

Objective 7. Make plans for continuing the work of the National Coalition after the fifth year of funding from The Danforth Foundation.
 At least 80% of member school districts will describe their plans for continuation of National Coalition work using internal resources.

100% of the member school districts currently sponsoring teen parenting programs that have been supported by the National Coalition will make a commitment to continue these programs.

At least 50% of the member school districts will start or continue a local foundation to support the learning of students.

At least 80% of member school districts will make a commitment to participate in the planning of a Center for Equity in Education at the University of Massachusetts as a way of continuing the work of the National Coalition.

From the beginning, the National Coalition has emphasized the importance of local school leadership. The Coalition did not infuse significant new financial resources into member districts, did not dictate a national agenda, and concentrated on teaching school personnel new skills and problem-solving processes they could adapt to their own purposes. So, at this point of transition after five years of external funding, it is not surprising that member schools express both interest and confidence in continuing the Coalition problem-solving processes with less external support. Plans exist for continuing National Coalition work using internal resources in each member district. All four school districts that have teen parent programs are fully committed to continuing them with their own resources. Six of seven school districts have developed or begun to plan local foundations to raise money for district needs at the CoalitionÕs suggestion. Finally, each of the districts would be pleased to participate in planning efforts for ongoing collaboration with the National Coalition. While a firm national plan has not been established, the districts are committed to working together and to considering emerging opportunities.

Recommended Directions for Project Completion

Through discussion with national staff and project facilitators, and through analysis of school and community reports, the evaluator recommends the following directions for completing the project.

1. Although project funding is ended in December, the National Coalition should husband its funds so that project activities can continue throughout the academic year, ending in June, 1996.

2. The Coalition has established a national laboratory of schools with cooperative working relationships, a common philosophy, and a commitment to reform. There is a sense of momentum present. This carefully developed resource should be used for continued in-depth investigation of particular problems, trends, and promising strategies.

3. The national staff should work closely with project facilitators from each member school district, to assist their efforts to have the local districts continue Coalition activities. The facilitators are the local standard bearers for Coalition ideas, and should be assisted in their roles as internal change agents. Winter and spring facilitator meetings should be organized to accomplish these purposes.

4. The Coalition should explore shared funding arrangements for continuation. The Danforth Foundation could contribute to project continuation, with other funding sources taking over the major share of funding while The Danforth Foundation continues to contribute in a supporting manner.

5. Several initiatives developed within the Coalition deserve broader exposure as promising strategies for reform. In particular, the CoalitionÕs work in establishing a systematic problem-solving process for local schools, its professional development program, its work in developing programs for teen parents, its family involvement initiatives, and its focus on developing the capacity of local schools to evaluate their own progress deserve to be described and disseminated to other educators.

6. This year, the Coalition should invite teachers and administrators in project schools to develop a series of papers that describe progress in their schools using Coalition approaches. These papers will serve as a source of information about what has been accomplished. This compilation should serve as the final evaluation report for the project.

7. The Danforth Foundation has an opportunity to support the dissemination of these papers and other information about the Coalition as a way of demonstrating its contributions to school reform through the funding of the National Coalition for Equality in Learning. The foundation has much to gain from assisting in the publication and distribution of information about the Coalition, and should consider a supplemental grant to support this final step in bringing the gains accomplished in member schools to the attention of the national educational community.

Conclusion

The National Coalition for Equality in Learning can take pride in its maturation as an effective school reform network over the five years of its existence. This year, 100% of the sixty-five member schools identified 166 priorities; for 88% of these priorities, quantitative or qualitative evaluation data were reported. Forty-one of sixty-five schools reported improved test scores or achievement gains, with many schools reporting increased parent involvement, new curriculum, reduced disciplinary incidents, better retention, improved attendance, reduced referrals to special education, or improvements in human relations. In 65% of the schools, students having difficulty learning were selected for special interventions.

When recognition ceremonies are completed, twenty of sixty-five member schools will have been recognized as Equality Schools (31% of member schools). Thirty-nine schools (60%) reported community involvement. Fifty schools (77%) reported family involvement as a strategy to accomplish school priorities. Plans exist for continuing National Coalition work using internal resources in each member district. With a minimum of external resources, sixty-five schools have learned to identify and resolve important problems blocking learning, and have made significant progress toward achieving a vision of equity in schooling.

The Year Five Evaluation Report has provided data and analysis that document progress made. The report also offers constructive criticism and suggestions intended to help Coalition members decide where they will take their project in the coming year. In the evaluatorÕs opinion, there is national significance in this project. It deserves much broader dissemination and consideration wherever educators work to create conditions for all children and youth to learn well.
The Executive Summary was prepared by: Ward J. Ghory Director of the Upper School Buckingham, Browne & Nichols School Cambridge, Massachusetts