- Career Planning and Implementation Conferences (CPIC) - Years One, Two, and Three
- The Chicago Public Schools Study
- The Real Game
- College Board Inspiration and Innovation Study
- Bridges Transitions Evaluation
- Evidence Based Practice Panel
- Delphi Study
The Massachusetts Department of Elementary and Secondary Education (DESE) partnered with CSCORE during the 2007-08, 2008-09, and 2009-10 school years to provide professional development and training to school counselors throughout the Commonwealth on assisting students in effective and meaningful career planning for the 21st century. Topics for each year progressively built on the content discussed in previous years and were chosen based on feedback and comments from counselors about what information and skills they needed.
Year OneThe first year of the career planning and implementation professional development series focused on developing the capacity of Massachusetts vocational high school educators to help all students construct career plans that support academic achievement and career development. CSCORE facilitated three one-day conferences held in Amherst, North Andover, and Bridgewater with 80 to 100 attendees at each conference; schools were encouraged to bring teams of school counselors, administrators and teachers to the event. Representatives from different institutions of higher education, MASCA, and DESE spoke at the conferences about the essential knowledge and skills needed to create effective career plans, best practices for using assessment information, and what the research demonstrates about the impact of career planning on student success. Following the one day event, conference attendees were able to choose from a follow-up menu of professional development options to be delivered throughout the school year from CSCORE staff and other experts in the field, including an on-line career development workshop, on-site or phone consultation about the career planning process at a participant’s school, or participation in a Leadership Academy.
Many school counselors who participated in year one of the CPIC commented that they lacked the knowledge and skills needed to navigate the numerous online and electronic options available for career planning in the 21st century. CSCORE and the Massachusetts DESE therefore decided to coordinate a statewide conference on the use of technology to support effective and meaningful career planning during year two of the CPIC. Conference topics included linking career information delivery systems with career planning systems, streamlining guidance functions through technology, creating Educational Proficiency Plans that also serve as career plans; and using technology for graduate follow-up data collection. Conference participants were also able to provide feedback to the Massachusetts Educational Finance Authority (MEFA) on the development of the Massachusetts College and Career Portal.
The Massachusetts School Counselor Leadership group formed during the sustained professional development series of the previous year continued its work during the second year of CPIC. CSCORE facilitated a weekend conference in which the group was tasked with identifying best practices in career development throughout the Commonwealth. The list of best practices included transition planning; classroom guidance lessons by grade level; goal setting; cross-cultural, integrated, and multi-modal career interventions; career advisories; and specific strategies for effective Individual Planning sessions. CSCORE presented the work of the Leadership group at the spring 2009 Massachusetts School Counseling Association (MASCA) conference.
Supporting career and college readiness through career planning was the focus of the third year of the CPIC professional development series. A statewide conference was held in December 2009 at Quinsigamond College and featured two education experts, Dr. Joyce Brown from the Chicago Public Schools, and Jibril Solomon from UMass Boston. Dr. Brown spoke about her 12-Touch Program, a framework of practices for school counselors to ensure that every child in the school is connected to a caring adult. Jibril Solomon discussed the importance of making a shift in thinking about assessment information as value capital to assist students in becoming college and career ready.A second career-related goal for the year was to revise the Massachusetts Career Development Education Benchmarks. CSCORE and DESE staff, along with school counselors throughout the state, rewrote sections of the Benchmarks to reflect changes that had occurred in the global marketplace since the Benchmarks were first written in 2000. In order to provide a chance for public comment, CSCORE posted on its web site a survey to allow various constituents across the Commonwealth to offer feedback on the revised Benchmarks.
Over the past several years, the Chicago Public Schools (CPS) has instituted a major reform initiative to revitalize and enhance the work of their school counselors. CPS school counselors have adopted a best practices approach, based on the ASCA National Model, that emphasizes the development and implementation of a comprehensive school counseling program. During the 2007-2008 school year, the Chicago Public Schools (CPS) commissioned CSCORE to conduct an evaluation study to examine the benefits to CPS students when school counselors implement key components of a comprehensive school counseling program.
Every public high school in Chicago was invited to participate in the study. The sample was representative of the diverse types of CPS high schools and included general, selective enrollment, military, magnet, career academy, and special education high schools. Principals and school counselors at participating high schools completed an electronic version of the Principal and Counselor Survey (Lapan & Carey, 2007) to assess activities delivered by school counselors in each high school during the previous school year. Student data on such indicators as percentage of students receiving free or reduced lunch, attendance, state standardized test scores, Advanced Placement scores and enrollment, dropout rate, and post secondary plans were also collected from the CPS district.
Major analyses of the Principal and Counselor Survey focused on three critical variables: Non-Guidance tasks, Individual Planning, and the 12 Touch Program. Both school counselors and principals were asked to assess the extent to which CPS high school counselors were carrying out such Non-Guidance tasks as performing hall duty, supervising the lunchroom, or acting as principal for the day. Individual Planning, one of the four primary components of a comprehensive school counseling program, involves assisting all students in exploring and planning for their educational and career futures, working with students one-on-one, in small groups, or in classroom settings. The 12 Touch Program is a menu of transition practices designed to intentionally connect freshman to high school. The program aims to better integrate entering ninth graders into their high schools by promoting a student’s sense of belonging and connection to school. The relationships between these variables and student data indicators were then further explored to examine the role of CPS school counselors in promoting students’ academic achievement, college readiness, and transition from 8th grade into high school.
Four key findings were reported from this study: CPS high school counselors play an important role in promoting student academic achievement; CPS high school counselors impact how students plan, prepare, search, apply for and enroll in college; CPS high school counselors play a vital role in helping 8th grade students successfully transition into high school; an implementation gap exists across CPS high schools in the organization and delivery of a comprehensive guidance program, thus advantaging some students and disadvantaging others. To read a full copy of the report, click on the following link:
The Center for School Counseling Outcome Research & Evaluation (CSCORE), funded by America’s Career Resource Network, conducted a multi-state, multi-year evaluation of The Real Game Series (RGS). The Real Game Series is a K-16 set of curriculum materials designed to bring interactive, experiential learning to classroom and group settings to increase students’ perceptions of the relevance and importance of their school experience (Jarvis & Keeley, 2003). The purpose of the evaluation was to determine the impact of the intervention on student career and academic outcomes. The evaluation focused on the RGS component that is designed for middle school (grade 7 and 8) implementation, The Real Game.
Using quantitative and qualitative methodologies CSCORE evaluated (a) the immediate learning outcomes of the Real Game; and (b) the secondary outcomes related to academic achievement such as school involvement, planning for the future, self-efficacy and motivation, homework completion, attention, and appropriate school behavior.
This evaluation of the Real Game career development education curriculum found that it is an effective intervention in several domains. The first goal of the evaluation was to determine whether students participating in the RG learned the specific content about careers and the world of work. On the post-test, the students who participated in the RG scored significantly higher on every measure of content knowledge (World of Work, Short-term Learning Objectives, and Matching) than the control group, indicating that they had learned this information more than the control group had, and that the learning that occurred is most likely due to the intervention.
The second goal of the evaluation was to determine whether participating in the RG results in measurable gains in career-related outcomes such as career maturity and career decision-making, better quality high school transition plans, and increased recognition of the relevance of academic education. On the secondary outcomes that were measured, students who participated in the RG scored significantly higher than the control group on the measure related to future orientation, indicating that they were more thoughtful about future plans than the control group. The RG group was not significantly different than the control group on the measure of career maturity. The long-term learning outcomes, measured qualitatively, found some indication that students who participated in the RG were more aware of the importance of making good decisions about high school academics, and that those choices would impact their future career options.
The final goal of the evaluation was to determine whether students who completed the Real Game intervention showed changes in factors that have been demonstrated by research to be related to academic achievement and appropriate goal-directed, school behavior (e.g., self-efficacy, motivation, school engagement/social bonding, decision-making skills, goal-setting skills, and social-problem solving skills). This study found that there were significant differences for the Real Game participants in the domains of self-efficacy, school engagement, and prosocial behavior, but not motivation, decision-making or goal-setting. On measures of self-efficacy, school engagement, and prosocial behaviors the post-test scores for the Real Game group were significantly higher than the control group, indicating that the intervention impacted these beliefs and behaviors.
In 2006, the College Board’s National Office for School Counselor Advocacy sponsored a study conducted by the National Center for School Counseling Outcome Research & Evaluation of the school counseling practices, skills, and dispositions that are found in these award-winning schools.
The Center for School Counseling Outcome Research & Evaluation conducted an evaluation of the Bridges Transitions career exploration and awareness program in Middle and High Schools across the country. The evaluation employed the Missouri Comprehensive Guidance Evaluation Survey to assess career and career related outcomes, and indicators of school engagement through academic achievement and attendance.
The evaluation was sponsored by Bridges Transitions.
Evidence Based Practice Panel
The Evidence-Based Practice Panel (EBP) was established in 2003, based on the recommendations derived from our Delphi Study of research needs for the field of school counseling. The panel is tasked with reviewing the research literature to define practices in the field of school counseling that are evidence-based or promising practices. Evidence-based practices are practices which have been rigorously studied and evaluated, and have accumulated research evidence which consistently shows that those practices are effective. Promising practices are practices that have been rigorously studied and evaluated, but have not yet accumulated enough research evidence. The EBP panel has developed a protocol and rubric for evaluating the extent to which school counseling practices are promising or evidence-based.
The Evidence Based Practice Panel Members
John Carey, Ph.D., Director, National Center for School Counseling Outcome Research & Evaluation, University of Massachusetts Amherst
Carey Dimmitt, Ph.D., Associate Director, National Center for School Counseling Outcome Research & Evaluation, University of Massachusetts Amherst
Trish Hatch, Ph.D., Director, School Counseling Program, San Diego State University;
Richard T. Lapan, Ph.D., Professor, University of Massachusetts Amherst
Susan C. Whiston, Ph.D., Professor, Indiana University
EBP Power Point Presentations
The rationale and work of the EBP Panel were presented during the ASCA Conference in Orlando, FL in June of 2005:
- Presentation with Full Color (best for viewing online- PDF, 442kb)
- Presentation in Black and White (best for printing – PDF, 295kb)
Report of the National Panel for Evidence-Based School Counseling: Outcome Research Coding Protocol and Evaluation of Student Success Skills and Second Step – Report prepared for ASCA conference in Orlando applying the EBP protocol to the Second Step and Student Success Skills programs.
The Center for School Counseling Outcome Research & Evaluation conducted a Delphi Study which began in mid-February, 2003. The study was completed to help address the need for additional research in the field of school counseling. The study determined the critical research questions and proposed a research agenda for the school counseling field. The results of the study were published in the Association of Counselor Education and Supervision journal.
For more specific details related to the results of this study, click here.
The Delphi Technique consists of several rounds of questions presented to a panel of experts in the field of school counseling. It is a research method for generating ideas and gathering comprehensive information from a wide variety of experts in a given field. The purpose of this process is to facilitate consensus on a certain topic. Data gathering occurs via email and consisted of three “rounds” of questions related to the needs of school counseling research. The Delphi Technique allows busy experts the opportunity to interact with one another to generate new ideas to solve a problem. This is done with the help of a researcher who facilitates the process and shares information with the expert panelists in a confidential manner.
An initial survey or questionnaire was sent to all Delphi panelists. These were returned to the researcher who compiled all the ideas that were presented. A second survey instrument was created based on this information. The second survey was then sent out to all panelists, who were asked to rank order, modify, and/or add to their initial responses based upon their review of other members’ ideas. This data was collected and reviewed by the researcher who created a final questionnaire based on the accumulated information that was presented by the panelists. The final instrument asked for further clarification and/or rank ordering of ideas that had been presented throughout the study process. A final report was prepared and distributed to panel members before results of the study were released to others.
Panelists were nominated to participate in this study based on their considerable knowledge and expertise in the field of school counseling. We believe their contribution to this study was invaluable, as each of them is already making exceptional contributions to the field of school counseling.