OAPA supports departments' efforts to improve their programs through three activities: Undergraduate Program Assessment, Instructional Benchmarks for Program Improvement, and AQAD Program Review.
In Spring 2013, each UMass Undergraduate Department was asked to review its current Student Learning Assessment practices and identify next steps for enhancing these efforts. OAPA facilitated this process by creating an initial inventory of each department's assessment practices (see Inventory Example). Departments vary in their approaches to assessing student learning, but overall, 96% of departments have established student learning outcomes for their majors.
Departmental assessment planning reports indicate that most departments use the Instructional Benchmarks (see description below) as one set of indirect assessment evidence. Many also use other sources of indirect evidence that they collect themselves. There are two noticeable shifts in Departments' plans when compared to their 2009 versions. In many cases, their student learning objectives have become more focused and clear, with our analysis showing 74% with well-articulated objectives. In addition, more departments are implementing or making plans to implement direct assessment to complement the virtually universal use of indirect assessment. Eleven percent of departments systematically conduct direct assessment (typically in disciplines with accreditation requirements for use of direct evidence); another 53 percent report use of direct assessments, but the activity is emergent rather than well established. These results indicate a good foundation for building greater participation in assessment generally, and direct assessment in particular.
Assessment plans for specific departments are available on the Undergraduate Program Assessment page.
The Instructional Benchmarks report, based on several sources of student feedback, provides departments with a method of evaluation that allows for interdepartmental comparisons. Originally developed in 2006, the report details students' ratings of their experiences in their majors (see Senior Survey), their experiences in courses (see SRTI), and their perceived learning gains (see NSSE) organized by department. Departments use the information to understand more about the quality of the experience for their majors and the quality of instruction experienced by the students they teach, whether those students are majors or undergraduates in General Education courses, and to make changes based on the results. A recent study found that 76 percent of departments have made improvements to the undergraduate experience informed, at least in part, by the benchmarking data. These improvements are in a broad range of areas, including writing preparation, advising, access to classes, and faculty accessibility.
The University's Academic Quality, Assessment and Development (AQAD) program review process requires departments to conduct a self-study based on AQAD standards (Campus Procedures for AQAD), and undergo a review by an external team. OAPA provided the Deans of the University's schools and colleges with a summary of the issues and concerns that had been outlined by each departmental review team and asked them to provide updates on what the departments had accomplished to address these issues. For the 58 departments and programs involved in this review, improvements were made in a wide range of areas; some common areas that departments addressed include faculty hiring (74%), program design (67%), and the undergraduate experience (31%).