University of Massachusetts Amherst Campus Procedures for Academic Quality Assessment and Development (AQAD)
Incorporating University of Massachusetts System Guidelines (Trustee Doc. T98-033)
Office of the Provost, September 2018
What is an AQAD? The Academic Quality Assessment and Development (AQAD) process is a component of the UMass System’s Performance Measurement System and is required of all academic units. The primary purpose of the AQAD is to assess the core academic functions of teaching, learning, research/professional/creative activity, public service, and academic outreach on a regular basis. Each UMass campus has established procedures for implementing AQADs that are in accordance with system-level guidelines adopted by the board of trustees (Doc. T98-033). In this document, “department” and “program” are used as synonyms for “unit undergoing review.”
The system guidelines require that programs be reviewed at least every seven years, but UMass Amherst has approval for some variances in order to match external review (e.g., accreditation) schedules. These include an eight-year cycle for Communication Disorders, and 10-year cycles for Music and Dance, all programs in the School of Management, and the School of Nursing, with an AQAD “mini-review” to be conducted at the five-year midpoints.
How can I use an AQAD to help my department/program? An AQAD is a tool that you can use to guide the pedagogical development and intellectual future of your department or program. It is an opportunity to assess your current state and ensure that you have clear strategic goals in teaching, research, and creative activity that are in tune with campus priorities and at the leading edge of your discipline. Although all AQADs must address the same general core criteria (see appendix) using the same general procedures, both the content and process may with the approval of the dean and provost be adjusted somewhat to suit a particular unit’s needs.
A common misconception. Sometimes, the AQAD process is viewed primarily as a way of making the case for additional faculty lines or other resources (e.g., improved graduate stipends). This approach misconceives the role of AQAD. Requests for additional resources are made through the annual budget process, not AQAD. Departments are, however, warmly encouraged to use the AQAD process to think about internal reallocation of existing resources or generation of new funding. Examples might include replacing retired faculty in new areas to reflect new disciplinary or pedagogical developments, improving graduate funding by reducing enrollment, and development of additional funding sources through CPE, self-paid master’s programs, research grants, or training grants.
Parts of an AQAD. An AQAD minimally consists of four documents: a program self-assessment, a report from external reviewers, a program response to the external report that includes an action plan, and the Dean’s response to the action plan. The conclusion of the AQAD process is a meeting of the department Head/Chair, the Dean, the Senior Vice Provosts, and the Provost.
- The centerpiece of the self-assessment is a short (@ 20 page) narrative that describes the current state of your department, its aspirations, and the key issues that it faces. (CVs or tabular data can be added as appendices.) The narrative should be developed in consultation with your Dean so that there is a common understanding of your goals. It is expected that the strategic planning documents you prepared in recent years will prove very useful (when appropriately updated to add new data and/or address additional topics) in the preparation of the self-assessment.
- The dean, in consultation with the department head/chair and faculty, will identify a team of at least two external reviewers, one of whom can be from another campus in the UMass system. Ideally, reviewers will come from public R1 universities and will have administrative experience at the level of a department chair or higher. The dean, department head/chair, and faculty will work together to determine the emphasis of the visiting team’s review, and University of Massachusetts Amherst Academic Quality Assessment and Development (AQAD) Procedures.
- The provost will have the opportunity to identify specific issues or questions that should be addressed. The provost will approve team membership and may add a member. The review team is charged with providing a written report based on the self-study, the questions posed by the dean and provost, and their experience on campus.
- The key product of the AQAD process is the action plan, which specifies how the department plans to address any issues raised during the review. The action plan should also indicate whether any subsequent action is needed once the review itself is completed (e.g., follow-up on specific issues, earlier-than-normal subsequent review, etc.).
Steps and timing of the AQAD review process
- The department head/chair, the dean, and the Provost’s Office meets to review procedures, answer questions, and discuss information needs. If the AQAD coincides with a disciplinary re- accreditation, the extent of the overlap will be discussed and a determination made as to what (if anything) needs to be done beyond accreditation requirements.
- OIR/OAPA provides data for the self-study, and can meet with the department to review the evidence and assist in the interpretation of the data (see appendix).
- Dean recommends external review team to the provost.
- After approval of external reviewers, the department begins to schedule external review and surrounding activities.
- The dean meets with department representatives to discuss the emphasis of self-study, with the assumption that areas of greatest concern will receive the greatest attention. A one-page summary of the self-study strategy is prepared by the department and forwarded to the dean and the Provost for feedback.
- Department begins self-study.
- Dean, in consultation with department, finalizes the emphasis of external review and submits it to the provost for approval.
- Department submits completed self-study to dean and provost.
4-6 weeks prior to external review
- Department sends the self-study to the visiting team. Consider posting ancillary documents (CVs, tabular data, course syllabi, etc.) online for the reviewers’ ease of access. The provost sends the visiting team a letter, thanking them in advance for their service, and laying out some areas of focus for the review.
February - March
- External reviewers visit. The provost presents the review committee with its charge during an entrance interview and conducts an exit interview at the end of their visit.
- The visiting team delivers its report to the department, with copies to the dean and the provost.
- The department submits a written response to the report and an action plan to the dean.
- The dean reviews and provides written comments on the report, response, and action plan to the faculty. The faculty may then respond to the dean’s comments.
- The dean forwards the self-study, the visiting team’s report, the department’s response and action plan, the dean’s comments, and faculty responses to the dean’s comments (if any) to the provost.
- The provost accepts the dean’s review.
- The Provost’s Office meets with the dean and the department head/chair to discuss the review and the action plan.
- The provost forwards an executive summary of the review to the President’s Office.
Appendix 1: Core Criteria for the Self-Study
Institutional data to address these questions are available through the Office of Institutional Research and the Office of Academic Planning and Assessment. The Strategic Planning/Unit Planning process, has provided an opportunity for most of the data to be organized into a format more conducive to examination and analysis.
1. Programs shall ensure that their goals and objectives are linked to the campus mission and strategic priorities, and to their strategy for improving their position within the discipline.
The program should evaluate its purpose and planning in light of the campus mission and strategic priorities, and should assess its standing among similar programs nationally. The review should answer the following questions:
- What is the program’s mission and is it clearly aligned with the campus mission and direction?
- How does the program’s mission relate to curriculum; enrollments; faculty teaching, research/professional/creative activity, and outreach? Is it aligned with the campus strategic priorities?
- Describe the program’s strategy for promoting diversity.
- Describe the program’s goals, challenges, and successes in nurturing a positive climate for students, staff, and faculty.
- How does the program contribute to campus-wide curricular needs through general education and service instruction?
- What is the program’s current standing within the discipline, especially with respect to research and graduate education? What goals does the program have in terms of its national standing?
2. Programs shall ensure that curriculum is relevant, rigorous, current and coherent.
The need to provide a high quality education for students should be the primary consideration when evaluating the relevancy, currency, and coherence of curricula.
Evaluation of the curriculum should reflect an awareness of changing knowledge, trends in the discipline, and the professional context for curriculum. The review should answer the following questions:
- How does the program determine curricular content? How does the curriculum relate to current existing standards, if any, of the discipline?
- What internal or external measures of review are employed to ensure that the curriculum is relevant and up to date?
- What guidance and support are provided to students to ensure their timely progress toward degree completion? In addition to describing your academic advising program, discuss the use of concentrations and/or model pathways in guiding students through the curriculum.
- Are the curricular offerings structured in a logical, sequential and coherent manner? Is there an appropriate balance between breadth and depth? How does the curriculum address and reinforce departmental and university-wide learning objectives?
- If consistent with the program mission, does the curriculum adequately prepare students for further study or employment? If applicable, include a description of how the curriculum offers pathways for internships or other experiential learning opportunities.
- In what way does the program contribute to the education of students in terms of general knowledge, critical thinking capacity and other essential cognitive skills?
3. Programs shall ensure faculty quality and productivity.
Programs shall ensure that faculty possess the expertise to assure effective curriculum development, instructional design and delivery, and evaluation of outcomes. Faculty should exhibit awareness of trends in the discipline and the professional field as appropriate. Collectively, faculty should be involved in teaching, research/ professional/creative activity, and public service/academic outreach as appropriate to the mission and regional context of the campus. The review should answer the following questions:
- Do faculty possess the appropriate background, experience and credentials?
- To what extent are faculty members current in relation to the knowledge base and content of the discipline and curricular offerings?
- Are the program expectations for faculty involvement in teaching, research/professional/creative activity, and public service/academic outreach activities appropriate; and how are these expectations met? Are these expectations consistent with program policies regarding teaching assignments, merit allocations, and other aspects of faculty roles and rewards?
- In what ways does the program foster professional development and growth of faculty? How are assistant and associate professors mentored? Does the department make effective use of the campus’s formative evaluation processes for faculty: AFR, 4.2 reappointment review (“mini-tenure”), and PMYR?
- In what ways do program faculty lend its professional expertise—as expressed through teaching and research, scholarly and creative activity—to off-campus constituencies?
4. Programs shall ensure teaching/learning environments that facilitate student success.
Programs shall provide learning environments that promote student success. Students are expected to learn both content and skills appropriate to the discipline. The program should indicate clear expectations for student learning outcomes. The teaching/learning environment should be accessible to all students, should include a variety of instructional methodologies, and should provide timely feedback to students. The review should answer the following questions:
- What is the program looking for in its students? What kind of students is the program well suited to serve? How does the program define “quality” in terms of admission to the program (when relevant)?
- To what extent does the program have articulated learning outcomes (content and skills) for students? (Outcomes should be articulated at both the undergraduate and graduate levels and by sub-fields, when relevant.) By what means are these outcomes measured? Are they achieved by the majority of students? For programs with graduate research or teaching assistantships: To what extent does the program have articulated learning outcomes for the TA/RA experience? By what means are these outcomes and the quality of the experience assessed?
- How does the department collect and use evidence to inform and improve the quality of the student experience and student learning? Describe the processes the department deploys to identify lines of inquiry, draw from existing evidence and collect additional evidence, and develop a collective understanding of findings and priorities for improvement.
- How are program expectations communicated to students? Are students kept informed of their progress in meeting intended program outcomes?
- How does the department use the assessment data on student learning outcomes?
- How is assessment of student learning outcomes used in reviewing and modifying program curriculum, advising, and other program elements, and in evaluating faculty? What are departmental expectations for faculty participation in program level student learning assessment and other inquiry into program effectiveness?
- In what ways are students advised regarding selection of the major and progress through the major?
- Describe any co-curricular, internship, research, and career-development opportunities outside the required curriculum that you provide for students.
- In what ways does the program evaluate student success following graduation and the program’s contribution to that success?
- What is the role of the core faculty in teaching lower division, upper division, and graduate courses? What is the rationale for these assignments?
- Metrics on first-year students and overall retention and graduation rates shall be provided for the period since the previous AQAD review.
5. Programs shall ensure that resources are used wisely.
Programs shall ensure that the resources available are used to meet goals and objectives, and as appropriate, engage in use of innovation to enhance resources. Programs also should engage in both intra- and inter-campus collaboration, and should demonstrate a commitment to effective and efficient use of resources. The review should answer the following questions:
- What process does the program use to allocate resources?
- In what ways does the program maximize the use of its human resources?
- In what ways does the program maximize the use of material resources such as space, equipment, operating funds, etc.? Is there an internal process for reallocating underutilized space to more productive purposes?
- What strategies does the program employ to develop alternate sources of revenue (private giving, grants and contracts, online education, MS programs, etc.), and to share costs with other public and private entities?