Background and Methods

OAPA Wordmark

In fall 2021, all UMass Amherst students and employees (N=34,052) were invited to participate in a Campus Climate Survey (CCS) to help the university better understand the challenges of creating a respectful and inclusive campus environment. The survey was sponsored by the university’s Office of Equity and Inclusion (OEI) and conducted by the Office of Academic Planning and Assessment (OAPA). OEI and OAPA collaborated over several months to plan the survey’s administration and content. The survey instrument included a set of core questions about campus climate perceptions and experiences at UMass Amherst, and items about social identity aspects. Additional sets of questions were tailored to each of four target populations (undergraduate students, graduate students, staff, and faculty). The survey also incorporated two main open-ended questions to gather details about participants’ experiences at the university and specific suggestions for ways to improve the campus climate. 

The 2021 survey was a follow-up to UMass Amherst’s 2016 Campus Climate Survey, and included many of the same core questions and population-specific questions to provide for internal benchmarking over time. Although the 2021 survey results do allow for comparisons over time, the fruitfulness of conducting such comparisons is diminished for two main reasons, one contextual and one structural. 

First, and most importantly, the campus working and learning context for the 2016 and 2021 surveys are dramatically different. The COVID-19 pandemic prompted the campus to shift in March 2020 to remote instruction (and support services) for all undergraduate and graduate students, remote teaching for all faculty and graduate student instructors, and remote work for many staff. Staff designated as “essential” continued in-person work, some were placed on long-term furlough, and all staff experienced pay-cuts associated with being furloughed for three weeks. As multiple studies have shown this period of remote work and learning was challenging for all involved for multiple and varied reasons (e.g., childcare responsibilities, social isolation, COVID-19 illness and caretaking, facility with online technology, etc.).  

Because remote learning and working conditions existed for the entirety of academic year 2020-21, the fall 2021 CCS was conducted during the first semester that students, faculty, and non-essential staff resumed in-person activities -- albeit within a “new normal” context of mask-wearing, social distancing, “hybrid” or remote work for many employees, and remote options for advising, counseling and other student services. By fall 2021, the student body included two cohorts that were experiencing the campus in-person for the first time, and employee ranks had diminished considerably due to pandemic-prompted early retirements, separations, and furloughs.  

There also have been dramatic structural changes that impact the feasibility of assessing change over time in that two of the university’s executive areas – Administration & Finance (A&F) and Student Affairs and Campus Life (SACL) – experienced significant reorganization post-2016. Human Resources was moved out of A&F to become a stand-alone executive area, and most of SACL’s facilities staff were shifted to A&F.  

Given that the university context for conducting the campus climate survey was dramatically different from 2016, our research team decided that all public reporting of the CCS results will focus on 2021 only, beginning with the initial topical report focused on Sense of Belonging. 

There are other important national and university contextual factors that should be considered when interpreting the 2021 Campus Climate Survey results. The period of time between the 2016 and 2021 Campus Climate Surveys was socially and politically tumultuous and divisive. Of particular relevance are the increased prominence and influence of the #MeToo and Black Lives Matter social movements, which gained substantial momentum during the first year of Donald Trump’s presidency. The murders of George Floyd and Breonna Taylor by police in 2020 prompted sustained periods of widespread protest against police brutality and anti-Black racism. As many considered the legacies of racism and worked to raise consciousness regarding racial injustice, others labeled such conversations “divisive” and mobilized against them. The COVID-19 pandemic also prompted a rise in xenophobia – and specifically in violent crimes targeting Asian Americans. Also relevant are the federal government’s controversial and dehumanizing immigration policies and practices, and plan for constructing a border wall between the United States and Mexico. The five years preceding the 2021 Campus Climate Survey were characterized by intense societal scrutiny of and protest against institutionalized policies, practices, and behaviors that threaten and violate the most basic of human rights, and serve as obstacles to achieving a more inclusive society. 

Perhaps most contextually impactful to survey responses, however, are prominent disturbing events that impacted the UMass Amherst campus directly over the first few weeks of the fall 2021 semester. First, an anonymous racist email was sent to members of a number of Black-centered student groups. Described by Vice Chancellor Nefertiti Walker as “vile, blatantly racist, and violently offensive,” the email remains of unknown origin despite the university’s hiring of an investigative team specializing in digital forensics. Additional anti-Black hate messages were sent via “Contact Us” online forms associated with registered student organizations, and a group of Black students was targeted by an occupant of a moving vehicle who yelled a racist epithet. Second, an alleged sexual assault at a campus fraternity prompted a sizeable student protest against sexual and gender-based violence that turned violent when participants broke windows at the fraternity house and flipped an on-site vehicle. This protest spurred a larger conversation about the Survivor’s Bill of Rights, and coincided with a number of similar protests against sexual violence on other college campuses. These incidents raised both campus and national attention to racism and sexual violence on the UMass Amherst campus and were very much a part of campus discourse when the climate survey was administered in November 2021.  

Methods 

The confidential online survey was administered primarily from November 3rd to 29th via Qualtrics. Efforts to maximize survey participation included a “prenotice” email; five reminder emails (to non-participants only); Moodle availability of students’ individualized survey links; a robust advertising campaign led by University Relations that featured video communications, posters, banners, electronic displays; paper versions translated into nine languages; optimization of question formats to enable participants to take the survey on their phone. 

The CCS achieved robust response rates for three of the four target populations: undergraduates (42%), graduate students (39%), and faculty (49%). The tables below compare demographic characteristics of survey participants to those of the corresponding target population (use control in upper right to select population). The tables for undergraduates, graduate students, and faculty show that for each of these three groups, there is a close demographic match, suggesting that a high level of demographic representativeness was achieved. (Please note that these tables, by necessity, are based on university database information.) That said, women are slightly overrepresented across all three populations. Among undergraduates, students in CNS majors are slightly overrepresented and ISOM students are slightly underrepresented. Among faculty, tenure-system faculty and professors are overrepresented.  

 

Relatively early in the survey administration process, OAPA researchers determined that participation among staff in Service/Maintenance (SM) and Skilled Crafts (SC) positions was lagging substantially relative to other staff classification groups (i.e., Administrator, Professional, Clerical/Secretarial, Technical/Paraprofessional). Outreach to the units employing the bulk of these staff revealed that a severe employee shortage stemming from the pandemic was one obstacle to participation because overextended staff were not able to devote time to taking the survey while on shift. Other likely deterrents to participation include low morale and trust in university administration, and a lack of ready access to email (the university’s primary communication mechanism). In an effort to increase the participation rate among SM and SC workers, the Staff version of the CCS remained open through December 15th, two weeks past the deadline for students and faculty. Unfortunately, this effort did not help to increase participation among SM and SC staff. 

Ultimately, participation among Service/Maintenance (SM) and Skilled Crafts (SC) staff members proved to be very low – 10 percent and 19 percent, respectively. Because SM and SC staff constitute more than one-third of all staff at the university (N=1819), the overall response rate for staff (39%) is lower than it would have been had SM and SC participation rates approximated those of other staff groupings. But more importantly, low participation among SM and SC staff negatively impacted the demographic representativeness of staff survey results for both the university overall, and for Administration and Finance (A&F) specifically, the executive area in which the vast majority of SM and SC staff work.  

As illustrated in the demographic table for staff overall (use control in upper right to select), SM staff are extremely underrepresented among survey participants: although they comprise 31% of all staff, they represent only 8% of survey participants. Because this low participation was highly concentrated in one executive area, A&F staff are very underrepresented among survey participants: although 42% of all staff are located in A&F, A&F staff represent only 22% of survey participants. Additional consequences are that Asian staff and men are very underrepresented among survey participants (and, conversely, White staff and women are overrepresented). 

The CCS research team determined that representation of SM and SC staff in the data set was so inadequate that it compromised the accuracy of staff results for the university overall, as well as for A&F. The research team therefore made the difficult decision to exclude A&F staff data from all quantitative reports on the staff results. Because survey participation rates for staff in nearly all of the university’s other nine executive areas were quite robust (e.g., 53% for Academic Affairs and SACL, 48% for Research and Engagement) the staff data for these areas are demographically representative and worthy of sharing. The second demographic table for staff, which excludes A&F (use control in upper right to select), shows that the demographic characteristics of survey participants closely match those of the target population across most categories. A few exceptions are that women are overrepresented by 6 percentage points, SM staff are underrepresented by 6 percentage points, and White staff are overrepresented by 5 percentage points. 

Measurement of Social Identity Aspects 

Awareness and consideration of social identity aspects, including how they intersect, is key to understanding our campus climate and to considering the multiplicity of perspectives and experiences within the campus community. It is important, therefore, to discuss differences between the way particular social identity aspects are measured and recorded in the university’s database compared to the campus climate survey itself. 

As explained in the 2016 Campus Climate Survey Abridged Report, the federal government requires all colleges and universities that receive government funding to gather and report annually a wide array of student and employee demographic characteristics. Consequently, UMass Amherst possesses considerable institutional data pertaining to the demographics of undergraduates, graduate students, staff, and faculty. The population statistics included in “UMass at a Glance” provide an extensive illustration of “who we are” as a community. This demographic summary includes gender, race/ethnicity, age, staff and faculty role, student residency, college/school affiliation, and more. However, an important limitation of some of these data is that they conform to established federal reporting standards that are not aligned with more contemporary conceptualizations of social identity aspects. For example, the federal government requires the university to report gender using the categories “male” and “female,” and fails to recognize the breadth of gender identities with which people identity, such as trans woman, trans man, nonbinary, and questioning. Additionally, the university databases for students, staff, and faculty do not include data about many important social identity aspects, including sexual orientation, disability, religion, and political viewpoint. 

The university’s Campus Climate Survey aims to extend knowledge of the social identity aspects of our community members in two main ways: 1) by utilizing more inclusive and conceptually progressive categories (relative to federal reporting categories) to measure race/ethnicity, gender identity, and sexual orientation, and 2) by gathering data on social identity characteristics not currently incorporated in the institution’s student and employee databases.