To help provide context for interpreting SRTI ratings, page two of the SRTI Section Report has always contained item means for the course as well as comparison group means controlling for class size and course level (undergraduate or graduate) at the department, school/college, and campus levels.
Another recurring theme in the feedback OAPA has received is a request for us to provide some kind of “confidence interval” to help individual faculty members and personnel committees understand what the true range of possible scores might be, and how to determine whether an observed difference in SRTI means was a difference worthy of attention. We made two changes to the SRTI Section Report to address this feedback:
1) We removed the bar chart that provided a visual representation of the course, department, school/college, and campus means because of concerns that it was encouraging over-interpretation of small differences in means;
2) We now include an interval measurement around each course mean rating called a “credible interval” (Figure 3). Conceptually similar to a confidence interval, the 90% credible interval is based on a different set of statistical assumptions that are more appropriate for SRTI data than traditional confidence interval construction.
Additional details on the construction and interpretation of the credible interval can be found in the following report: Interpreting Ratings in Context: The Credible Interval.
Figure 3: SRTI Section Report Excerpt – Mean Comparisons Within Class Size