UMassAmherst: Research, Educational Measurement, and Psychometrics (REMP) s
Research, Educational Measurement, and Psychometrics (REMP)

Student News: Poster Award for Hanwook Yoo

At the 2010 meeting of the American Educational Research Association (AERA), held in Denver, CO, AERA's Division D held its second annual In-Progress Research Gala. At this session, graduate students present posters of their research with an audience of interested researchers, including a panel of reviewers who will provide feedback on the studies.

This year, REMP doctoral student Hanwook Yoo was honored with the third-place award at the In-Progress Research Gala for his research study, entitled "Robustness of IRT-Based Single-Administration Estimates of the Consistency and Accuracy of Examinee Proficiency Classifications." Yoo's abstract follows:

"In an era of large-scale state assessments, it has become common for states to report single administration estimates of decision consistency (DC) and decision accuracy (DA). Since such DC and DA estimates exist based on the important work of researchers such as Livingston and Lewis (1995), they are used routinely by testing companies and the states, and they do provide the states with valuable technical information. However, because of one or two possible shortcomings, researchers are beginning now to generate item response theory (IRT) based solutions. These IRT-based approaches seem to be based on generally reasonable assumptions (i.e., test unidimensionality and model fit) with state test data. It seems fitting too to have an IRT-based approach because many testing companies and states are using IRT models and methods already to develop tests, to study bias, and to equate test scores. Since IRT model assumptions can be violated (e.g., some level of model misfit including test multidimensionality), the purpose of this study will be to investigate the robustness of the IRT-based approach for estimating DC and DA to departures in the data from the IRT model assumptions. Two other purposes will include (1) a review of the relevant research in which we will contrast true score and IRT-based approaches, and (2) a discussion of promising software for carrying out the analyses within an IRT framework. As we have not seen any usage to date of these IRT-based procedures, the findings should be quite important to the measurement field."

Also noteworthy: At the inaugural In-Progress Research Gala held in San Diego, CA, in 2009, REMP's Tia Sukin was awarded the first-place honor for her work, entitled "The Effect of Deleting Anchor Items on the Classification of Examinees: An Exploration of Item Flagging Criteria and Anchor Item Properties."



Resources

Center for Educational Assessment

Key Links

Contact

Find out more!