Ask any teacher: Education is not a linear process. It’s not as simple as “student gathers information, processes it, and applies it.” In any classroom, there are questions and errors, sometimes even arguments. Different students learn differently. Education can be messy, even chaotic, because it’s a human process, and humans are messy and chaotic. 

Image

That means there’s no one-size-fits-all method for teaching, which can make training teachers difficult. But researchers in the Manning College of Information and Computer Sciences (CICS) at the UMass Amherst have received a grant from the National Science Foundation to develop simulated student agents, which are AI systems designed to act the way real students do in learning situations. 

“We are exploring how to use AI to simulate student behavior,” said Andrew Lan, associate professor and the CICS undergraduate computer science program director. “Applications include, for example, tutor training, to help teachers anticipate student errors and promote better engagement and motivation.”

Current AI models can’t quite achieve that. “GenAI models, such as LLMs, are not good at mimicking real student learning behavior yet,” Lan said. 

Existing student models are only designed for passive learning activities such as question answering; this project will create truly interactive student models that can hold conversations with teachers-in-training and engage with material in individualized ways, like real students. The result will be better preparation for teachers and more effective educational content.

The current project focuses on simulating middle-school math students to help human educators – tutors, teachers and curriculum designers, as well as researchers – understand how individual students respond to learning experiences; it will also help them test approaches to instruction. 

The researchers have three goals for this project: to reproduce realistic student errors; to study how to infer and summarize engagement and understanding from open-ended learning activities, such as discussions with tutors and peers; and to study how simulated students can help evaluate and improve approaches to education. 

Experimental studies and co-design sessions – workshops where designers and end users collaborate – with pre-service teachers and other educational stakeholders will help the team devise solutions and evaluate how effectively the simulated students support decision making for education. 

Lan and colleagues expect the project to yield new student simulation and modeling algorithms, concrete evidence regarding the simulated students’ effectiveness for training teachers and perhaps publicly available tools to support future research on optimizing AI for pedagogical uses. 

“It will enable a highly scalable tutor/teacher preparation process and immediate feedback for learning content developers,” Lan said.

With current AI models, the scientific challenge is significant, Lan added. “So far, even state-of-the-art generative AI models perform horribly at making student-like errors and responding to feedback the way real students do,” he said. “But most importantly, we are excited about the potential practical benefit of high-fidelity student simulators, such as tutor training and even letting students engage in learning-by-teaching activities.”

The research could benefit both human and AI tutors, Lan added, “but our primary focus is on benefiting human tutors. We are going to run some studies with real human teachers interacting with AI-powered teachable student agents.”

Ultimately, the researchers hope, the technology will support a stronger STEM (science, technology, engineering and math) workforce in the future through enhanced education in mathematics.

Lan’s co-primary investigators are Jacob Whitehill, associate professor of computer science at Worcester Polytechnic Institute (WPI), and Stacy Shaw, assistant professor of social science and policy studies at WPI. 

This project is funded by the Research on Innovative Technologies for Enhanced Learning (RITEL) program that supports early-stage exploratory research in emerging technologies for teaching and learning.