Automated vehicles must operate in complex environments that involve interactions among human drivers, cyclists, and pedestrians. Traditional rule based approaches may miss subtle factors that contribute to safety risks in these settings. This project will integrate multi modal datasets, including video, radar, LiDAR, GPS, and text based information from logs and incident reports, to train large language models capable of analyzing and predicting potential risk scenarios. The research will create a unified pipeline that fuses sensor data with language inputs so that the model can generate structured assessments and clear explanations of conditions that may influence safety.
The project will test the system in simulation and controlled real world environments. Additional evaluations will include cybersecurity challenges and human machine interface testing to ensure that the system is robust, practical, and informative for users. The final framework will support transportation agencies, developers, and industry partners in understanding and addressing AV safety challenges by providing tools for improved risk detection, situational reasoning, and communication.