Pre-Rate My Professor: Predicting Course Ratings And Response Rates From Lms Activity In College Courses

Richard T. Scruggs, University of Pennsylvania

Abstract

College teaching is primarily assessed through the use of course ratings, which are expected to act as both summative and formative feedback. Considering the significance of teaching in academia and the amount of time professors spend on teaching and related activities, it is particularly important that ratings are effective formative feedback. In this study, methods from learning analytics and data mining are used in an effort to predict course ratings and response rates on ratings surveys from students’ activity in the course learning management system, with the goal of making predicted ratings available to faculty early.

Regression and classification methods used in this study included linear and logistic regression, random forests, and gradient boosting and features were chosen based on their inclusion in earlier studies predicting individual student success and motivation. However, none of the models created for this study were not able to accurately predict either course ratings or response rates on either the entire data sample or a subset of classes with higher LMS activity. This may have been caused by difficulties with aggregating features or outcome variables to the class level, which was necessary due to the confidentiality of the student ratings. It may also result from the complexity of good college teaching: unlike individual student grades or motivation, which have been successfully predicted, there are many successful and unsuccessful forms of teaching.