In college admissions, grades and test scores don’t tell the whole story. There is a growing need for scalable tools to measure personal qualities like perseverance, leadership, and intrinsic motivation. This study explored the validity of a RoBERTa model, originally fine-tuned on 150-word extracurricular essays, in assessing personal qualities from full-length college admissions essays. Across a sample of 1,724 essays, AI-generated scores showed strong alignment with human ratings (mean r = .60, range = .47–.73), while remaining largely uncorrelated with demographic characteristics and academic metrics such as GPA and SAT scores. These findings demonstrate the potential of AI models to provide efficient, interpretable, and privacy-preserving assessments of personal qualities.