arXiv preprint ·
Incremental Predictive Value of LLM Tutoring Interactions for Student Performance
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
We evaluate whether semester-level interaction features from a large language model tutor improve assignment-score prediction beyond constant baselines. StudyChat provides 16,851 student-assistant exchanges from two offerings of an undergraduate artificial intelligence course, with outcomes matched to 175 of 203 student-semester records. Six fitted estimators and training-fold mean and median baselines share repeated nested cross-validation. After averaging each student's held-out predictions across ten repeats, Gradient Boosting achieves mean absolute error of 3.023 grade points, versus 3.138 for the median baseline. The paired difference is -0.114, with a conditional 95% bootstrap interval of [-0.245,0.013]. Representation changes and a semester-specific median yield similarly small gains. In the lowest observed-score quartile, however, boosting's error is 7.30 points, versus 8.07 for the median and 6.67 for the mean; Ridge has the lowest fitted-model error at 6.37 despite its worse overall error of 3.331. Scores concentrate near the ceiling: 160 students (91.4%) score above 90, and the lowest quartile contributes 62.1% of boosting's total absolute error. This retrospective comparison shows how baseline choice and error distribution affect predictive value, informing evaluation of student-support tools without establishing a learning effect.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computation & Language / NLP
Propasafe-Hybrid: A Text-Based Hybrid Propaganda Detection Tool
The Policies of Programming Language Ecosystems: A "Qualitative" Analysis
Learning by Collaborating in the Age of AI: Research Mentorship for Polymathic, Workflow-Centered Science
FraudSociety: A Generative Agent-Based Society for the Financial-Fraud Economy
A Battery for AI Propensity Measurement: Rubrics, Synthetic Item Banks, and Annotations
Get the few papers worth your time in Computation & Language / NLP every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.