Design and Evaluation of a BERT- and T5-Based Automated Essay Scoring and Feedback Framework
DOI:
https://doi.org/10.26877/asset.v8i4.2924Keywords:
Automated essay scoring, BERT T5, natural language processing, deep learning, supervised learningAbstract
Manual essay assessment is often time-consuming, subjective, and difficult to implement consistently in large-scale educational settings. Therefore, this study aims to develop and evaluate an integrated AES framework that combines essay scoring, spelling error detection, semantic relevance analysis, and automated feedback generation. The proposed framework integrates BERT for essay score prediction, pyspellchecker for spelling error detection, SBERT for semantic relevance analysis, and T5 for automated feedback generation. Experiments were conducted using a publicly available dataset consisting of 1,785 essays for model development and 592 essays for testing. The BERT model achieved strong predictive performance with an MAE of 0.2381, RMSE of 0.2847, and R² of 0.9513 on the test dataset, indicating high agreement with human scoring. The spelling detection module successfully identified linguistic errors, while the SBERT model produced relevance scores ranging from 0.58 to 0.95. The T5 model generated feedback that was consistent with essay scores, spelling quality, and semantic relevance. These findings demonstrate that integrating multiple NLP components can extend AES functionality beyond score prediction by providing comprehensive writing evaluation and formative feedback. Practically, the proposed framework supports scalable essay assessment and automated feedback generation. However, its generalizability is limited on a single-topic dataset and synthetic feedback data.
References
[1] P. Lagakis and S. Demetriadis, “Automated essay scoring: A review of the field,” in 2021 International Conference on Computer, Information and Telecommunication Systems (CITS), 2021, pp. 1–6. doi: https://doi.org/10.1109/CITS52676.2021.9618476.
[2] D. Ramesh and S. K. Sanampudi, “An automated essay scoring systems: a systematic literature review,” Artif. Intell. Rev., vol. 55, no. 3, pp. 2495–2527, 2022, doi: https://doi.org/10.1007/s10462-021-10068-2.
[3] H. Misgna, B.-W. On, I. Lee, and G. S. Choi, “A survey on deep learning-based automated essay scoring and feedback generation.,” Artif. Intell. Rev., vol. 58, no. 2, p. 36, 2025, doi: https://doi.org/10.1007/s10462-024-11017-5.
[4] A. Bilonikar, H. Chandra, V. Dapolikar, A. Halbe, and N. Mulla, “Automated Evaluation and Quiz Generation using Deep Learning and Transformers,” in 2026 IEEE International Conference for Convergence in Computing Technology (I3CTCON), 2026, pp. 1–6. doi: https://doi.org/10.1109/I3CTCON68242.2026.11507516.
[5] Zaki and M. Zayyanu, “R EVOLUTIONISING T RANSLATION T ECHNOLOGY : A C OMPARATIVE S TUDY OF V ARIANT T RANSFORMER M ODELS - BERT , GPT AND T5,” Computer Science & Engineering: An International Journal (CSEIJ), vol. 14, no. 3, pp. 15–27, 2024, doi: https://doi.org/10.5121/cseij.2024.14302.
[6] A. M. Elmassry, N. Zaki, N. Alsheikh, and M. Mediani, “A Systematic Review of Pretrained Models in Automated Essay Scoring,” IEEE Access, vol. 13, pp. 121902–121917, 2025, doi: https://doi.org/10.1109/ACCESS.2025.3584784.
[7] W. Xu, R. Mahmud, and W. L. Hoo, “A Systematic Literature Review: Are Automated Essay Scoring Systems Competent in Real-Life Education Scenarios?,” IEEE Access, vol. 12, pp. 77639–77657, 2024, doi: https://doi.org/10.1109/ACCESS.2024.3399163.
[8] S. Gombert et al., “From the Automated Assessment of Student Essay Content to Highly Informative Feedback: a Case Study,” Int. J. Artif. Intell. Educ., vol. 34, no. 4, pp. 1378–1416, 2024, doi: https://doi.org/10.1007/s40593-023-00387-6.
[9] H. Misgna, B.-W. On, I. Lee, and G. S. Choi, “A survey on deep learning-based automated essay scoring and feedback generation: H. Misgna et al.,” Artif. Intell. Rev., vol. 58, no. 2, p. 36, 2024, doi: https://doi.org/10.1007/s10462-024-11017-5.
[10] C. Myneni, V. M. Devadiga, V. K. K, N. S., and N. Z., “EduCheck: An AI-Driven Framework for Multi-Modal Assessment and Performance Analysis,” in 2025 3rd DMIHER International Conference on Artificial Intelligence in Healthcare, Education and Industry (IDICAIHEI), 2025, pp. 1–6. doi: https://doi.org/10.1109/IDICAIHEI65991.2025.11378451.
[11] D. Wang and J. Wang, “The impact mechanism of AES on improving English writing achievement,” Sci. Rep., vol. 15, no. 1, p. 3928, 2025, doi: https://doi.org/10.1038/s41598-025-88021-4.
[12] F. Li, X. Xi, Z. Cui, D. Li, and W. Zeng, “Automatic Essay Scoring Method Based on Multi-Scale Features,” 2023. doi: https://doi.org/10.3390/app13116775.
[13] R. H. Chassab, L. Q. Zakaria, and S. Tiun, “An optimized BERT fine-tuned model using an artificial bee colony algorithm for automatic essay score prediction,” PeerJ Comput. Sci., vol. 10, p. e2191, 2024, doi: https://doi.org/10.7717/peerj-cs.2191.
[14] E. Fianu, F. Amankwah-Sarfo, P. Ofori, J. K. Amoako, and H. Sumani, “From Traditional Machine Learning Models to Large Language Models: A Systematic Literature Review of Automated Essay Scoring,” SN Comput. Sci., vol. 7, no. 5, p. 406, 2026, doi: https://doi.org/10.1007/s42979-026-05028-y.
[15] T. Miao and D. Xu, “KWM-B: Key-information weighting methods at multiple scale for automated essay scoring with BERT,” Electronics (Basel)., vol. 14, no. 1, p. 155, 2025, doi: https://doi.org/10.3390/electronics14010155.
[16] J. Xue, X. Tang, and L. Zheng, “A Hierarchical BERT-Based Transfer Learning Approach for Multi-Dimensional Essay Scoring,” IEEE Access, vol. 9, pp. 125403–125415, 2021, doi: https://doi.org/10.1109/ACCESS.2021.3110683.
[17] N. Rasdianah, M. Madania, T. S. Tuloli, W. S. Abdulkadir, H. Ahmad, and T. B. A. Suwandi, “Studi Efek Samping Obat Antituberkulosis (OAT) Pada Pasien TB Paru,” Journal Syifa Sciences and Clinical Research, vol. 4, no. 3, 2022, doi: https://doi.org/10.37311/jsscr.v4i3.16657.
[18] A. Abdulkarem and A. Krivtsun, “An analysis of automated essay scoring frameworks,” Iraqi Journal for Applied Science, vol. 1, no. 3, pp. 71–81, 2024, doi: https://doi.org/10.69923/av1gt264.
[19] C. T. Lim, C. H. Bong, W. S. Wong, and N. K. Lee, “A comprehensive review of automated essay scoring (AES) research and development,” Pertanika J. Sci. Technol., vol. 29, no. 3, pp. 1875–1899, 2021, doi: https://doi.org/10.47836/pjst.29.3.27.
[20] Y. Nie, “Automated essay scoring with SBERT embeddings and LSTM-Attention networks,” PeerJ Comput. Sci., vol. 11, p. e2634, 2025, doi: https://doi.org/10.7717/peerj-cs.2634.
[21] S. Riyanto et al., “Employing SBERT for Essay Assessment,” in 2024 International Conference on Computer, Control, Informatics and its Applications (IC3INA), 2024, pp. 255–260. doi: https://doi.org/10.1109/IC3INA64086.2024.10732430.
[22] W.-C. Chang, F. X. Yu, Y.-W. Chang, Y. Yang, and S. Kumar, “Pre-training tasks for embedding-based large-scale retrieval,” arXiv preprint arXiv:2002.03932, 2020, doi: https://doi.org/10.48550/arXiv.2002.03932.
[23] S. Gombert et al., “From the automated assessment of student essay content to highly informative feedback: A case study,” Int. J. Artif. Intell. Educ., vol. 34, no. 4, pp. 1378–1416, 2024, doi: https://doi.org/10.1007/s40593-023-00387-6.
[24] U. Majeed, M. Jaffery, A. Nusrat, and S. R. Tahir, “Investigating The Effect Of Automated Writing Evaluation Through Deep Neural Network And Teachers’ Written Evaluation On English Writing Performance,” J. Vis. Exp., no. 231, p. e69995, 2026, doi: https://doi.org/10.3791/69995.
[25] C. Raffel et al., “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of machine learning research, vol. 21, no. 140, pp. 1–67, 2020, doi: https://doi.org/10.48550/arXiv.1910.10683.
[26] M. S. Gunawan and S. Karina, “Human Rights and Constitutional Supremacy in Indonesia: Normative Guarantees and Policy Disharmony in Natural Resource Management, Freedom of Expression, and Public Participation,” Journal of Law and Social Politics, vol. 4, no. 2, pp. 112–121, 2026, doi: https://doi.org/10.59261/jlsp.v4i2.106.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Advance Sustainable Science Engineering and Technology
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
