Tool-Integrated Large Language Models for Stepwise Debugging and Feedback in Engineering Programming Laboratories

DOI:

https://doi.org/10.63646/RGIA7544

Keywords:

Engineering Programming Education, Large Language Models, Program Debugging, Execution Feedback, Formative Assessment, Learning Scaffolds, Secure Code Execution

Abstract

Engineering programming laboratories require students to interpret compiler output, form hypotheses, inspect program behavior, and revise code while instructors provide timely guidance. Conventional diagnostics can be precise yet difficult for novices to translate into a next action; unconstrained language-model responses can be fluent yet unsupported, overly complete, or inconsistent with a laboratory's learning goals. This article specifies a tool-integrated large language model (LLM) architecture that converts authorized compiler, test, static-analysis, and debugger evidence into staged, evidence-linked feedback. A sandboxed execution broker runs only instructor-approved tools and records immutable run snapshots. A diagnostic normalizer maps raw outputs to typed evidence objects. A structured response schema binds each observation or interpretation to evidence identifiers. Deterministic validators check schema, identifier existence, snapshot and run freshness, source integrity, and policy limits; they do not by themselves prove semantic entailment. Semantic support is treated as a separate evaluation question. The LLM may propose a qualified fault hypothesis and a hint at a policy-selected level, but cannot change code, grades, or exercise tests. A synthetic heat-conduction exercise illustrates the data flow and learner interaction. The article contributes a reference architecture, event and claim-level evidence schemas, a stepwise feedback protocol, security controls, a design traceability method, and a prospective verification plan. It reports no classroom deployment, participant data, or learning effects; these remain questions for implementation and future evaluation.

How to Cite

Luo, Z., & Xiong, Y. (2026). Tool-Integrated Large Language Models for Stepwise Debugging and Feedback in Engineering Programming Laboratories. Trends and Reviews of Technological Engineering in Education, 2(2), 52-88. https://doi.org/10.63646/RGIA7544

References

Ala-Mutka, K. (2005). A survey of automated assessment approaches for programming assignments. Computer Science Education, 15(2), 83–102. https://doi.org/10.1080/08993400500150747

Keuning, H., Jeuring, J., & Heeren, B. (2019). A systematic literature review of automated feedback generation for programming exercises. ACM Transactions on Computing Education, 19(1), Article 3. https://doi.org/10.1145/3231711

Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021). Evaluating large language models trained on code. arXiv. https://doi.org/10.48550/arXiv.2107.03374

Chen, X., Lin, M., Schärli, N., & Zhou, D. (2024). Teaching large language models to self-debug. In Proceedings of the International Conference on Learning Representations. https://doi.org/10.48550/arXiv.2304.05128

Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., & Narasimhan, K. (2024). SWE-bench: Can language models resolve real-world GitHub issues? In Proceedings of the International Conference on Learning Representations. https://doi.org/10.48550/arXiv.2310.06770

Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Lago, A. D., et al. (2022). Competition-level code generation with AlphaCode. Science, 378(6624), 1092–1097. https://doi.org/10.1126/science.abq1158

Robins, A., Rountree, J., & Rountree, N. (2003). Learning and teaching programming: A review and discussion. Computer Science Education, 13(2), 137–172. https://doi.org/10.1076/csed.13.2.137.14200

Qian, Y., & Lehman, J. (2017). Students’ misconceptions and other difficulties in introductory programming: A literature review. ACM Transactions on Computing Education, 18(1), Article 1. https://doi.org/10.1145/3077618

Lister, R., Adams, E. S., Fitzgerald, S., Fone, W., Hamer, J., Lindholm, M., McCartney, R., Moström, J. E., Sanders, K., Seppälä, O., Simon, B., & Thomas, L. (2004). A multi-national study of reading and tracing skills in novice programmers. ACM SIGCSE Bulletin, 36(4), 119–150. https://doi.org/10.1145/1041624.1041673

Lister, R., Simon, B., Thompson, E., Whalley, J. L., & Prasad, C. (2006). Not seeing the forest for the trees: Novice programmers and the SOLO taxonomy. In Proceedings of the 11th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education (pp. 118–122). https://doi.org/10.1145/1140124.1140157

Jadud, M. C. (2006). Methods and tools for exploring novice compilation behaviour. In Proceedings of the Second International Workshop on Computing Education Research (pp. 73–84). https://doi.org/10.1145/1151588.1151600

Luxton-Reilly, A., McMillan, E., Stevenson, E., Tempero, E., & Denny, P. (2018). Ladebug: An online tool to help novice programmers improve their debugging skills. In Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education (pp. 159–164). https://doi.org/10.1145/3197091.3197098

Ko, A. J., & Myers, B. A. (2008). Debugging reinvented: Asking and answering why and why not questions about program behavior. In Proceedings of the 30th International Conference on Software Engineering (pp. 301–310). https://doi.org/10.1145/1368088.1368130

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487

Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. https://doi.org/10.1080/03075070600572090

Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795

Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17(2), 89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x

VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221. https://doi.org/10.1080/00461520.2011.611369

Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023). Toolformer: Language models can teach themselves to use tools. In Advances in Neural Information Processing Systems, 36. https://doi.org/10.48550/arXiv.2302.04761

Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations. https://doi.org/10.48550/arXiv.2210.03629

Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. (2023). Self-refine: Iterative refinement with self-feedback. In Advances in Neural Information Processing Systems, 36. https://doi.org/10.48550/arXiv.2303.17651

Singh, R., Gulwani, S., & Solar-Lezama, A. (2013). Automated feedback generation for introductory programming assignments. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation (pp. 15–26). https://doi.org/10.1145/2499370.2462195

Denny, P., Kumar, V., & Giacaman, N. (2023). Conversing with Copilot: Exploring prompt engineering for solving CS1 problems using natural language. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education (pp. 1136–1142). https://doi.org/10.1145/3545945.3569823

Kazemitabaar, M., Chow, J., Ma, C., Ericson, B. J., Weintrop, D., & Grossman, T. (2023). Studying the effect of AI code generators on supporting novice learners in introductory programming. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Article 455. https://doi.org/10.1145/3544548.3580919

Sarsa, S., Denny, P., Hellas, A., & Leinonen, J. (2022). Automatic generation of programming exercises and code explanations using large language models. In Proceedings of the 2022 ACM Conference on International Computing Education Research (pp. 27–43). https://doi.org/10.1145/3501385.3543957

Koutcheme, C., Dainese, N., & Hellas, A. (2025). Direct repair optimization: Training small language models for educational program repair improves feedback. Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025), 564–581. https://doi.org/10.18653/v1/2025.bea-1.41

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems, 33, 9459–9474. https://doi.org/10.48550/arXiv.2005.11401

Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), Article 248. https://doi.org/10.1145/3571730

Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315–1325. https://doi.org/10.1080/02602938.2018.1463354

Chi, M. T. H., de Leeuw, N., Chiu, M.-H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18(3), 439–477. https://doi.org/10.1207/s15516709cog1803_3

Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.org/10.1207/s15516709cog1202_4

Prince, M. J., & Felder, R. M. (2006). Inductive teaching and learning methods: Definitions, comparisons, and research bases. Journal of Engineering Education, 95(2), 123–138. https://doi.org/10.1002/j.2168-9830.2006.tb00884.x

Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43–59. https://doi.org/10.1007/BF02505024

National Institute of Standards and Technology. (2022). Secure software development framework (SSDF) version 1.1: Recommendations for mitigating the risk of software vulnerabilities (NIST SP 800-218). https://doi.org/10.6028/NIST.SP.800-218

National Institute of Standards and Technology. (2024). Secure software development practices for generative AI and dual-use foundation models: An SSDF community profile (NIST SP 800-218A). https://doi.org/10.6028/NIST.SP.800-218A

National Institute of Standards and Technology. (2023). Artificial intelligence risk management framework (AI RMF 1.0) (NIST AI 100-1). https://doi.org/10.6028/NIST.AI.100-1