Interaction Design (IxD) of an Intelligent Tutor for Programming Learning Based on LLM
DOI:
https://doi.org/10.47756/aihc.y9i1.137Keywords:
Interaction Design, Generative Artificial Intelligence, Computer Science Education, Large Language Models, Intelligent Tutoring SystemsAbstract
The emergent behavior of automatic programming exhibited by Large Language Models (LLMs) has raised uncertainty about the future of programming and its teaching. To better understand this phenomenon, we conducted a field study with programming instructors and students that informed the design of an intelligent tutor to integrate Generative Artificial Intelligence (GAI) into the educational environment. The resulting tool, EVA-Tutor (Virtual Learning Environment), supports the teaching and learning process of programming by establishing bidirectional communication between the student and the LLM through a GPT-4 API and a set of prompts designed to guide and motivate the student with personalized feedback. Rather than solving the problem for the student, the tool helps direct them toward solving it independently. A preliminary evaluation with students and instructors provides evidence of EVA-Tutor's utility and ease of use for problem-solving, knowledge acquisition, and the development of programming skills.
Downloads
References
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., ... & McGrew, B. (2023). Gpt-4 technical report. arXiv.org, 2303.08774. https://doi.org/10.48550/arXiv.2303.08774
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., . . . Amodei, D. (2020). Language Models are Few-Shot Learners. Advances in neural information processing systems, 33, 1877-1901. https://doi.org/10.48550/arXiv.2005.14165
Chen, E., Huang, R., Chen, H. S., Tseng, Y. H., & Li, L. Y. (2023). GPTutor: a ChatGPT-powered programming tool for code explanation. International Conference on Artificial Intelligence in Education (pp. 321-327). Cham: Springer Nature Switzerland. https://doi.org/10.48550/arXiv.2305.01863 DOI: https://doi.org/10.1007/978-3-031-36336-8_50
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159. https://doi.org/10.1037/0033-2909.112.1.155 DOI: https://doi.org/10.1037//0033-2909.112.1.155
Dwivedi, Y.K., Kshetri, N., Hughes, L. et al. (2023). Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal Of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642 DOI: https://doi.org/10.1016/j.ijinfomgt.2023.102642
Geng, C., Zhang, Y., Pientka, B., & Si, X. (2023). Can ChatGPT pass an introductory level functional Language programming course? arXiv.org, 2305.02230. https://doi.org/10.48550/arXiv.2305.02230
Jacques, L. (2023). Teaching CS-101 at the Dawn of ChatGPT. ACM Inroads, 14(2), 40-46. https://doi.org/10.1145/3595634 DOI: https://doi.org/10.1145/3595634
Jiao, H., Peng, B., Zong, L., Zhang, X., & Li, X. (2024). Gradable ChatGPT Translation Evaluation. arXiv.org, 2401.09984. https://doi.org/10.48550/arXiv.2401.09984
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022). Large Language Models are Zero-Shot Reasoners. Advances in neural information processing systems, 35, 22199-22213. https://doi.org/10.48550/arXiv.2205.11916
Qureshi, B. (2023). Exploring the Use of ChatGPT as a Tool for Learning and Assessment in Undergraduate Computer Science Curriculum: Opportunities and Challenges. arXiv.org, 2304.11214. https://doi.org/10.48550/arxiv.2304.11214
Rahman, M. M., & Watanobe, Y. (2023). ChatGPT for Education and Research: Opportunities, Threats, and Strategies. Applied Sciences, 13(9), 5783. https://doi.org/10.3390/app13095783 DOI: https://doi.org/10.3390/app13095783
Rajabi, P., Taghipour, P., Cukierman, D., & Doleck, T. (2023). Exploring ChatGPT’s impact on post-secondary education: A qualitative study. Proceedings of the 25th Western Canadian Conference on Computing Education (Article 9, 1-6). Association for Computing Machinery. https://doi.org/10.1145/3593342.3593360 DOI: https://doi.org/10.1145/3593342.3593360
Richards, M., Waugh, K., Slaymaker, M., Petre, M., Woodthorpe, J., & Gooch, D. (2023). Bob or Bot: Exploring ChatGPT’s answers to University Computer Science Assessment. ACM Transactions On Computing Education (vol. 25, pp. 1-32). https://doi.org/10.1145/3633287 DOI: https://doi.org/10.1145/3633287
Sifaleras, A., & Lin, F. (2024) Generative Intelligence and Intelligent Tutoring Systems. En: Lecture notes in computer science. Springer. https://doi.org/10.1007/978-3-031-63028-6 DOI: https://doi.org/10.1007/978-3-031-63028-6
Su, Y., Lin, Y., & Lai, C. (2023). Collaborating with ChatGPT in argumentative writing classrooms. Assessing Writing, 57, 100752. https://doi.org/10.1016/j.asw.2023.100752 DOI: https://doi.org/10.1016/j.asw.2023.100752
Yang, X., Wang, Q., & Lyu, J. (2023). Assessing ChatGPT’s Educational Capabilities and Application Potential. ECNU Review of Education, 0(0). https://doi.org/10.1177/20965311231210006 DOI: https://doi.org/10.1177/20965311231210006
Wang, T., Díaz, D. V., Brown, C., & Chen, Y. (2023). Exploring the Role of AI Assistants in Computer Science Education: Methods, Implications, and Instructor Perspectives. Symposium on Visual Languages and Human-Centric Computing (pp. 92-102). IEEE. https://doi.org/10.48550/arXiv.2306.03289 DOI: https://doi.org/10.1109/VL-HCC57772.2023.00018
Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2022). Self-Consistency Improves Chain of Thought Reasoning in Language Models. arXiv.org, 2203.11171. https://doi.org/10.48550/arXiv.2203.11171
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Advances in neural information processing systems. https://doi.org/10.48550/arXiv.2201.11903
White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., & Schmidt, D. C. (2023). A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. arXiv.org, 2301.11382. https://doi.org/10.48550/arXiv.2302.11382
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Oleksiy Levchuk, Carlos Sánchez, Nancy Pacheco, Jesús Favela

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
AMexIHC make every effort to ensure the accuracy and rigour of all the information (the "Content") contained in out publications. However, AMexIHC and our representatives make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by AMexIHC. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information.