A Study of LLM-Powered Student Query Support

Authors

  • Cecilia Delgado-Solorzano Clemson University
  • Elias Tzoc Clemson University
  • Suzanne Rook Clemson University
  • Christopher Vinson Clemson University
  • Carlos Toxtli Clemson University

DOI:

https://doi.org/10.47756/aihc.y9i1.141

Keywords:

Learning, Large Language Models, User Study

Abstract

In this paper, we explore the use of Large Language Models (LLMs) to help students improve their information-seeking skills while encouraging the use of references to aid library literacy efforts. This study aims to expand the reach of library support by introducing an approach that leverages the capabilities of LLMs and well-structured prompts. Our approach begins with surveying the current changes students have faced in the last two years concerning their study habits and how they search for information. We subsequently propose a multi-step system prompt, referred as prompting architecture, for foundational and instructed LLMs. The proposed prompt architecture powers a web application named LibRef. We explore the adaptability of the prompting architecture to different information retrieval needs by refining search prompts and providing academic references. A field experiment is conducted using LibRef in academic settings. Our results suggest that the use of LibRef enhances students’ academic information-seeking experience. Our research underscores the potential of prompting architectures in procedural refinement of academic queries from students. We believe our findings can provide valuable insights on the current capabilities of LLMs for instructing students to provide more targeted prompts as well as incentivize the use of references.

Downloads

Download data is not yet available.

References

Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big?. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 610–623. DOI: https://doi.org/10.1145/3442188.3445922

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.

Kun Huang, Xiaoyu Wang, Shichao Luo, Qiuping Su, and Lei Li. 2022. What difficulties did the college students encountered in information seeking during the COVID-19 pandemic? Data and Information Management 6, 2 (2022), 100005. DOI: https://doi.org/10.1016/j.dim.2022.100005

James R Lewis. 1995. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction 7, 1 (1995), 57–78. DOI: https://doi.org/10.1080/10447319509526110

Isaiah Michael Omame and Juliet C Alex-Nmecha. 2020. Artificial intelligence in libraries. In Managing and adapting library information services for future users. IGI Global, 120–144. DOI: https://doi.org/10.4018/978-1-7998-1116-9.ch008

Laria Reynolds and Kyle McDonell. 2021. Prompt programming for large language models: Beyond the few-shot paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1–7. DOI: https://doi.org/10.1145/3411763.3451760

Sei-Ching Joanna Sin and Kyung-Sun Kim. 2008. Use and non-use of public libraries in the information age: A logistic regression analysis of household characteristics and library services variables. Library & information science research 30, 3 (2008), 207–215. DOI: https://doi.org/10.1016/j.lisr.2007.11.008

JD Zamfirescu-Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang. 2023. Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. DOI: https://doi.org/10.1145/3544548.3581388

Downloads

Published

2024-11-30

How to Cite

[1]
Delgado-Solorzano, C. et al. 2024. A Study of LLM-Powered Student Query Support. Avances en Interacción Humano-Computadora. 9, 1 (Nov. 2024), 21–25. DOI:https://doi.org/10.47756/aihc.y9i1.141.

Issue

Section

Research Papers

Similar Articles

1 2 3 4 5 6 7 8 9 > >> 

You may also start an advanced similarity search for this article.