Title: Measuring Teachers’ AI Literacy through ChatGPT Prompt Engineering
Abdelbasset DOU
Mostaganem University, Algeria
abdelbasset.dou.etu@univ-mosta.dz
https://orcid.org/0000-0003-2633-6904
Kelthoume RIM
Khenchela University, Algeria
kelthoume.rim@univ-khenchela.dz
https://orcid.org/0009-0008-3752-4496
Abstract
While the potential of generative AI in education is widely acknowledged, gaps in educators’ AI-specific knowledge and skills pose significant challenges to successful integration. In the Algerian EFL teaching context, this study investigates the relationship between prompt engineering skills and the perceived effectiveness of AI-generated responses, as a subjective measure of teachers’ AI literacy. Understanding this relationship is crucial for informing the development of professional development programs tailored to enhance EFL teachers’ AI literacy and their ability to effectively utilize AI tools in the classroom. Forty-two EFL teachers, recruited through convenience sampling, participated in a prompt engineering task using ChatGPT, followed by a questionnaire assessing their perceptions of AI tools and their self-reported AI literacy. A specially designed rubric evaluated the quality of the teachers’ prompts, while a rating scale measured their satisfaction with ChatGPT’s responses. The results revealed a moderate positive correlation between prompt quality and the perceived effectiveness of AI-generated outputs. Teachers who crafted clearer, more specific, and strategically structured prompts were more likely to rate ChatGPT’s responses as valuable for EFL instruction. Qualitative analysis of teacher reflections revealed general technological confidence but also uncertainty and a lack of AI-specific literacy. This gap was evident in their difficulties optimizing prompts for desired outcomes and reliance on generalized phrasing. The study highlights the need for targeted professional development initiatives to improve EFL teachers’ AI literacy, thereby empowering them to harness the transformative potential of generative AI in language education.
Keywords: AI literacy, ChatGPT, EFL teachers, prompt engineering
How to Cite this Paper :
Dou, A., & Rim, K. (2025). Measuring Teachers’ AI Literacy through ChatGPT Prompt Engineering. Atras Journal, 6(2), 64-75
References:
Bender, S. M. (2024). Awareness of artificial intelligence as an essential digital literacy: ChatGPT and Gen-AI in the classroom. Changing English, 31(2), 161-174. https://doi.org/10.1080/1358684X.2024.2309995
Cain, W. (2024). Prompting change: exploring prompt engineering in large language model AI and its potential to transform education. TechTrends, 68(1), 47-57. https://doi.org/10.1007/s11528-023-00896-0
Evmenova, A. S., Regan, K., Mergen, R., & Hrisseh, R. (2024). Improving Writing Feedback for Struggling Writers: Generative AI to the Rescue?. TechTrends, 1-13. https://doi.org/10.1007/s11528-024-00965-y
Gregory, S. F. (2024). Empowering Teaching With Prompt Engineering: How to Integrate Curriculum, Standards, and Assessment for a New Age. In Transforming Education With Generative AI: Prompt Engineering and Synthetic Content Creation (pp. 239-260). IGI Global. https://doi.org/10.4018/979-8-3693-1351-0.ch012
Jia, F., Sun, D., & Looi, C. K. (2024). Artificial intelligence in science education (2013–2023): Research trends in ten years. Journal of Science Education and Technology, 33(1), 94-117. https://doi.org/10.1007/s10956-023-10077-6
Jiang, J., Vetter, M. A., & Lucia, B. (2024). Toward a ‘More-Than-Digital’AI Literacy: Reimagining Agency and Authorship in the Postdigital Era with ChatGPT. Postdigital Science and Education, 1-18. https://doi.org/10.1007/s42438-024-00477-1
Kong, S. C., Cheung, W. M. Y., & Zhang, G. (2021). Evaluation of an artificial intelligence literacy course for university students with diverse study backgrounds. Computers and Education: Artificial Intelligence, 2, 100026. https://doi.org/10.1016/j.caeai.2021.100026
Laupichler, M. C., Aster, A., Schirch, J., & Raupach, T. (2022). Artificial intelligence literacy in higher and adult education: A scoping literature review. Computers and Education: Artificial Intelligence, 3, 100101. https://doi.org/10.1016/j.caeai.2022.100101
Lee, U., Jung, H., Jeon, Y., Sohn, Y., Hwang, W., Moon, J., & Kim, H. (2023). Few-shot is enough: Exploring ChatGPT prompt engineering method for automatic question generation in English education. Education and Information Technologies, 29, 11483–11515. https://doi.org/10.1007/s10639-023-12249-8
Li, E., Li, S., & Yuan, X. (2022). Adoption and perception of artificial intelligence technologies by children and teens in education. In International Conference on Human-Computer Interaction (pp. 69-79). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-031-05643-7_5
Liu, T. C. (2022). A case study of the adaptive learning platform in a Taiwanese Elementary School: Precision education from teachers’ perspectives. Education and Information Technologies, 27(5), 6295-6316. https://doi.org/10.1007/s10639-021-10851-2
Lo, L. S. (2023). The CLEAR path: A framework for enhancing information literacy through prompt engineering. The Journal of Academic Librarianship, 49(4), 102720. https://doi.org/10.1016/j.acalib.2023.102720
Marvin, G., Hellen, N., Jjingo, D., & Nakatumba-Nabende, J. (2023). Prompt engineering in large language models. In International Conference on Data Intelligence and Cognitive Informatics (pp. 387-402). Singapore: Springer Nature Singapore. https://doi.org/10.1007/978-981-99-7962-2_30
Ranade, N., Saravia, M., & Johri, A. (2024). Using rhetorical strategies to design prompts: a human-in-the-loop approach to make AI useful. AI & Society, 1-22. https://doi.org/10.1007/s00146-024-01905-3
Sperling, K., Stenberg, C. J., McGrath, C., Åkerfeldt, A., Heintz, F., & Stenliden, L. (2024). In search of artificial intelligence (AI) literacy in Teacher Education: A scoping review. Computers and Education Open, 100169. https://doi.org/10.1016/j.caeo.2024.100169
Walter, Y. (2024). Embracing the future of Artificial Intelligence in the classroom: the relevance of AI literacy, prompt engineering, and critical thinking in modern education. International Journal of Educational Technology in Higher Education, 21(1), 1-29. https://doi.org/10.1186/s41239-024-00448-3.
Copyright for all articles published in ATRAS belongs to the author. The authors also grant permission to the publisher to publish, reproduce, distribute, and transmit the articles. ATRAS publishes accepted papers under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) License. Authors submitting papers for publication in ATRAS agree to apply the CC BY-NC 4.0 license to their work. For non-commercial purposes, anyone may copy, redistribute material, remix, transform, and construct material in any media or format, provided that the terms of the license are observed and the original source is properly cited.