Detection of AI-generated Writing in Students’ Assignments: A Comparative Analysis of Some Tools’ Reliability
Imen Hanane BENARAB
Higher School of Management and Digital Economy, Tipaza, Algeria
ibenarab@esgen.edu.dz
https://orcid.org/0009-0001-5070-6584
Abstract
This article answers the question: How reliable are current detection tools at identifying human or AI writing in students’ assignments? It aims to test the reliability of these tools through a comparative analysis of 10 of the most popular ones. This enabled us to assess the reliability and robustness of such tools in the face of various writing manipulations that some students may perform while producing their work to hide the artificial origin. We revealed the limitations of each tool taken individually and the need to combine several to overcome their deficiencies and use them to detect the presence of AI writings in students’ work.
Keywords: Artificial Intelligence, Generative AI, AI writing detector, students’ assignments, Open AI
DOI:
https://doi.org/10.70091/atras/AI.17
How to Cite this Paper :
Benarab, I. H. (2024). Detection of AI-Generated Writing in Students’ Assignments: A Comparative Analysis of Some Tools’ Reliability. Atras Journal, 5 (Special Issue), 271-286.
References:
Bourg, C. et al. (2024). Generative AI for trustworthy, open, and equitable scholarship. An MIT Exploration of Generative AI. https://doi.org/10.21428/e4baedd9.567bfd15.
Bozkurt, A. (2024). GenAI et al.: Cocreation, authorship, ownership, academic ethics and integrity in a time of generative AI. Open Praxis, 16(1).1-10. doi: 10.55982/openpraxis.16.1.654
Chaka, C. (2024). Reviewing the performance of AI detection tools in differentiating between AI-generated and human-written texts: A literature and integrative hybrid review. Journal of Applied Learning & Teaching, 7(1),1-12.
Driessen, K. (2024). Best AI detector: Free & premium tools compared https://www.scribbr.com/ai-tools/best-ai-detector/
Elkhatat, A.M., Elsaid, K., & Almeer, S. (2023). Evaluating the efficacy of AI content detection tools in differentiating between human and AI-generated text. International Journal for Educational Integrity, 19(1). 1-16. https://doi.org/10.1007/s40979-023-00140-5
Flanagin, A., Kendall-Taylor, J., & Bibbins-Domingo, K. (2023). Guidance for authors, peer reviewers, and editors on the use of AI, language models, and chatbots. JAMA. 330(8). 702–703.
Fleckenstein, J., Meyer, J., Jansen, T., Keller, S.D., Köller, O., & Möller, J. (2024). Do teachers spot AI? Evaluating the detectability of AI-generated texts among student essays. Computers and Education: Artificial Intelligence. Vol. 6. https://doi.org/10.1016/j.caeai.2024.100209.
Gabriele, N. (2024). Peut-on se fier aux détecteurs d’IA afin de contrôler un texte ou du contenu web ? Available at https://scribweb.fr/fiabilite-detecteurs-ia/
Grimes, M., Von Krogh, G., Feuerriegel, S., Rink, F., & Gruber, M. (2023). From scarcity to abundance: Scholars and scholarship in an age of generative Artificial Intelligence. Academy of Management Journal, 66(6), 1617-1624.
Hartshorne, D. (2024). The 6 best AI content detectors in 2024. Available at https://zapier.com/blog/ai-content-detector/
Hoke, T. (2024). Relying on generative AI in scholarly work has its pitfalls. ASCE. Available at https://www.asce.org/publications-and-news/civil-engineering-source/civil-engineering-magazine/issues/magazine-issue/article/2024/03/relying-on-generative-ai-in-scholarly-work-has-its-pitfalls
Hosseini, M., & Holmes, K. (2023). The evolution of library workplaces and workflows via generative AI. College & Research Libraries, 84(6),836-842. doi: 10.5860/crl.84.6.836
Jovanović, M. & Campbell, M. (2022). Generative Artificial Intelligence: Trends and prospects. Computer, 55(10). 107-112.
Kaswan, K.S., Dhatterwal, J.S., Malik, K., & Baliyan, A. (2023). Generative AI: A review on models and applications. International Conference on Communication, Security and Artificial Intelligence (ICCSAI). November. Greater Noida. India. doi:10.1109/ICCSAI59793.2023.10421601
Lo, L. S. (2023). AI policies across the globe: Implications and recommendations for libraries. IFLA Journal. 49(4). 645-649.
Manning, A.J. (2024). What is original scholarship in the age of AI? The Harvard Gazette. Available at https://news.harvard.edu/gazette/story/2024/05/how-is-generative-ai-changing-education-artificial-intelligence/
Marzuki, M., Widiati, U., Rusdin, D., Darwin, R., & Indrawati, I. (2023). The impact of AI writing tools on the content and organization of students’ writing: EFL teachers’ perspective. Cogent Education, 10(2). https://doi.org/10.1080/2331186X.2023.2236469
Rafner, J., Beaty, R.E., Kaufman, J.C., Lubart, T., & Sherson, J. (2023). Creativity in the age of generative AI. Nature Human Behaviour, 7(11), 1836-1838.
Roza, N. (2024). The best AI content detection tools and services for 2024. Available at https://nikolaroza.com/best-ai-content-detectors-tools-apps/
Sadasivan, V.S., Kumar, A., Balasubramanian, S., Wang, W., & Feizi, S. (2023). Can AI-generated text be reliably detected? ArXiv, abs/2303.11156.
Schuster, T., Schuster. R., Shah, D.J., & Barzilay, R. (2020). The limitations of stylometry for detecting machine-generated fake news. Computational Linguistics, 46(302). 1-18. doi:10.1162/coli_a_00380
UNESCO. (2024). Guidance for generative AI in education and research. Available at https://www.unesco.org/en/articles/guidance-generative-ai-education-and-research
Walters, W. (2023). The effectiveness of software designed to detect AI-generated writing: A comparison of 16 AI text detectors. Open Information Science, 7(1), 20220158. https://doi.org/10.1515/opis-2022-0158
Warner, A. (2023). The false promise of generative AI detectors. Available at https://multilingual.com/the-false-promise-of-generative-ai-detectors/
Copyright for all articles published in ATRAS belongs to the author. The authors also grant permission to the publisher to publish, reproduce, distribute, and transmit the articles. ATRAS publishes accepted papers under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) License. Authors submitting papers for publication in ATRAS agree to apply the CC BY-NC 4.0 license to their work. For non-commercial purposes, anyone may copy, redistribute material, remix, transform, and construct material in any media or format, provided that the terms of the license are observed and the original source is properly cited.