Language and Communication Students' Perceived Mastery in AI Chatbot Prompt Engineering: A Study on Vibe Coding in Educational Mobile Application Development
Keywords:
prompt engineering; AI chatbot; vibe coding; mobile application development; sustainable development educationAbstract
This study examines language and communication students’ self-perceived mastery of AI chatbot prompt engineering, with particular attention to “vibe coding” for educational mobile application development. In this study, vibe coding refers to the deliberate use of prompts to shape tone, style, audience orientation, and user-facing educational content. Using a pattern-based framework centred on roles, constraints, and examples, the study investigates how students combine structure and exploratory prompting. A quantitative cross-sectional survey was administered online during March–July 2025 (Semester 2). Complete responses from 120 undergraduates were analysed from approximately 126 invited students (analytic response rate = 95.2%). The questionnaire demonstrated excellent overall internal consistency (Cronbach’s ? = .965). Results indicate that students usually begin tasks with structured prompts but later move towards mixed or unstructured prompting styles, suggesting a control-then-explore sequence. Longer exposure to AI chatbots and more extensive prompt-engineering training were associated with higher self-perceived competency and output efficiency, whereas weekly usage frequency did not show statistically significant differences. Educational background was associated with prompting style and usage, but not with overall perceived competency or output efficiency, and gender differences were negligible. The findings suggest that scaffolded instruction in prompt engineering can support more confident and consistent student use of AI chatbots. Future research should triangulate self-reports with behavioural logs, archived prompts, and performance-based outputs.
https://doi.org/10.26803/ijlter.25.7.17
References
Ahmad, N., Alias, F. A., Hamat, M., & Mohamed, S. A. (2024). Reliability analysis: Application of Cronbach’s Alpha in research instruments. Journal of Computer and Mathematical Sciences. https://appspenang.uitm.edu.my/sigcs/2024-2/Articles/20244_ReliabilityAnalysis-ApplicationOfCronbachsAlphaInResearchInstruments.pdf
Borromeo, A. S., Manaloto, A. M., Santos, M. J. M. D., Antonio, R. P., Soyosa, M. D., & Wider, W. (2025). Harnessing generative AI in nursing education: A bibliometric review. Teaching and Learning in Nursing. https://doi.org/10.1016/j.teln.2025.04.014
Bujang, M. A., Omar, E. D., Foo, D. H. P., & Hon, Y. K. (2024). Sample size determination for conducting a pilot study to assess reliability of a questionnaire. Restorative Dentistry & Endodontics, 49(1), e3. https://doi.org/10.5395/rde.2024.49.e3
Cain, W. (2024). Prompting change: Exploring prompt engineering in large language model AI and its potential to transform education. TechTrends, 68(1), 47–57. https://doi.org/10.1007/s11528-023-00896-0
Fritsch, T. (2024). Chatbots: An overview of current issues and challenges. In Advances in Information and Communication (pp. 84–104). Springer. https://doi.org/10.1007/978-3-031-53960-2_7
Giray, L. (2023). Prompt engineering with ChatGPT: A guide for academic writers. Annals of Biomedical Engineering, 51(12), 2629–2633. https://doi.org/10.1007/s10439-023-03272-4
Jeon, J., & Lee, S. (2023). Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT. Education and Information Technologies, 28, 15873–15892. https://doi.org/10.1007/s10639-023-11834-1
Knoth, N., Tolzin, A., Janson, A., & Leimeister, J. M. (2024). AI literacy and its implications for prompt engineering strategies. Computers and Education: Artificial Intelligence, 6, 100225. https://doi.org/10.1016/j.caeai.2024.100225
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., … Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274
Kowang, T. O., Yew, L. K., Fei, G. C., & Hee, O. C. (2025). Determinants of artificial intelligence acceptance among undergraduates. International Journal of Evaluation and Research in Education, 14(4). https://doi.org/10.11591/ijere.v14i4.32565
Lee, D., & Palmer, E. (2025). Prompt engineering in higher education: A systematic review to help inform curricula. International Journal of Educational Technology in Higher Education, 22, 7. https://educationaltechnologyjournal.springeropen.com/articles/10.1186/s41239-025-00503-7
McGrath, C., Farazouli, A., & Cerratto Pargman, T. (2024). Generative AI chatbots in higher education: A review of an emerging research area. Higher Education, 89, 1533–1549. https://doi.org/10.1007/s10734-024-01288-w
Nyimbili, F., & Nyimbili, L. (2024). Types of purposive sampling techniques with their examples and application in qualitative research studies. British Journal of Multidisciplinary and Advanced Studies, 5(1), 90–99. https://doi.org/10.37745/bjmas.2022.0419
O’Dea, X., Ng, D. T. K., O’Dea, M., & Shkuratskyy, V. (2024). Factors affecting university students’ generative AI literacy: Evidence and evaluation in the UK and Hong Kong contexts. Policy Futures in Education. https://doi.org/10.1177/14782103241287401
OpenAI. (2025). Best practices for prompt engineering with the OpenAI API. OpenAI Help Center. https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-the-openai-api
Patterson, A., Frydenberg, M., & Basma, L. (2024). Examining generative artificial intelligence adoption in academia: A UTAUT perspective. Issues in Information Systems, 25(3), 238–251. https://iacis.org/iis/2024/3iis2024_238-251.pdf
Quamar, A. H., Schmeler, M. R., McCue, M., et al. (2023). Test–retest reliability of the Electronic Instrumental Activities of Daily Living Satisfaction Assessment (EISA): A cohort study. American Journal of Occupational Therapy, 77(6), 7706205140. https://doi.org/10.5014/ajot.2023.050285
Rokeman, N. R. M. (2024). Likert measurement scale in education and social sciences: Explored and explained. EDUCATUM Journal of Social Sciences, 10(1), 77–89. https://doi.org/10.37134/ejoss.vol10.1.7.2024
Siripipatthanakul, S., Muthmainnah, B., Asrifan, A., et al. (2023). Quantitative research in education. ResearchGate. https://www.researchgate.net/publication/369013292QuantitativeResearchinEducation
Toker Gokce, A., Deveci Topal, A., Kolburan Geçer, A., & Eren, C. D. (2024). Investigating the level of artificial intelligence literacy of university students using decision trees. Education and Information Technologies, 30, 6765–6784. https://doi.org/10.1007/s10639-024-13081-4
Wang, C. (2024). Exploring students’ generative AI-assisted writing processes: Perceptions and experiences from native and nonnative English speakers. Technology, Knowledge and Learning, 30, 1825–1846. https://doi.org/10.1007/s10758-024-09744-3
Walter, Y. (2024). Embracing the future of Artificial Intelligence in the classroom: The relevance of AI literacy, prompt engineering, and critical thinking in modern education. International Journal of Educational Technology in Higher Education, 21, 15. https://doi.org/10.1186/s41239-024-00448-3
White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer Smith, J., & Schmidt, D. C. (2023). A prompt pattern catalog to enhance prompt engineering with ChatGPT. arXiv. https://doi.org/10.48550/arXiv.2302.11382
Wong, T. A., Tan, K. T. L., Darmaraj, S. R., Loo, J. T. K., & Ng, A. H. H. (2025). Social capital development in online education and its impact on academic performance and satisfaction. Higher Education, Skills and Work Based Learning. https://doi.org/10.1108/heswbl-12-2023-0332
Woo, D. J., Wang, D., Yung, T., & Guo, K. (2024). Effects of a prompt engineering intervention on undergraduate students’ AI self-efficacy, AI knowledge, and prompt engineering ability: A mixed methods study. Education and Information Technologies. https://arxiv.org/pdf/2408.07302
Yang, Y., Wen, X., & Maidin, S. S. (2024). Generative AI tools in higher education emerging research: A bibliometric analysis of co citation and co word analysis. In Proceedings of the 2024 3rd International Conference on Artificial Intelligence and Education (ICAIE 2024). ACM. https://doi.org/10.1145/3722237.3722266
Yusoff, M. S. B., Arifin, W. N., & Hadie, S. N. H. (2021). ABC of questionnaire development and validation for survey research. Education in Medicine Journal, 13(1), 97–108. https://doi.org/10.21315/eimj2021.13.1.10
Zakariya, Y. F. (2022). Cronbach’s alpha in mathematics education research: Its appropriateness, overuse, and alternatives. Frontiers in Psychology, 13, 1074430. https://doi.org/10.3389/fpsyg.2022.1074430
Zamfirescu Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023). Why Johnny can’t prompt: How non AI experts try (and fail) to design LLM prompts. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23). https://doi.org/10.1145/3544548.3581388
Zhang, D., Yang, H., He, Y., & Guo, W. (2025). Modeling the relationships between secondary school students’ AI learning attitude, AI literacy and AI career interest. Education and Information Technologies. https://doi.org/10.1007/s10639-025-13715-1
Zhao, D. (2024). The impact of AI-enhanced natural language processing tools on writing proficiency: An analysis of language precision, content summarization, and creative writing facilitation. Education and Information Technologies, 30(1), 8055–8086. https://doi.org/10.1007/s10639-024-13145-5
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Nur Izzati Khairuddin, Muhammad Haziq Abd Rashid, Hairul Azhar Mohamad, Amir Lukman Abd Rahman, Pavithran Ravinthra Nath, Urai Salam, Maksetbay Mambetniyazov
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All articles published by IJLTER are licensed under a Creative Commons Attribution Non-Commercial No-Derivatives 4.0 International License (CCBY-NC-ND4.0).
