Efficacy Perception Scale for the Use of Artificial Intelligence in Foreign Language Teaching: Validity and Reliability Study
DOI:
https://doi.org/10.46328/ijonse.5845Keywords:
Foreign language teaching, Artificial intelligence, Scale developmentAbstract
This study aims to develop a valid and reliable scale to measure teachers' self-efficacy perception towards the use of artificial intelligence (AI) in foreign language teaching. In the study, the scale development process started with a literature review and a draft of 39 items was created in line with expert opinions. As a result of pilot application and validity and reliability analyses, a final scale with 18 items was obtained. Exploratory and confirmatory factor analyses showed that the scale had a two-dimensional structure (Planning and Instruction and Measurement and Evaluation). According to the EFA results, the scale explained 76.75% of the total variance. Factor loadings ranged between .585 and 1.007 and item-total correlations were between .639 and .879, indicating that the scale items had sufficient discrimination. As a result of all these procedures, a valid and reliable scale was developed to measure the perception of competence in using artificial intelligence in foreign language teaching.
References
Bandura, A. (1997). Self-efficacy: The exercise of control. Freeman.
Blake, R. (2013). Brave new digital classroom: Technology and foreign language learning. Georgetown University Press.
Bloch, F., Jackson, M.O., & Tebaldi, P. (2023). Centrality measures in networks. Social Choice and Welfare, 61, 413-453. https://doi.org/10.1007/s00355-023-01456-4
De Freitas, S. (2018). Are Games Effective Learning Tools? A Review of Educational Games. Educational Technology and Society, 21, 74-84.
Derakhshan, A., & Hasanabbasi, S. (2015). Social networks for language learning. Theory and Practice in Language Studies, 5(5), 1090-1095, http://dx.doi.org/10.17507/tpls.0505.
Dunn T. J., Baguley T, & Brunsden V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105, 399-412. https://doi.org/10.1111/bjop.12046
Epskamp, S., Borsboom, D., & Fried, E. I. (2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50(1), 195–212. https://doi.org/10.3758/s13428-017-0862-1
Ertmer, P. A., & Ottenbreit-Leftwich, A. (2010). Teacher technology change: How knowledge, beliefs, and culture intersect. Journal of Research on Technology in Education, 42(3), 255-284. https://doi.org/10.1080/15391523.2010.10782551
European Union Agency for Fundamental Rights. (2022). Bias in algorithms – Artificial intelligence and discrimination. https://fra.europa.eu/en/publication/2022/bias-algorithm
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104
Gerbing, D. W., Janet G. H., & Elizabeth B. F. (1994). A large-scale second-order structural equation model of the ınfluence of management participation on organizational planning benefits. Journal of Management 20 (4), 859–885. https://doi.org/10.1016/0149-2063(94)90033-7.
Ghosh, S. S. (2024). Assessing the Efficacy of Intelligent Tutoring Systems in Language Teacher Education: A Quantitative Study. Computers in the Schools, 1–24. https://doi.org/10.1080/07380569.2024.2435308
Golonka, E. M., Bowles, A. R., Frank, V. M., Richardson, D. L., & Freynik, S. (2014). Technologies for foreign language learning: A review of technology types and their effectiveness. Computer Assisted Language Learning, 27(1), 70-105. https://doi.org/10.1080/09588221.2012.700315
Heift, T., & Schulze, M. (2007). Errors and intelligence in computer-assisted language learning. Parsers and pedagogues. Routledge, https://doi.org/10.4324/9780203012215
Holmes, W., Bialik, M., & Fadel, C. (2019). Artificial intelligence in education: Promises and implications for teaching and learning. The Center for Curriculum Redesign.
Jiang, M. Y.-C., Jong, M. S.-Y., Lau, W. W.-F., Chai, C.-S., & Wu, N. (2021). Using automatic speech recognition technology to enhance EFL learners’ oral language complexity in a flipped classroom. Australasian Journal of Educational Technology, 37(2), 110–131. https://doi.org/10.14742/ajet.6798
Kartal, S., & Bardakçı, M. (2018). Ölçek geliştirme çalışmalarında dikkat edilmesi gereken istatistiksel kriterler. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 9(2), 123-140. https://doi.org/10.21031/epod.395937
Karagöz, Y. (2017). SPSS ve AMOS uygulamalı nitel-nicel-karma bilimsel araştırma yöntemleri ve yayın etiği. Ankara: Nobel Akademik Yayıncılık.
Kline, R.B. (2011). Principles and practice of structural equation modeling. New York: The Guilford Press.
Kukulska-Hulme, A., & Shield, L. (2008). An overview of mobile-assisted language learning: From content delivery to supported collaboration and interaction. ReCALL, 20(3), 271-289. https://doi.org/10.1017/S0958344008000335
Lin, C.C.; Huang, A.Y., & Lu, O.H. (2023). Artificial intelligence inintelligent tutoring systems toward sustainable education: A systematicreview. Smart Learning Environments, 10(1), DOI: 10.1186/s40561-023-00260-y
Lin, T. J., & Lan, Y. J. (2015). Language Learning in Virtual Reality Environments: Past, Present, and Future. EducationalTechnology & Society, 18 (4), 486–497
Language Learning in Virtual Reality Environments: Past, Present, and Future.
Luckin, R., Holmes, W., Griffiths, M., & Forcier, L. B. (2016). Intelligence unleashed: An argument for AI in education. Pearson Education.
Marsh, H.W., Hau, K.-T., & Grayson, D. (2005). Goodness of fit evaluation in structural equation modeling. A. Maydeu-Olivares & J. McArdle (Eds.), Psychometrics. A festschrift to Roderick P. McDonald. Erlbaum, Hillsdale, NJ (2005), pp. 275-340
Onesi-Ozigagun, O., Ololade, Y.J., Eyo-Udo, N.L. & Ogundipe, D.O. (2024). Revolutionizingeducation through AI: a comprehensive review of enhancing learning experiences. International Journal of Applied Research in Social Sciences, 6(4), 589-607, DOI: 10.1186/s40561-023-00260-y
Paladines, J., & Ramirez, J. (2020). A systematic literature review of intelligent tutoring systems withdialogue in natural language. IEEE Access, 8, 164246–164267.
https://doi.org/10.1109/ACCESS.2020.3021383
Qiu, X.b., Shan, C., Yao, J. & Fu, Q.K. (2024). The effects of virtual reality on EFL learning: A meta-analysis. Educ Inf Technol 29 (2), 1379–1405 https://doi.org/10.1007/s10639-023-11738-0
Robinaugh, D.J., Millner, A.J., & McNally, R.J., 2016. Identifying highly influential nodes in the complicated grief network. J Abnorm Psychol 125, 747–757. https://doi.org/10.1037/abn0000181
Sack, O. & Röcker, C. (2013). Privacy and security issues in technology-enhanced foreign language learning environments: Exlporing users’ knowledge about technological processes of diverse user groups. Universal Journal of Psychology, 1(2), 72-83. ttps://doi.org/10.13189/ujp.2013.010207
Scherer, R., Siddiq, F., & Tondeur, J. (2019). The technology acceptance model (TAM): A meta-analytic structural equation modeling approach to explaining teachers’ adoption of digital technology in education. Computers & Education, 128, 13-35. https://doi.org/10.1016/j.compedu.2018.09.009
Selwyn, N. (2021). Education and Technology: Key Issues and Debates. Bloomsbury Publishing.
Shadiev R, & Liu J. (2023). Review of research on applications of speech recognition technology to assist language learning. ReCALL, 35(1):74-88. doi:10.1017/S095834402200012X
Shermis, M.D., & Burstein, J. (Eds) (2013). Handbook of Automated Essay Evaluation: Current applications and new directions. (2013). Journal of Writing Research, 5(2), 239-243. https://doi.org/10.17239/jowr-2013.05.02.4
Teo, T. (2008). Pre-service teachers’ attitudes towards computer use: A Singapore survey. Australasian Journal of Educational Technology, 24(4), 413-424. https://doi.org/10.14742/ajet.1201
Thi-Nhu Ngo T, Hao-Jan Chen, H. & Kuo-Wei Lai, K. (2024). The effectiveness of automatic speech recognition in ESL/EFL pronunciation: A meta-analysis. ReCALL, 36(1):4-21. doi:10.1017/S0958344023000113
Tondeur, J., van Braak, J., Ertmer, P. A., & Ottenbreit-Leftwich, A. (2016). Understanding the relationship between teachers’ pedagogical beliefs and technology use in education: A systematic review of qualitative evidence. Educational Technology Research Development, 65(3), 555-575. https://doi.org/10.1007/s11423-016-9492-z
Van Borkulo, C. D., van Bork, R., Boschloo, L., Kossakowski, J. J., Tio, P., Schoevers, R. A., Borsboom, D., & Waldorp, L. J. (2022). Comparing network structures on three aspects: A permutation test. Psychological Methods. https://doi.org/10.1037/met0000476
Venkatesh, V., & Morris, M. G. (2000). Why don't men ever stop to ask for directions? Gender, social influence, and their role in technology acceptance and usage behavior. MIS Quarterly, 24(1), 115-139. https://doi.org/10.2307/3250981
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – Where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 1-27. https://doi.org/10.1186/s41239-019-0171-0
Zhang J. (2025). Integrating chatbot technology into English language learning to enhance student engagement and interactive communication skills. Journal of Computational Methods in Sciences and Engineering. doi:10.1177/14727978241312992
Downloads
Published
Issue
Section
License
Copyright (c) 2026 International Journal on Studies in Education
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Articles may be used for research, teaching, and private study purposes. Authors alone are responsible for the contents of their articles. The journal owns the copyright of the articles. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of the research material.
The author(s) of a manuscript agree that if the manuscript is accepted for publication in the International Journal on Studies in Education (IJonSE), the published article will be copyrighted using a Creative Commons “Attribution 4.0 International” license. This license allows others to freely copy, distribute, and display the copyrighted work, and derivative works based upon it, under certain specified conditions.
Authors are responsible for obtaining written permission to include any images or artwork for which they do not hold copyright in their articles, or to adapt any such images or artwork for inclusion in their articles. The copyright holder must be made explicitly aware that the image(s) or artwork will be made freely available online as part of the article under a Creative Commons “Attribution 4.0 International” license.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
