Artificial Intelligence and Multimodal Technologies in Psychology: A Scoping Review of Speech, Language, Gesture, Communication, and Human-Computer Interaction Research: (2021- 2026)
DOI:
https://doi.org/10.47941/japsy.3903Keywords:
Artificial Intelligence, Multimodal Artificial Intelligence, Human-Computer Interface, Recognition of Speech, Technologies for Assisting People with DisabilitiesAbstract
Purpose: This scoping review systematically captures and describes the existing research in psychology how AI and multimodal technologies are used for speech, language, gesture, communication, and human - computer interaction (HCI). It also analyzes the research on these issues published between January 2021 and June 2026.
Methodology: The review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) and the Joanna Briggs Institute (JBI) Population–Concept–Context (PCC) framework. The electronic search was initiated in the Scopus database mainly focusing on identifying Scopus-indexed peer-reviewed journal articles with Psychology and related cross-disciplinary areas.
Findings: The findings reveal that a great number of interdisciplinary research projects focusing on AI-enabled communication technologies were conducted during the period under the review. Multimodal learning, deep learning, machine learning, computer vision, natural language processing, transformer architectures, large language models, affective computing, speech recognition, gesture recognition, emotion recognition, conversational AI, and adaptive intelligent systems were the most commonly mentioned technologies in the literature.
Unique contribution to theory, practice and policy: The study gives policymakers the insight that standardized benchmarking of the communication AI systems, open and transparent AI governance systems, data privacy measures, inclusive and equitable datasets, and regulations of Ethical AI development and deployment should be made priority in policy guidelines on the use of AI communication systems in communication. This scoping review has one of its primary contributions to the literature through the provision of an excellent map of AI multimodal communication technology evidence between 2021 and 2026. The study, from a theoretical point of view, enriches the existing knowledge by bringing AI-related work together with psychological communication educational, health rehabilitation accessibility, and HCI evidence in a singular system.
Downloads
References
Alsaleh, A. (2026). Alsaleh, A. (2026). The influence of artificial intelligence on individuals with disabilities. Acta Psychologica, 262, 106010.https://doi.org/10.1016/j.actpsy.2025.106010
Arksey, H., & O'malley, L. (2005). Scoping studies: towards a methodological framework. International journal of social research methodology, 8(1), 19-32.
Bailey, A. L., Abraham, M. E., Dong, J. Y., Torres, I. M., Wang, Y. P., Zamora, E., & Zhang, X. (2025). Translanguaging : Conceptual underpinnings of equity-oriented instructional and assessment practices with adolescent multilingual learners. Linguistics and Education, 87(February), 101418. https://doi.org/10.1016/j.linged.2025.101418
Chaturvedi, R., Verma, S., Das, R., & Dwivedi, Y. K. (2026). Technological Forecasting & Social Change Social companionship with artificial intelligence : Recent trends and future avenues. Technological Forecasting & Social Change, 193(November 2022), 122634. https://doi.org/10.1016/j.techfore.2023.122634
Cheng, K. Y. (2026). Principles of consent and non-addiction in AI grief bots. Journal of Responsible Technology, 26(May), 100177. https://doi.org/10.1016/j.jrt.2026.100177
Dwivedi, Y. K., Hughes, L., Baabdullah, A. M., Ribeiro-navarrete, S., Giannakis, M., Al-debei, M. M., Dennehy, D., Metri, B., Buhalis, D., Cheung, C. M. K., Conboy, K., Doyle, R., Dubey, R., Dutot, V., Felix, R., Goyal, D. P., Gustafsson, A., Hinsch, C., Jebabli, I., … Fosso, S. (2022). International Journal of Information Management Metaverse beyond the hype : Multidisciplinary perspectives on emerging challenges , opportunities , and agenda for research , practice and policy. International Journal of Information Management, 66(July), 102542. https://doi.org/10.1016/j.ijinfomgt.2022.102542
Elzer, S., Carberry, S., & Zukerman, I. (2011). The automated understanding of simple bar charts. Artificial Intelligence, 175(2), 526–555. https://doi.org/10.1016/j.artint.2010.10.003
Feng, S., Zhang, H., & Ga, D. (2025). Computers and Education : Artificial Intelligence Mapping the evolution of AI in education : Toward a co-adaptive and human-centered paradigm. 9(December). https://doi.org/10.1016/j.caeai.2025.100513
Giberga, A., Guerra, E., Ahufinger, N., Igualada, A., Aguilera, M., & Esteve-gibert, N. (2025). Prosody and gestures help pragmatic processing in children with Developmental Language Disorder. Journal of Communication Disorders, 115(December 2024), 106525. https://doi.org/10.1016/j.jcomdis.2025.106525
Górriz, J. M., Arco, J. E., Atzmueller, M., Ballarini, F., Barakova, E., Bologna, G., Bonomini, P., Duro, R. J., Elizondo, D., Julian, V., Kotz, G., Kutt, K., Leming, M., Lope, J. De, Macas, B., Martinez, J. J., Mekyska, J., Nalepa, G. J., Novais, P., … Zhu, Z. (2023). Computational approaches to Explainable Artificial Intelligence : Advances in theory , applications and trends. Information Fusion, 100(July), 101945. https://doi.org/10.1016/j.inffus.2023.101945
Hassani, S., Dackermann, U., Mousavi, M., & Li, J. (2024). A systematic review of data fusion techniques for optimized structural health monitoring. Information Fusion, 103(November 2023), 102136. https://doi.org/10.1016/j.inffus.2023.102136
Jiang, T., Sun, Z., Fu, S., & Lv, Y. (2024). Human-AI interaction research agenda : A user-centered perspective. 8(February).
Kayıs, E., Sarı, S., Dogan, N., Mumcu, E., Arı, R., Ozbaran, B., & Hatipo, A. (2026). A new approach in autism diagnosis : Evaluating natural interaction using point of view ( POV ) glasses. 116(December 2025). https://doi.org/10.1016/j.ajp.2025.104798
Kristina, D., Colasante, T., Roldan, E., Carlos, J., Martinez, R., Li, H., Mukherjee, S., & Malti, T. (2026). An experimental study exploring human – AI complementarity in early social-emotional learning. Computers and Education Open, 10(January), 100331. https://doi.org/10.1016/j.caeo.2026.100331
Levy, E. T., & McNeill, D. (1992). Speech, gesture, and discourse. Discourse processes, 15(3), 277-301.
Liu, J., Shi, Y., Gu, C., & Li, Q. (2025). Optimization design and application of mechanical characteristics in ergonomics of children ’ s intelligent toys. Systems and Soft Computing, 7(December 2024), 200245. https://doi.org/10.1016/j.sasc.2025.200245
Liu, Z., Cheon, S., Stanbury, A., Jiao, X., Xing, W., & Kang, H. (2026). Computers and Education : Artificial Intelligence Towards contextual-based AI : A scoping review of artificial intelligence in X reality for personalized learning. Computers and Education: Artificial Intelligence, 10(December 2025), 100523. https://doi.org/10.1016/j.caeai.2025.100523
Ma, W., & Liu, Q. (2023). Language and health studies in the era of holistic health : Achievements and prospects. Language and Health, 1(2), 1–11. https://doi.org/10.1016/j.laheal.2023.11.001
Mcstay, A., & Bakir, V. (2025). Soft law for unintentional empathy : addressing the governance gap in emotion-recognition AI technologies. Journal of Responsible Technology, 23(July), 100126. https://doi.org/10.1016/j.jrt.2025.100126
Oruma, S. O. (2026). Computer Standards & Interfaces Enhancing security , privacy , and usability in social robots : A software development framework. 96(June 2025).
Peters, M. D., Marnie, C., Tricco, A. C., Pollock, D., Munn, Z., Alexander, L., ... & Khalil, H. (2021). Updated methodological guidance for the conduct of scoping reviews. JBI evidence implementation, 19(1), 3-10.
Quan, S., Tu-shea, X., Ding, Y., Du, Y., Zheng, Q., & Gerdich, L. E. (2026). Computers and Education : Artificial Intelligence Conversational AI in children ’ s home literacy learning : effectiveness , advantages , challenges , and family perception. Computers and Education: Artificial Intelligence, 10(December 2025), 100549. https://doi.org/10.1016/j.caeai.2026.100549
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., ... & Sutskever, I. (2021, July). Learning transferable visual models from natural language supervision. In International conference on machine learning (pp. 8748-8763). PmLR.
Seaborn, K., Altmeyer, M., Rikaku, G., Ku, B., Kobuki, S., & Urakami, J. (2025). International Journal of Human - Computer Studies The Voice Experience Inventory ( VOXI ): Validating a consensus-driven instrument for measuring user impressions of computer voice $. International Journal of Human - Computer Studies, 203(June), 103576. https://doi.org/10.1016/j.ijhcs.2025.103576
Shneiderman, B. (2022). Human-centered AI. Oxford University Press.
Terkourafi, M., & Hansen, M. M. (2026). Hearer ’ s Meaning 2 . 0 : A reply to Li and Xie (2025). Journal of Pragmatics, 261, 4–18. https://doi.org/10.1016/j.pragma.2026.04.007
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., ... & Straus, S. E. (2018). PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Annals of internal medicine, 169(7), 467-473.
Turner, M., & Tour, E. (2025). Using a translanguaging multimodal approach to develop teachers ’ language awareness in linguistically diverse classrooms in Australia. Teaching and Teacher Education, 153(November 2024), 104835. https://doi.org/10.1016/j.tate.2024.104835
Wu, Q., Wang, W., Liu, Q., Zhang, R., Suen, Y., & Billinghurst, M. (2025). International Journal of Human - Computer Studies Augmented tabletop interaction as an assistive tool : Tidd ’ s role in daily life skills training for autistic children. International Journal of Human - Computer Studies, 202(April), 103517. https://doi.org/10.1016/j.ijhcs.2025.103517
Xiao, F., Li, Z., Lin, J., Zou, X., Yang, D., & Zou, W. (2025). Computers and Education : Artificial Intelligence Leveraging an LLM-enhanced bilingual conversational agent for EFL children ’ s dialogic reading : Insights from children , parents , and educators. Computers and Education: Artificial Intelligence, 9(December 2024), 100484. https://doi.org/10.1016/j.caeai.2025.100484
Yang, W. (2022). Computers and Education : Artificial Intelligence Artificial Intelligence education for young children : Why , what , and how in curriculum design and implementation ☆. Computers and Education: Artificial Intelligence, 3(February), 100061. https://doi.org/10.1016/j.caeai.2022.100061
Zhang, B., & Xie, M. (2026). Computers in Human Behavior Reports CALA-Net : Intelligent depression detection with multimodal fusion and Tree-of-Thoughts framework. Computers in Human Behavior Reports, 22(March), 101051. https://doi.org/10.1016/j.chbr.2026.101051
Zhu, Y., Gao, T., Fan, L., Huang, S., Edmonds, M., Liu, H., Gao, F., Zhang, C., Qi, S., Nian, Y., Tenenbaum, J. B., & Zhu, S. (2020). Dark , Beyond Deep : A Paradigm Shift to Cognitive AI with Humanlike Common Sense. Engineering, 6(3), 310–345. https://doi.org/10.1016/j.eng.2020.01.011
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Tabinda Khawaja Sarfraz, Abbas Rashid Butt
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution (CC-BY) 4.0 License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
