IoT-Enabled Multimodel Emotion Detection and Assistive Smartwatch for Special-Needs Children
DOI:
https://doi.org/10.51903/jtie.v5i1.471Keywords:
IoT, Smartwatch, Physiological Sensing, Facial Expression Recognition, Multimodel Data Fusion.Abstract
Children with neurodevelopmental and communication challenges often find it difficult to express emotional discomfort, stress, or urgent needs, which can delay timely support and increase anxiety. To address this issue, this paper presents an IoT-enabled assistive smartwatch designed to support near real-time emotion inference and basic communication for children with special needs. The proposed system integrates physiological sensing, including heart rate variability, galvanic skin response, and skin temperature, with camera-based facial expression analysis using a multimodal data fusion approach. In addition to emotion-aware monitoring, the smartwatch provides a simple icon-based communication interface and a routine reminder module to support daily activities. The framework enables caregiver-side visualization of emotional trends, communication events, and adherence to routines through a cloud-based dashboard. The system is evaluated through a design-level feasibility assessment using simulated experiments and analysis of benchmark datasets. Overall, this work presents a technically feasible and ethically conscious assistive framework that highlights the potential of combining IoT and emotion-aware computing for supportive care applications.
References
Akter, S., & Jeong, H. (2022). Privacy-Preserving On-Device Emotion Recognition for Wearable Health Systems. IEEE Consumer Electronics Magazine, 11(2), 48–56. https://doi.org/10.1109/mce.2021.3136651
Cano, S., Cubillos, C., Alfaro, R., et al. (2024). Wearable Solutions Using Physiological Signals for Stress Monitoring in Individuals with Autism Spectrum Disorder: A Systematic Review. Sensors, 24(24), 8137. https://doi.org/10.3390/s24248137
Gedam, S., et al. (2025). Analyzing Mental Stress Using Wearable Physiological Data and Machine Learning. Scientific Reports, 15, 6951. https://doi.org/10.1038/s41598-025-06918-6
Goodfellow, I., et al. (2013). Challenges in Representation Learning: A Report on Three Machine-Learning Contests. Neural Information Processing Systems, 117-124. https://doi.org/10.1007/978-3-642-42051-1_16
Haque, Y., et al. (2024). State-of-the-Art Stress Prediction from Heart-Rate Variability Using Artificial Intelligence. Cognitive Computation and Systems, 16(2), 455-481. https://doi.org/10.1007/s12559-023-10200-0
Hossain, M. S., & Muhammad, G. (2021). Emotion Recognition Using Deep Learning and IoT Framework for Health Monitoring. IEEE Internet of Things Journal, 8(5), 3513–3521. https://doi.org/10.1109/jiot.2020.3007661
Hwang, G., & Huang, Y. (2024). Emotion Recognition Using PPG Signals of a Smartwatch: A Machine-Learning Approach. Sensors, 25(1), 18. https://doi.org/10.3390/s25010018
Ismail, S. N. M. S., et al. (2024). A Systematic Review of Emotion Recognition Using Cardio-Based Physiological Signals. Journal of Affective Computing, 10(1), 156-183. https://doi.org/10.1016/j.jac.2024.01.012
Lazarou, E., et al. (2024). Predicting Stress Levels Using Physiological Data: Real-Time Models and Wearable Implementations. Neuroscience and Biobehavioral Reviews, 11(2), 76. https://doi.org/10.1016/j.neubiorev.2023.105123
Li, G., Li, Y., & Song, R. (2021). Emotion Recognition Based on Fusion of Facial Expressions and Physiological Signals. IEEE Access, 9, 103895–103906. https://doi.org/10.1109/access.2021.3098696
Lian, H., Schuller, B., Valstar, M., et al. (2023). A Survey of Deep Learning-Based Multimodal Emotion Recognition. Information Fusion, 79, 1–21. https://doi.org/10.1016/j.inffus.2021.10.003
Liu, Y., Wei, W., & Qiu, M. (2022). Edge-AI Enabled Emotion Recognition for Wearable Devices. IEEE Internet of Things Magazine, 5(3), 30–36. https://doi.org/10.1109/iotm.0001.2100085
Mavroudi, E., Mareschal, D., & Kirkham, N. (2020). Wearable Devices as Aids for Developmental Disorders: A Systematic Review. PLOS ONE, 15(9), e0239339. https://doi.org/10.1371/journal.pone.0239339
Paniagua-Gómez, M., et al. (2025). Trends and Challenges in Real-Time Stress Detection Using IoT-Enabled Wearables. Electronics, 14(13), 2581. https://doi.org/10.3390/electronics14132581
Pratama, A., & Nugroho, R. (2023). Implementation of Information System Security Using ISO/IEC 27001 in Academic Information Systems. Jurnal Informatika dan Sistem Informasi, 9(2), 130-141. https://doi.org/10.51903/jt9z2f09
Rashid, T., Bajwa, I. S., & Kim, J. (2025). An IoT-Based Smart Emotion Recognition System Using Internal Body Parameters. Scientific Reports, 16, 7210. https://doi.org/10.1038/s41598-025-08321-7
Ringeval, F., et al. (2021). AVEC 2021: Multimodal Affect Recognition and Analysis Challenge. Proceedings of the ACM International Conference on Multimedia, 3–12. https://doi.org/10.1145/3474085.3475707
Qosidah, N., & Susilo, B. W. (2024). Enhancing Employee Performance Through AI-Driven Business Communication: A Case Study. Journal of Management and Informatics, 3(2), 163-179. https://doi.org/10.51903/jmi.v3i2.29
Salamea-Villarreal, L., et al. (2022). Edge-AI and IoT Designs for Wearable Affective Systems: A Survey. IEEE Internet of Things Journal, 9(12), 9876–9895. https://doi.org/10.1109/jiot.2022.3187654
Salas-Cáceres, J., et al. (2024). Multimodal Emotion Recognition Based on Fusion of Visual and Audio Cues. Multimedia Tools and Applications, 11(3), 1–12. https://doi.org/10.1007/s11042-024-20227-6
Schmidt, P., et al. (2018). Introducing WESAD: A Multimodal Dataset for Wearable Stress and Affect Detection. Proceedings of the ACM International Conference on Multimodal Interaction, 400–408. https://doi.org/10.1145/3242969.3242985
Wang, C., Bota, P. J., Fred, A., et al. (2022). A Multimodal Wearable System for Emotion Recognition. IEEE Transactions on Affective Computing, 13(4), 227–239. https://doi.org/10.1109/taffc.2020.3036805
Wibowo, M. C., & Santoso, J. T. (2024). Empowering Urban Farmers: an Asynchronous Learning Application for Greenhouse Management. Journal of Technology Informatics and Engineering, 3(2), 138–150. https://doi.org/10.51903/jtie.v3i2.185
Xefteris, V. R., et al. (2023). Stress Detection Based on Wearable Physiological Sensors in Laboratory and Real-Life Environments. Sensors and Actuators A: Physical, 345, 113374. https://doi.org/10.1016/j.sna.2023.113374
Zhang, K., Li, Y., & Wang, J. (2022). Deep Learning-Based Emotion Recognition in IoT-Enabled Wearable Systems. IEEE Sensors Journal, 22(14), 7688–7699. https://doi.org/10.1109/jsen.2022.3168247
Downloads
Published
Issue
Section
License
Copyright (c) 2026 A. Samrin, S. Saidharshini, S. Shalima, B. Moohambiga
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
