Survey on Real-world Applications and Challenges of Deep Learning-Enhanced Techniques to assist visually impaired
Keywords:
Object Detection, Visually impaired, Accuracy Analysis, Assisting aid, Small object detectionAbstract
This comprehensive survey report delves deeply into the real-world applications and complicated issues inherent in deep learning-enhanced wearable solutions for people with vision impairments. It stresses the global incidence of visual impairment, particularly in underserved areas, and follows the growth of assistive devices over time. The study examines deep learning's revolutionary function, demonstrating its impact through real-world case studies such as OrCam MyEye and Brain-Computer Interfaces. It does, however, rigorously identify the various technical challenges, such as data accessibility and real-time processing, as well as ethical concerns, such as privacy and fairness. In conclusion, while the paper highlights the potential of deep learning to empower people with disabilities, it also calls for the continual resolution of these obstacles to construct a more inclusive and accessible future. We need to focus on designing small size object detection and object recognition systems which consider varying size images, to address the problems faced by the visually impaired in their passive and active stages, according to the study.
Downloads
References
Sahand Shahalinejad, Mehdi Nooshyar ,Mir Majid Fakheri urmia, Nima ImaniObject Recognition and Detection Using Convolutional Neural Network in Deep Learning,in bioRχiv ,(2023).
Satyaki Mukherjee; Tanya Sharma; Anshika Singh; Samiappan Dhanalakshmi,in International Conference on Recent Advances in Electrical, Electronics, Ubiquitous Communication, and Computational Intelligence (RAEEUCCI), 2023.
Mishra, S., Naqvi, T. K., & Dwivedi, P. K. Current Challenges and Perspectives of Flexible and Wearable Sensors. Flexible and Wearable Sensors: Materials, Technologies, and Challenges, 351, (2023).
Kuriakose, B., Shrestha, R., & Sandnes, F. E. DeepNAVI: A deep learning based smartphone navigation assistant for people with visual impairments. Expert Systems with Applications, 212, 118720. (2023).
Wang, H., Wang, G., & Li, X. An RGB-D camera-based indoor occupancy positioning system for complex and densely populated scenarios. Indoor and Built Environment, 1420326X231155112. (2023).
Mygal, G., Protasenko, O., Kobrina, N., & Mykhailova, E.). Ergonomic thinking in the design of human-machine systems.], (2023).
Seng, Kah Phooi, Li-Minn Ang, Eno Peter, and Anthony Mmonyi. "Machine Learning and AI Technologies for Smart Wearables" Electronics 12, no. 7: 1509. https://doi.org/10.3390/electronics12071509, 2023.
Bergsten, P., & Kihan, M. Spatialisation of Binaural Audio With Head Tracking in First-Person Computer Games, (2023).
Yanagawa, M., Ito, R., Nozaki, T., Fujioka, T., Yamada, A., Fujita, S., ... & Naganawa, S. New trend in artificial intelligence-based assistive technology for thoracic imaging. La radiologia medica, 1-14, (2023).
Rahmayanti, A., Kahar, S., & Puspitasari, T. D. Utilizing Deep Learning Algorithm for Enhanced Visual Product Identification. EUROPEAN JOURNAL OF INNOVATION IN NONFORMAL EDUCATION, 3(7), 45-50, (2023).
Renju Rachel Varghese; Pramod Mathew Jacob; Midhun Shaji; Abhijith R; Emil Saji John; Sebin Beebi Philip,An Intelligent Voice Assistance System for Visually Impaired using Deep Learning,in , 2022.
Ganesan, J., Azar, A. T., Alsenan, S., Kamal, N. A., Qureshi, B., & Hassanien, A. E.Deep learning reader for visually impaired. Electronics, 11(20), 3335, (2022)..
Adegboro, S., & Faniran, T. S. Development of Smart Intelligent Walking Aid 3rd Eye for the Blind Using Ultrasonic Sensor. University of Ibadan Journal of Science and Logics in ICT Research, 8(2), 27-36, (2022).
.Harsh Shah; Rishi Shah; Shlok Shah; Paawan SharmaDangerous Object Detection for Visually Impaired People using Computer Vision,in International Conference on Artificial Intelligence and Machine Vision IEEE, (2022).
Boldini, A., Ma, X., Rizzo, J. R., & Porfiri, M. A virtual reality interface to test wearable electronic travel aids for the visually impaired. In Nano-, Bio-, Info-Tech Sensors and Wearable Systems (Vol. 11590, pp. 50-56). SPIE, (2021, March).
Mrozek, D., Kwaśnicki, S., Sunderam, V., Małysiak-Mrozek, B., Tokarz, K., & Kozielski, S. Comparison of Speech Recognition and Natural Language Understanding Frameworks for Detection of Dangers with Smart Wearables. In International Conference on Computational Science (pp. 471-484). Cham: Springer International Publishing., (2021, June).
Jiang, S., Kang, P., Song, X., Lo, B. P., & Shull, P. B. Emerging wearable interfaces and algorithms for hand gesture recognition: A survey. IEEE Reviews in Biomedical Engineering, 15, 85-102, (2021).
Ntakolia, C., & Iakovidis, D. K. A route planning framework for smart wearable assistive navigation systems. SN Applied Sciences, 3(1), 104, (2021).
.Nitin Kumar; Anuj Jain,Smart Navigation Detection using Deep-learning for Visually Impaired Person,in IEEE 2nd International Conference On Electrical Power and Energy Systems (ICEPES), 2021.
Saravanan Alagarsamy; T. Dhiliphan Rajkumar; K. P. L. Syamala; Ch. Sandya Niharika; D. Usha Rani,K. Balaji,An Real Time Object Detection Method for Visually Impaired Using Machine Learning,in International Conference on Artificial Intelligence and Machine Vision,IEEE, 2021.
Tapu, R., Mocanu, B., & Zaharia, T. Wearable assistive devices for visually impaired: A state of the art survey. Pattern Recognition Letters, 137, 37-52, (2020).
Basu, A., Petropoulakis, L., Di Caterina, G., & Soraghan, J. Assistive technology evolving as intelligent system. New Trends in Computational Vision and Bio-inspired Computing: Selected works presented at the ICCVBIC 2018, Coimbatore, India, 289-303, (2020).
Olwal, A., Balke, K., Votintcev, D., Starner, T., Conn, P., Chinh, B., & Corda, B. Wearable subtitles: Augmenting spoken communication with lightweight eyewear for all-day captioning. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (pp. 1108-1120), (2020, October).
Alamdari, N., Lobarinas, E., & Kehtarnavaz, N. Personalization of hearing aid compression by human-in-the-loop deep reinforcement learning. IEEE Access, 8, 203503-203515, (2020).
.Tanvir Ahmad,Yinglong Ma,Muhammad Yahya,Belal Ahmad,Shah Nazir,and Amin ul HaqObject Detection through Modified YOLO Neural Network,Volume 2020 | Article ID 8403262, 2020.
Yanfen Li; Hanxiang Wang; L. Minh Dang; Tan N. Nguyen; Dongil Han; Ahyun Lee; Insung Jang Hyeonjoon Moon,A Deep Learning-Based Hybrid Framework for Object Detection and Recognition in Autonomous Driving,IEEE Access ( Volume: 8) ,(2020)
Jafarzadeh, M., Hussey, D. C., & Tadesse, Y. (2019, September). Deep learning approach to control of prosthetic hands with electromyography signals. In IEEE International Symposium on Measurement and Control in Robotics (ISMCR) (pp. A1-4). IEEE(2019).
Tang, S. Y., Hoang, N. S., Chui, C. K., Lim, J. H., & Chua, M. C. Development of wearable gait assistive device using recurrent neural network. In IEEE/SICE International Symposium on System Integration (SII) (pp. 626-631). IEEE, (2019, January).
Mohammed, S. L., Al-Naji, A., Farjo, M. M., & Chahl, J. Highly accurate water level measurement system using a microcontroller and an ultrasonic sensor. In IOP Conference Series: Materials Science and Engineering (Vol. 518, No. 4, p. 042025). IOP Publishing, (2019, May).
Hakim, H., & Fadhil, A. Navigation system for visually impaired people based on RGB-D camera and ultrasonic sensor. In Proceedings of the International Conference on Information and Communication Technology (pp. 172-177), (2019, April).
.Murad, C. Tools to support voice user interface design. In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services (pp. 1-5), (2019, October).
World Health Organization (WHO), Visual impairment and blindness, accessed on http://www.who.int/news-room/fact-sheets/detail/blindness-and-visual-impairment, (23.05.2018).
Katzschmann, R. K., Araki, B., & Rus, D. Safe local navigation for visually impaired users with a time-of-flight and haptic feedback device. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 26(3), 583-593, (2018).
Hamayun A. Khan,DM-L Based Feature Extraction and Classifier Ensemble for Object Recognition,Journal of Signal and Information Processing,Vol.9 No.2, May 2018
Wenzhi Zhao; Shihong Du; William J. Emery,Object-Based Convolutional Neural Network for High-Resolution Imagery Classification, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing ( Volume: 10, Issue: 7, July 2017)
Wenzhou; Shihong Du; William J. Emery,Object-Based Convolutional Neural Network for High-Resolution Imagery Classification,in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing ( Volume: 10, Issue: 7, July 2017)
Gao Huang; Zhuang Liu; Laurens Van Der Maaten; Kilian Q. Weinberger,Densely Connected Convolutional Networks, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017
.Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand Marco Andreetto Hartwig Adam,MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,arXiv:1704.04861v1 [cs.CV] 17 Apr 2017
François Chollet,Xception: Deep Learning with Depthwise Separable Convolutions,IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017
François Chollet,Xception: Deep Learning with Depthwise Separable Convolutions,IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017
Lazar, J., Feng, J. H., & Hochheiser, H. Research methods in human-computer interaction. Morgan Kaufmann, (2017).
Ishihara, T., Vongkulbhisal, J., Kitani, K. M., & Asakawa, C. Beacon-guided structure from motion for smartphone-based navigation. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 769-777). IEEE, (2017, March).
.Hamzah, R., & Fadzil, M. I. M. Voice4Blind: The talking braille keyboard to assist the visual impaired users in text messaging. In 2016 4th International Conference on User Science and Engineering (i-USEr) (pp. 265-270). IEEE, (2016, August).
Mohamedali, F., & Matoorian, N. Support dementia: using wearable assistive technology and analysing real-time data. In International Conference on Interactive Technologies and Games (ITAG) (pp. 50-54). IEEE, (2016, October).
Kaiming He; Xiangyu Zhang; Shaoqing Ren; Jian Sun,Deep Residual Learning for Image Recognition,in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016
Joseph Redmon, Ali FarhadiYOLO9000: Better, Faster, Stronger,in Computer Vision and Pattern Recognition ,arXiv:1612.08242,2016
Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik, Rich feature hierarchies for accurate object detection and semantic segmentation, in Computer Vision and Pattern Recognition,arXiv:1311.2524, (2015).
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian SunSpatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitio,Computer Vision and Pattern Recognition,arXiv:1406.4729,2015
Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik, Rich feature hierarchies for accurate object detection and semantic segmentation,Computer Vision and Pattern Recognition,arXiv:1311.2524, 2015
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun,Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition,aComputer Vision and Pattern Recognition,rXiv:1406.4729,2015
Christian Szegedy; Wei Liu; Yangqing Jia; Pierre Sermanet; Scott Reed; Dragomir Anguelov; Dumit Erhan,,Vincent Vanhoucke,Andrew Rabinovich,Going deeper with convolutions in IEEE Conference on Computer Vision and Pattern Recognition (CVPR)IEEE 2015
.Karen Simonyan, Andrew Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition,Computer Vision and Pattern Recognition,aXiv:1409.1556, (2015)
.Guanbin Li, Yizhou Yu,Visual Saliency Based on Multiscale Deep Featuresin in Computer Vision and Pattern Recognition, arXiv:1503.08663, (2015)
Martinez-Sala, A. S., Losilla, F., Sánchez-Aarnoutse, J. C., & García-Haro, J. Design, implementation and evaluation of an indoor navigation system for visually impaired people. Sensors, 15(12), 32168-32187, (2015).
. Lane, N. D., Bhattacharya, S., Georgiev, P., Forlivesi, C., & Kawsar, F. An early resource characterization of deep learning on wearables, smartphones and internet-of-things devices. In Proceedings of the 2015 international workshop on internet of things towards applications (pp. 7-12), (2015, November).
.Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik, Rich feature hierarchies for accurate object detection and semantic segmentation,Computer Vision and Pattern Recognition, arXiv:1311.2524v5, 2014
Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik,Rich feature hierarchies for accurate object detection and semantic segmentation(in Computer Vision and Pattern Recognition,arXiv:1311.2524, 2014,
.Matthew D Zeiler, Rob Fergus,Visualizing and Understanding Convolutional Networksin Computer Vision and Pattern Recognition arXiv:1311.2901,(2013),
Oz, C., & Leu, M. C. American sign language word recognition with a sensory glove using artificial neural networks. Engineering Applications of Artificial Intelligence, 24(7), 1204-1213 (2011).
Yelamarthi, K., Haas, D., Nielsen, D., & Mothersell, S. RFID and GPS integrated navigation system for the visually impaired. In 53rd IEEE International Midwest Symposium on Circuits and Systems (pp. 1149-1152). IEEE, (2010, August).
Matthew D. Zeiler; Dilip Krishnan; Graham W. Taylor; Rob Fergus,Deconvolutional networks in : IEEE Computer Society Conference on Computer Vision and Pattern Recognition , 2010
Nicholson, J., Kulyukin, V., & Coster, D. ShopTalk: independent blind shopping through verbal route directions and barcode scans. The Open Rehabilitation Journal, 2(1), (2009).
Akita, J., Ito, K., Ono, T., & Okamoto, M.CyARM: haptic sensing device for spatial localization on basis of exploration by arms. Advances in Human-Computer Interaction, (2009).
. Dakopoulos, D., & Bourbakis, N. G. Wearable obstacle avoidance electronic travel aids for blind: a survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(1), 25-35, (2009).
Peres, S. C., Best, V., Brock, D., Frauenberger, C., Hermann, T., Neuhoff, J. G., ... & Stockman, A. Auditory interfaces. HCI beyond the GUI: design for haptic, speech, olfactory, and other nontraditional interfaces, 147-195, (2008).
. Rubin, J., & Chisnell, D. Handbook of usability testing: How to plan, design, and conduct effective tests. John Wiley & Sons., (2008).
. Cardin, S., Thalmann, D., & Vexo, F. A wearable system for mobility improvement of visually impaired people. The Visual Computer, 23, 109-11, (2007).
V. Kulyukin, C. Gharpure and J. Nicholson, "RoboCart: toward robot-assisted navigation of grocery stores by the visually impaired," IEEE/RSJ International Conference on Intelligent Robots and Systems, Edmonton, AB, Canada, pp. 2845-2850, doi: 10.1109/IROS.2005.1545107, (2005)
.Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (Brain–computer interfaces for communication and control. Clinical neurophysiology, 113(6), 767-7912002)
Kay, L. Auditory perception of objects by blind persons, using a bioacoustic high resolution air sonar. The Journal of the Acoustical Society of America, 107(6), 3266-3275, (2000).
.L. Itti; C. Koch; E. Niebur,A model of saliency-based visual attention for rapid scene analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence ( Volume: 20, Issue: 11, November 1998)
.Object Detection and Recognition for Visually Impaired Using Cross-Stage-Partial Network Algorithms,
. https://www.nngroup.com/articles/usability-101-introduction-to-usability/
. https://www.noorcam.com/en-ae/noorcam-myeye
.Obstacle and planar object detection using sparse 3D information for a smart walke
. Kaiming He, Georgia Gkioxari, Piotr Dollár, Ross Girshick,arXiv:1703.06870,24 Jan 2018 .
.Image et Reconnaissance de Forme-Systèmes Intelligents et Communicants (IRF-SIC) Laboratory, Faculty of Sciences, Ibn Zohr University, Agadir 80000, Morocco
Tahira Shehzadi, Khurram Azeem Hashmi, Didier Stricker, Muhammad Zeshan Afzal,Object Detection with Transformers: A Review,Computer Vision and Pattern Recognition,10 Jul 2023
.Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko,Computer Vision and Pattern Recognition,arXiv:2005.12872 ,28 May 2020
.Forrest N. Iandola1 , Song Han2 , Matthew W. Moskewicz1 , Khalid Ashraf1 , William J. Dally2 , Kurt Keutzer1 1DeepScale∗ & UC Berkeley 2Stanford Univer
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
