AI Based Support System for Blind People with an Integrated Reading Assistant
Keywords:
Artificial Intelligence (AI), Blind or Visually Impaired People (BVIP), Nvidia NGC and Video Processing.Abstract
Mobility, navigation and reading are some of the major problems for the Blind or Visually Impaired People (BVIP). BVIP are still struggling with all of them. In the current scenario of the booming of Artificial Intelligence (AI) where everything is shifted to be “smart”, we are not able to solve the problems of these BVIP. They can't even do their daily chores properly whereas we all are getting our house operated on voice assistants. As we know wearable technologies take a hike after the merging with AI. This can give the BVIP a great shift to ease their living. In this paper a prototype is presented. It is wearable like a badge on the chest or can be fitted on the spectacles that will assist the BVIP in reading documents, objects identification, and face recognition. It is implemented using the state-of-the art models provided by the Nvidia NGC. It carries out real-time video processing of visual input provided by the camera installed in it. After processing, it classifies different objects and measures their distances from it. It does this using people net and B13D proximity segmentation models of Nvidia NGC. Once the image or video analysis is done, auditory feedback is given to the user. There is a reading assistant in the gadget that can read any document present in front of it. Reading is performed using text extraction models from images.
Downloads
References
J. Xiao, S. L. Joseph, X. Zhang, B. Li, X. Li, and J. Zhang, “An assistive navigation framework for the visually impaired,” IEEE Trans. HumanMach. Syst., vol. 45, no. 5, pp. 635–640, Oct. 2015.
Karmel, A. Sharma, M. Pandya, and D. Garg, “IoT based assistive device for deaf, dumb and blind people,” Procedia Comput. Sci., vol. 165, pp. 259–269, Nov. 2019.
Y. Liu, N. R. B. Stiles, and M. Meister, “Augmented reality powers a cognitive assistant for the blind,” eLife, vol. 7, Nov. 2018, Art. no. e37841.
K. Patil, Q. Jawadwala, and F. C. Shu, “Design and construction of electronic aid for visually impaired people,” IEEE Trans. Human-Mach. Syst., vol. 48, no. 2, pp. 172–182, Apr. 2018.
EU H2020 Project. Sound of Vision Natural Sense of Vision Through Acoustics and Haptics. Accessed: Dec. 13, 2020.
Imadu, T. Kawai, Y. Takada, and T. Tajiri, ‘‘Walking guide interface mechanism and navigation system for the visually impaired,’’ in Proc. 4th Int. Conf. Hum. Syst. Interact., HSI, May 2011, pp. 34–39.
M. Zöllner, S. Huber, H.-C. Jetter, and H. Reiterer, ‘‘NAVI—A proof-ofconcept of a mobile navigational aid for visually impaired based on the microsoft Kinect,’’ in Proc. IFIP Conf. Hum.-Comput. Interact., 2011, pp. 584–587
G. Capi and H. Toda, ‘‘Development of a new robotic system for assisting visually impaired people,’’ Int. J. Social Robot., vol. 4, no. S1, pp. 33–38, Nov. 2012.
C. Feng, S. Valaee, A. W. S. Au, S. Reyes, S. Sorour, S. N. Markowitz, D. Gold, K. Gordon, and M. Eizenman, ‘‘Anonymous indoor navigation system on handheld mobile devices for visually impaired,’’ Int. J. Wireless Inf. Netw., vol. 19, no. 4, pp. 352–367, Dec. 2012.
S. Kammoun, M. J.-M. Macé, B. Oriola, and C. Jouffrais, ‘‘Towards a geographic information system facilitating navigation of visually impaired users,’’ in Proc. Int. Conf. Comput. Handicapped Persons, 2012, pp. 521–528.
J. Zegarra and R. Farcy, ‘‘GPS and inertial measurement unit (IMU) as a navigation system for the visually impaired,’’ in Proc. Int. Conf. Comput. Handicapped Persons, 2012, pp. 29–32.
C. Liao, P. Choe, T. Wu, Y. Tong, C. Dai, and Y. Liu, ‘‘RFID-based road guiding cane system for the visually impaired,’’ in Proc. Int. Conf. CrossCultural Design, 2013, pp. 86–93.
M. Nakajima and S. Haruyama, ‘‘New indoor navigation system for visually impaired people using visible light communication,’’ EURASIP J. Wireless Commun. Netw., vol. 2013, no. 1, p. 37, Dec. 2013.
J. Balata, J. Franc, Z. Mikovec, and P. Slavik, ‘‘Collaborative navigation of visually impaired,’’ J. Multimodal User Interface, vol. 8, no. 2, pp. 175–185, Jun. 2014.
Q.-H. Nguyen, H. Vu, T. H. Tran, D. Van Hamme, P. Veelaert, W. Philips, and Q. H. Nguyen, ‘‘A visual SLAM system on mobile robot supporting localization services to visually impaired people,’’ in Proc. Eur. Conf. Comput. Vis., 2014, pp. 716–729.
M. Hosny, R. Alsarrani, and A. Najjar, ‘‘Indoor wheelchair navigation for the visually impaired,’’ in Proc. Int. Conf. Hum. -Comput. Interact., 2015, pp. 411–417.
Y. H. Lee and G. Medioni, ‘‘Wearable RGBD indoor navigation system for the blind,’’ in Proc. Eur. Conf. Comput. Vis., 2015, pp. 493–508.
D. Ni, A. Song, L. Tian, X. Xu, and D. Chen, ‘‘A walking assistant robotic system for the visually impaired based on computer vision and tactile perception,’’ Int. J. Social Robot., vol. 7, no. 5, pp. 617–628, Nov. 2015.
Bhowmick and S. M. Hazarika, ‘‘An insight into assistive technology for the visually impaired and blind people: State-of-the-art and future trends,’’ J. Multimodal User Interface, vol. 11, no. 2, pp. 149–172, Jun. 2017.
Q.-H. Nguyen, H. Vu, T.-H. Tran, and Q.-H. Nguyen, ‘‘Developing a way-finding system on mobile robot assisting visually impaired people in an indoor environment,’’ Multimedia Tools Appl., vol. 76, no. 2, pp. 2645–2669, Jan. 2017.
R. Tapu, B. Mocanu, and T. Zaharia, ‘‘A computer vision-based perception system for visually impaired,’’ Multimedia Tools Appl., vol. 76, no. 9, pp. 11771–11807, May 2017.
D. Dakopoulos and N. G. Bourbakis, “Wearable obstacle avoidance elec[1]tronic travel aids for blind: A survey,” IEEE Trans. Syst. Man, Cybern. Part C Appl. Rev., vol. 40, no. 1, pp. 25–35, Jan. 2010.
E. E. Pissaloux, R. Velazquez, and F. Maingreaud, “A new framework for cognitive mobility of visually impaired users in using tactile device,” IEEE Trans. Human-Mach. Syst., vol. 47, no. 6, pp. 1040–1051, Dec. 2017.
K. Patil, Q. Jawadwala, and F. C. Shu, “Design and construction of electronic aid for visually impaired people,” IEEE Trans. Human-Mach. Syst., vol. 48, no. 2, pp. 172–182, Apr. 2018.
J. Bai, S. Lian, Z. Liu, K. Wang, and D. Liu, “Virtual-blind-road following based wearable navigation device for blind people,” IEEE Trans. Consum. Electron., vol. 64, no. 1, pp. 136–143, Feb. 2018
J. Bai, S. Lian, Z. Liu, K. Wang, and D. Liu, “Smart guiding glasses for visually impaired people in indoor environment,” IEEE Trans. Consum. Electron., vol. 63, no. 3, pp. 258–266, Aug. 2017.
R. K. Katzschmann, B. Araki, and D. Rus, “Safe local navigation for visually impaired users with a time-of-flight and haptic feedback device,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 26, no. 3, pp. 583–593, Mar. 2018.
J. Villanueva and R. Farcy, “Optical device indicating a safe free path to blind people,” IEEE Trans. Instrum. Meas., vol. 61, no. 1, pp. 170–177, Jan. 2012.
B. Jiang, J. Yang, Z. Lv, and H. Song, “Wearable vision assistance system based on binocular sensors for visually impaired users,” IEEE Internet Things J., vol. 6, no. 2, pp. 1375–1383, Apr. 2019
X. Yang, S. Yuan, and Y. Tian, “Assistive clothing pattern recognition for visually impaired people,” IEEE Trans. Human-Mach. Syst., vol. 44, no. 2, pp. 234–243, Apr. 2014.
L. Tepelea, I. Buciu, C. Grava, I. Gavrilut, and A. Gacsadi, “A vision module for visually impaired people by using raspberry PI platform,” in Proc.15th Int. Conf. Eng. Modern Electr. Syst. (EMES), Oradea, Romania, 2019, pp. 209–212.
B. Li et al., “Vision-based mobile indoor assistive navigation aid for blind people,” IEEE Trans. Mobile Comput., vol. 18, no. 3, pp. 702–714, Mar. 2019
L. Dunai, G. Peris-Fajarnés, E. Lluna, and B. Defez, “Sensory naviga[1]tion device for blind people,” J. Navig., vol. 66, no. 3, pp. 349–362, May 2013.
V.-N. Hoang, T.-H. Nguyen, T.-L. Le, T.-H. Tran, T.-P. Vuong, and N. Vuillerme, “Obstacle detection and warning system for visually impaired people based on electrode matrix and mobile kinect,” Vietnam J. Comput. Sci., vol. 4, no. 2, pp. 71–83, Jul. 2016.
C. I. Patel, A. Patel, and D. Patel, “Optical character recognition by open-source OCR tool Tesseract: A case study,” Int. J. Comput. Appl., vol. 55, no. 10, pp. 50–56, Oct. 2012.
R. Keefer, Y. Liu, and N. Bourbakis, “The development and evaluation of an eyes-free interaction model for mobile reading devices,” IEEE Trans. Human-Mach. Syst., vol. 43, no. 1, pp. 76–91, Jan. 2013.
Chalamandaris, S. Karabetsos, P. Tsiakoulis, and S. Raptis, “A unit selection text-to-speech synthesis system optimized for use with screen readers,” IEEE Trans. Consum. Electron., vol. 56, no. 3, pp. 1890–1897, Aug. 2010.
Andò, S. Baglio, V. Marletta, and A. Valastro, “A haptic solution to assist visually impaired in mobility tasks,” IEEE Trans. Human-Mach. Syst., vol. 45, no. 5, pp. 641–646, Oct. 2015.
V. V. Meshram, K. Patil, V. A. Meshram, and F. C. Shu, “An astute assistive device for mobility and object recognition for visually impaired people,” IEEE Trans. Human-Mach. Syst., vol. 49, no. 5, pp. 449–460, Oct. 2019
J. Xiao, S. L. Joseph, X. Zhang, B. Li, X. Li, and J. Zhang, “An assistive navigation framework for the visually impaired,” IEEE Trans. Human [1] Mach. Syst., vol. 45, no. 5, pp. 635–640, Oct. 2015.
F. Lan, G. Zhai, and W. Lin, “Lightweight smart glass system with audio aid for visually impaired people,” in Proc. IEEE Region 10 Conf., Macao, China, 2015, pp. 1–4.
M. M. Islam, M. S. Sadi, K. Z. Zamli, and M. M. Ahmed, “Developing walking assistants for visually impaired people: A review,” IEEE Sens. J., vol. 19, no. 8, pp. 2814–2828, Apr. 2019.
D. Dakopoulos and N. G. Bourbakis, “Wearable obstacle avoidance elec[1]tronic travel aids for blind: A survey,” IEEE Trans. Syst. Man, Cybern. Part C Appl. Rev., vol. 40, no. 1, pp. 25–35, Jan. 2010.
S. Al-Fahoum, H. B. Al-Hmoud, and A. A. Al-Fraihat, ‘‘A smart infrared microcontroller-based blind guidance system,’’ Act. Passive Electron. Compon., vol. 2013, pp. 1–7, Jun. 2013.
A. Nada, M. A. Fakhr, and A. F. Seddik, ‘‘Assistive infrared sensor based smart stick for blind people,’’ in Proc. Sci. Inf. Conf. (SAI), Jul. 2015, pp. 1149–1154.
Wachaja, P. Agarwal, M. R. Adame, K. Möller, and W. Burgard, ‘‘A navigation aid for blind people with walking disabilities,’’ in Proc. IROS Workshop Rehabil. Assistive Robot., Sep. 2014, pp. 13–14.
M. H. A. Wahab, A. A. Talib, H. A. Kadir, A. Johari, A. Noraziah, R. M. Sidek, and A. A. Mutalib, ‘‘Smart cane: Assistive cane for visually impaired people,’’ IJCSI International Journal of Computer Science Issues, Vol. 8, Issue 4, No 2, pp 21, July 2011.
J. Bai, S. Lian, Z. Liu, K. Wang, and D. Liu, “Smart guiding glasses for visually impaired people in indoor environment,” IEEE Trans. Consum. Electron., vol. 63, no. 3, pp. 258–266, Aug. 2017.
Blindness and vision impairment, World Health Organization, Geneva, Switzerland, Oct. 2019.
S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137–1149, Jun. 2017.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
