SceneGuide: An Indoor and Outdoor Scene Recognition Wearable Aid for Visually Impaired People
Keywords:
Computer vision, machine learning, scene recognition, visually impaired, wearable aidAbstract
This paper proposes a new SceneGuide wearable aid for providing information about the surrounding scene to the visually impaired people. Its main feature is its ability to understand the scene and offer simplified information in an intuitive way. SceneGuide aid is designed as a wearable jacket with low-power embedded processing unit, monocular camera, and Bluetooth headphones. It is a lightweight, low-cost, battery-operated blind assistive aid. The aid employs a novel, computationally efficient model, using multi-feature fusion and multi-level optimum feature selection approach. SceneGuide serves as a complementary assistive aid to the conventional white cane and helps reduce the cognitive information load and anxiety experienced by visually impaired people. The functional evaluation of the aid presented scene recognition accuracy of 95.25% on a custom dataset and 85.82% on the 15 Scene Standard Dataset. This aid was evaluated with 10 blindfolded volunteers. The volunteers expressed 77% acceptance towards usability to identify the scene with lower levels of confusion and anxiety. This highlights that the SceneGuide aid can enhance the understanding of visually impaired people about their surroundings.
Downloads
References
World Report on Vision. 2022. [Online]. Available: https://www.who.int/news-room/fact-sheets/detail/blindness-and-visual-impairment
Seybold, Diana. "Investigating stress associated with mobility training through consumer discussion groups." Journal of Visual Impairment & Blindness 87, no. 4 (1993): 111-112.
S. Bhatlawande, M. Mahadevappa, J. Mukherjee, M. Biswas, D. Das and S. Gupta, "Design, Development, and Clinical Evaluation of the Electronic Mobility Cane for Vision Rehabilitation," in IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 22, no. 6, pp. 1148-1159, Nov. 2014..
Lopes, Sérgio I., José MN Vieira, Óscar FF Lopes, Pedro RM Rosa, and Nuno AS Dias. "MobiFree: a set of electronic mobility aids for the blind." Procedia Computer Science 14 (2012): 10-19.
Balakrishnan, Meenakshi Rao, Parigi Vedanti Madhusudhan Valiyaveetil, Sashi Kumar Paul, Rohan Venkatesan, Arun Kumar Harikesavan, Karthikeyan Kolappan, Bhagavatheesh Chanana, Piyush Mehra, Dheeraj, "A Split Grip Cane Handle Unit With Tactile Feedback for Directed Ranging", Patent, WO/2015/121872, issued 20/08/2015
Dernayka, Aya, Michel-Ange Amorim, Roger Leroux, Lucas Bogaert, and René Farcy. "Tom Pouce III, an electronic white cane for blind people: Ability to detect obstacles and mobility performances." Sensors 21, no. 20 (2021): 6854.
S. S. Bhatlawande, J. Mukhopadhyay and M. Mahadevappa, "Ultrasonic spectacles and waist-belt for visually impaired and blind person," 2012 National Conference on Communications (NCC), Kharagpur, India, 2012, pp. 1-4.
Katzschmann, Robert K., Brandon Araki, and Daniela Rus. "Safe local navigation for visually impaired users with a time-of-flight and haptic feedback device." IEEE Transactions on Neural Systems and Rehabilitation Engineering 26, no. 3 (2018): 583-593.
Kilian, Jakob, Alexander Neugebauer, Lasse Scherffig, and Siegfried Wahl. "The unfolding space glove: A wearable spatio-visual to haptic sensory substitution device for blind people." Sensors 22, no. 5 (2022): 1859.
J. Bai, S. Lian, Z. Liu, K. Wang and D. Liu, "Virtual-Blind-Road Following-Based Wearable Navigation Device for Blind People," in IEEE Transactions on Consumer Electronics, vol. 64, no. 1, pp. 136-143, Feb. 2018.
Meshram, V. V., Patil, K., Meshram, V. A., & Shu, F. C. (2019). An astute assistive device for mobility and objectrecognition for visually impaired people. IEEE Transactions on Human-Machine Systems, 49(5), 449-460.
Dutta, Senjuti, Mridul S. Barik, Chandreyee Chowdhury, and Deep Gupta. "Divya-Dristi: A smartphone-based campus navigation system for the visually impaired." In 2018 Fifth International Conference on Emerging Applications of Information Technology (EAIT), pp. 1-3. IEEE, 2018.
Garcia-Macias, J. Antonio, Alberto G. Ramos, Rogelio Hasimoto-Beltran, and Saul E. Pomares Hernandez. "Uasisi: A modular and adaptable wearable system to assist the visually impaired." Procedia Computer Science 151 (2019): 425-430.
Tap Tap See App, 2012 [Mobile App]. Available: https://play.google.com/store/apps/details?id=com.msearcher.taptapsee.android&hl=en_US
Be My Eyes App, 2015 [Mobile App]. Available: https://play.google.com/store/apps/details?id=com.bemyeyes.bemyeyes&hl=en&gl=US
Szummer, Martin, and Rosalind W. Picard. "Indoor-outdoor image classification." In Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database, pp. 42-51. IEEE, 1998.
Oliva, Aude, and Antonio Torralba. "Modeling the shape of the scene: A holistic representation of the spatial envelope." International journal of computer vision 42 (2001): 145-175.
Han, Yina, and Guizhong Liu. "A hierarchical GIST model embedding multiple biological feasibilities for scene classification." In 2010 20th International Conference on Pattern Recognition, pp. 3109-3112. IEEE, 2010.
Lowe, David G. "Distinctive image features from scale-invariant keypoints." International journal of computer vision 60 (2004): 91-110.
Fei-Fei, Li, and Pietro Perona. "A bayesian hierarchical model for learning natural scene categories." In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), vol. 2, pp. 524-531. IEEE, 2005.
Lazebnik, Svetlana, Cordelia Schmid, and Jean Ponce. "Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories." In 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06), vol. 2, pp. 2169-2178. IEEE, 2006.
Yang, Jianchao, Kai Yu, Yihong Gong, and Thomas Huang. "Linear spatial pyramid matching using sparse coding for image classification." In 2009 IEEE Conference on computer vision and pattern recognition, pp. 1794-1801. IEEE, 2009.
Sánchez, Jorge, Florent Perronnin, Thomas Mensink, and Jakob Verbeek. "Image classification with the fisher vector: Theory and practice." International journal of computer vision 105 (2013): 222-245.
Ojala, Timo, Matti Pietikäinen, and David Harwood. "A comparative study of texture measures with classification based on featured distributions." Pattern recognition 29, no. 1 (1996): 51-59.
Xiao, Jianxiong, Krista A. Ehinger, James Hays, Antonio Torralba, and Aude Oliva. "Sun database: Exploring a large collection of scene categories." International Journal of Computer Vision 119 (2016): 3-22.
Ahonen, Timo, Jiří Matas, Chu He, and Matti Pietikäinen. "Rotation invariant image description with local binary pattern histogram fourier features." In Image Analysis: 16th Scandinavian Conference, SCIA 2009, Oslo, Norway, June 15-18, 2009. Proceedings 16, pp. 61-70. Springer Berlin Heidelberg, 2009.
Song, Kechen, and Yunhui Yan. "A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects." Applied Surface Science 285 (2013): 858-864.
Qian, Xueming, Xian-Sheng Hua, Ping Chen, and Liangjun Ke. "PLBP: An effective local binary patterns texture descriptor with pyramid representation." Pattern Recognition 44, no. 10-11 (2011): 2502-2515.
Wu, Jianxin, and Jim M. Rehg. "Centrist: A visual descriptor for scene categorization." IEEE transactions on pattern analysis and machine intelligence 33, no. 8 (2010): 1489-1501.
Dalal, Navneet, and Bill Triggs. "Histograms of oriented gradients for human detection." In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), vol. 1, pp. 886-893. Ieee, 2005.
Xiao, Jianxiong, Krista A. Ehinger, James Hays, Antonio Torralba, and Aude Oliva. "Sun database: Exploring a large collection of scene categories." International Journal of Computer Vision 119 (2016): 3-22.
Ke, Yan, and Rahul Sukthankar. "PCA-SIFT: A more distinctive representation for local image descriptors." In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004., vol. 2, pp. II-II. IEEE, 2004.
Malhi, Arnaz, and Robert X. Gao. "PCA-based feature selection scheme for machine defect classification." IEEE transactions on instrumentation and measurement 53, no. 6 (2004): 1517-1525.
Ye, Jieping, Ravi Janardan, and Qi Li. "Two-dimensional linear discriminant analysis." Advances in neural information processing systems 17 (2004).
Prewitt, Judith MS. "Object enhancement and extraction." Picture processing and Psychopictorics 10, no. 1 (1970): 15-19.
LeCun, Yann, Léon Bottou, Yoshua Bengio, and Patrick Haffner. "Gradient-based learning applied to document recognition." Proceedings of the IEEE 86, no. 11 (1998): 2278-2324.
Gong, Yunchao, Liwei Wang, Ruiqi Guo, and Svetlana Lazebnik. "Multi-scale orderless pooling of deep convolutional activation features." In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VII 13, pp. 392-407. Springer International Publishing, 2014.
Yang, Songfan, and Deva Ramanan. "Multi-scale recognition with DAG-CNNs." In Proceedings of the IEEE international conference on computer vision, pp. 1215-1223. 2015.
Chen, Liang-Chieh, George Papandreou, Florian Schroff, and Hartwig Adam. "Rethinking atrous convolution for semantic image segmentation." arXiv preprint arXiv:1706.05587 (2017).
He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Deep residual learning for image recognition." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778. 2016.
Szegedy, Christian, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. "Going deeper with convolutions." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1-9. 2015.
Huang, Gao, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q. Weinberger. "Densely connected convolutional networks." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4708. 2017.
Wang, Chen, Guohua Peng, and Bernard De Baets. "Deep feature fusion through adaptive discriminative metric learning for scene recognition." Information Fusion 63 (2020): 1-12.
Wang, Chen, Guohua Peng, and Bernard De Baets. "Embedding metric learning into an extreme learning machine for scene recognition." Expert Systems with Applications 203 (2022): 117505.
Parshapa, P. ., & Rani, P. I. . (2023). A Survey on an Effective Identification and Analysis for Brain Tumour Diagnosis using Machine Learning Technique. International Journal on Recent and Innovation Trends in Computing and Communication, 11(3), 68–78. https://doi.org/10.17762/ijritcc.v11i3.6203
Muñoz, S., Hernandez, M., González, M., Thomas, P., & Anderson, C. Enhancing Engineering Education with Intelligent Tutoring Systems using Machine Learning. Kuwait Journal of Machine Learning, 1(2). Retrieved from http://kuwaitjournals.com/index.php/kjml/article/view/116
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
