High-Performance Video Retrieval Using SIFT and Deep Learning Methods
Keywords:
SIFT, Shot Boundary Detection, CNN, RNN, LSTM, Key Frame Extraction, CBVRAbstract
Shot boundary detection and key frame extraction are critical steps for video indexing, summarization, and retrieval. This paper proposes an advanced approach that combines a sophisticated SIFT (Scale-Invariant Feature Transform) keypoint matching algorithm with deep learning-based key frame extraction techniques. The SIFT-based method effectively captures both abrupt and gradual shot transitions, overcoming the limitations of existing algorithms that perform well only for abrupt changes. For key frame extraction, a deep learning framework leveraging convolutional neural networks (CNNs) for spatial feature representation and recurrent neural networks (RNNs/LSTMs) for temporal modeling is employed. This approach automatically identifies the most informative and representative frames while reducing redundancy, enabling efficient processing of large-scale video data. Extensive experiments on benchmark video datasets demonstrate that the proposed algorithms significantly outperform traditional methods in terms of accuracy, robustness, and computational efficiency, making them highly suitable for modern video analysis applications.
Downloads
References
C. Cotsaces, N. Nikolaidis, and I. Pitas, “Video Shot Detection and Condensed Representation”, IEEE Signal Processing Magazine, March, 2006, pp. 28-37, 2006
Srinivasa Naveen Kumar, G., Reddy, V.S.K., Balivada, L.K. (2023). Content-Based Video Retrieval Using Deep Learning Algorithms. In: Reddy, V.S., Prasad, V.K., Wang, J., Rao Dasari, N.M. (eds) Intelligent Systems and Sustainable Computing. ICISSC 2022. Smart Innovation, Systems and Technologies, vol 363. Springer, Singapore. https://doi.org/10.1007/978-981-99-4717-1_52
J. H. Yuan, H. Y. Wang, and B. Zhang, “A formal study of shot boundary detection”, Journal of Transactions on Circuits and Systems for Video Technology, vol. 17, no. 2, pp. 168-186, February 2007.
Zhao Guang-sheng , A Novel Approach for Shot Boundary Detection and Key Frames Extraction, 2008 International Conference on Multimedia and Information Technology,IEEE
GS Naveen Kumar, VSK Reddy, “Detection of Shot Boundaries and Extraction of Key Frames for Video Retrieval”, International Journal of Knowledge-Based and Intelligent Engineering Systems, Vol. 24, Issue 1, ISSN: 1327-2314, Indexed in: Scopus, Web of Science: Emerging Sources Citation Index, DBLP, Google Scholar, ACM Digital Library
GS Naveen Kumar, VSK Reddy, S. Srinivas Kumar “Video Shot Boundary Detection and Keyframe Extraction for Video Retrieval”, International Conference on Computational Intelligence and Informatics ICCII-2017, Sep 2017, Springer Publication, AISC Series, (Scopus Indexed)
A. Hassanien, M. Elgharib, A. Selim, S.-H. Bae, M. Hefeeda, and W. Matusik, “Large-scale, Fast and Accurate Shot Boundary Detection through Spatio-temporal Convolutional Neural Networks,” arXiv preprint arXiv:1705.03281, 2017.
A. Benoughidene and F. Titouna, “A Novel Method for Video Shot Boundary Detection Using a CNN-LSTM Approach,” International Journal of Multimedia Information Retrieval, vol. 11, pp. 653–667, 2022.
Jordi Mas and Gabriel Fernandez, “Video Shot Boundary Detection based on Color Histogram”, Digital Television Center (CeTVD), La Salle School of Engineering, Ramon Llull University, Barcelona, Spain.
D. G. Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, vol. 60, pp. 91-110, 2004.
J. Zhang, Y. Wang, and D. Li, “Shot Boundary Detection with 3D Depthwise Convolutions and Visual Attention,” Sensors, vol. 23, no. 16, p. 7022, 2023.
Hannane, Rachida, et al. "An efficient method for video shot boundary detection and keyframe extraction using SIFT-point distribution histogram." International Journal of Multimedia Information Retrieval 5.2 (2016): 89-104.
GS Naveen Kumar, VSK Reddy “Key frame Extraction using Rough Set Theory for Video Retrieval”, International Conference on Soft Computing and Signal Processing ICSCSP-2018, Jun 2018, Springer Publication, AISC series, Scopus Indexed
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
