Multi Disaster Building Damage Assessment with Deep Learning using Satellite Imagery Data
Keywords:
Disaster, building damage, deep learning, satellite imageryAbstract
Massive disaster events that have an impact on infrastructure damage, especially buildings, require the latest tools and technology so that an assessment of the damage that occurs can be carried out quickly and efficiently. Artificial Intelligence (AI), especially with machine learning (ML) and deep learning (DL), is one approach that can be a solution. The study uses satellite imagery data from the xBD dataset repository with a proposed two-stage deep learning model to assess the level of damage to buildings consisting of a building segmentation approach using the PSPNet and UNet models with the ResNet-18 backbone and the ResNet50 model for classification. The flood fill algorithm is inserted between the segmentation and classification stages with the aim of producing better extraction of segmentation results. The deep learning model with PSPNet and ResNet-18 produces an evaluation value and an accuracy value of building damage classification which is slightly better than the previous building segmentation research, with F1 values of 0.8494 and 0.8338 for precision, respectively. Referring to the resulting evaluation value, future research is still very open to developing models to achieve better results.
Downloads
References
D. Pavlinovic, “Climate and weather-related disasters surge five-fold over 50 years, but early warnings save lives-WMO report,” UN News, 2021. https://news.un.org/en/story/2021/09/1098662
H. Ritchie, P. Rosado, and M. Roser, “Natural Disasters Data Explorer,” Natural Disasters, 2018. https://ourworldindata.org/natural-disasters
S. Li and H. Tang, “Classification of Building Damage Triggered by Earthquakes Using Decision Tree,” Math Probl Eng, vol. 2020, pp. 1–15, 2020, doi: 10.1155/2020/2930515.
H. Ma, Y. Liu, Y. Ren, D. Wang, L. Yu, and J. Yu, “Improved CNN classification method for groups of buildings damaged by earthquake, based on high resolution remote sensing images,” Remote Sens (Basel), vol. 12, no. 2, pp. 1–16, Jan. 2020, doi: 10.3390/rs12020260.
L. Ma, Y. Liu, X. Zhang, Y. Ye, G. Yin, and B. A. Johnson, “Deep learning in remote sensing applications: A meta-analysis and review,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 152, pp. 166–177, 2019, doi: 10.1016/j.isprsjprs.2019.04.015.
Y. Hu, S. Gao, D. Lunga, W. Li, S. Newsam, and B. Bhaduri, “GeoAI at ACM SIGSPATIAL,” SIGSPATIAL Special, vol. 11, no. 2, pp. 5–15, Dec. 2019, doi: 10.1145/3377000.3377002.
H. Ghanbari, M. Mahdianpari, S. Homayouni, and F. Mohammadimanesh, “A Meta-Analysis of Convolutional Neural Networks for Remote Sensing Applications,” IEEE J Sel Top Appl Earth Obs Remote Sens, vol. 14, pp. 3602–3613, 2021, doi: 10.1109/JSTARS.2021.3065569.
A. Trekin, G. Novikov, G. Potapov, V. Ignatiev, and E. Burnaev, “Satellite imagery analysis for operational damage assessment in Emergency situations,” in International Conference on Business Information Systems, Feb. 2018, pp. 347–358. [Online]. Available: http://arxiv.org/abs/1803.00397
Q. D. Cao and Y. Choe, “Building Damage Annotation on Post-Hurricane Satellite Imagery Based on Convolutional Neural Networks,” Natural Hazards, vol. 103, no. 3, pp. 3357–3376, Jul. 2018, doi: 10.1007/s11069-020-04133-2.
I. Bouchard, M. È. Rancourt, D. Aloise, and F. Kalaitzis, “On Transfer Learning for Building Damage Assessment from Satellite Imagery in Emergency Contexts,” Remote Sens (Basel), vol. 14, no. 11, pp. 1–29, Jun. 2022, doi: 10.3390/rs14112532.
R. Gupta et al., “Creating xBD: A dataset for assessing building damage from satellite imagery,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Nov. 2019, pp. 10–17. [Online]. Available: http://arxiv.org/abs/1911.09296
Y. Lecun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, May 2015, doi: 10.1038/nature14539.
V. François-Lavet, P. Henderson, R. Islam, M. G. Bellemare, and J. Pineau, “An introduction to deep reinforcement learning,” Foundations and Trends in Machine Learning, vol. 11, no. 3–4, pp. 219–354, Dec. 2018, doi: 10.1561/2200000071.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” in Proceedings of the IEEE, 1998, vol. 86, no. 11, pp. 2278–2323. doi: 10.1109/5.726791.
Y. Heryadi and E. Irwansyah, Deep Learning: Aplikasinya di Bidang Geospasial. Depok - Jawa Barat: AWI Technology Press, 2020. Accessed: Dec. 25, 2022. [Online]. Available: https://books.google.com/books/about/Deep_Learning_Aplikasinya_di_Bidang_Geos.html?id=UorwDwAAQBAJ
K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” in 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, Sep. 2015, pp. 1–14. [Online]. Available: http://arxiv.org/abs/1409.1556
O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical image computing and computer-assisted intervention, 2015, vol. 9351, pp. 234–241. doi: 10.1007/978-3-319-24574-4_28.
B. Leibe, J. Matas, N. Sebe, and M. Welling, “Identity Mappings in Deep Residual Networks,” European conference on computer vision, vol. 9906 LNCS. Springer Verlag, pp. 630-645-, 2016. doi: 10.1007/978-3-319-46493-0.
L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam, “Rethinking Atrous Convolution for Semantic Image Segmentation,” arXiv preprint arXiv:1706.05587, Jun. 2017, [Online]. Available: http://arxiv.org/abs/1706.05587
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Commun ACM, vol. 60, no. 6, pp. 84–90, Jun. 2017, doi: 10.1145/3065386.
R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2014, pp. 580–587. [Online]. Available: http://arxiv.
R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Region-Based Convolutional Networks for Accurate Object Detection and Segmentation,” IEEE Trans Pattern Anal Mach Intell, vol. 38, no. 1, pp. 142–158, Jan. 2016, doi: 10.1109/TPAMI.2015.2437384.
E. Shelhamer, K. Rakelly, J. Hoffman, and T. Darrell, “Clockwork Convnets for Video Semantic Segmentation,” in European Conference on Computer Vision, Aug. 2016, pp. 852–868. [Online]. Available: http://arxiv.org/abs/1608.03609
J. Long, E. Shelhamer, and T. Darrell, “Fully Convolutional Networks for Semantic Segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2019, vol. 7, pp. 43369–43382.
Y. Liao, M. E. Mohammadi, and R. L. Wood, “Deep learning classification of 2D orthomosaic images and 3D point clouds for post-event structural damage assessment,” Drones, vol. 4, no. 2, pp. 1–19, Jun. 2020, doi: 10.3390/drones4020024.
R. Gupta and M. Shah, “RescueNet: Joint Building Segmentation and Damage Assessment from Satellite Imagery,” in Proceedings - International Conference on Pattern Recognition, Apr. 2020, pp. 4405–4411. [Online]. Available: http://arxiv.org/abs/2004.07312
J. Sublime and E. Kalinicheva, “Automatic post-disaster damage mapping using deep-learning techniques for change detection: Case study of the Tohoku tsunami,” Remote Sens (Basel), vol. 11, no. 9, pp. 1–20, May 2019, doi: 10.3390/rs11091123.
D. E. Kislov and K. A. Korznikov, “Automatic windthrow detection using very-high-resolution satellite imagery and deep learning,” Remote Sens (Basel), vol. 12, no. 7, pp. 1–17, Apr. 2020, doi: 10.3390/rs12071145.
A. J. Cooner, Y. Shao, and J. B. Campbell, “Detection of urban damage using remote sensing and machine learning algorithms: Revisiting the 2010 Haiti earthquake,” Remote Sens (Basel), vol. 8, no. 10, pp. 1–17, Oct. 2016, doi: 10.3390/rs8100868.
W. Liu and F. Yamazaki, “Extraction of collapsed bridges due to the 2011 Tohoku-oki earthquake from post-event SAR images,” Journal of Disaster Research, vol. 13, no. 2, pp. 281–290, Mar. 2018, doi: 10.20965/jdr.2018.p0281.
F. Wang, J. P. Kerekes, Z. Xu, and Y. Wang, “Residential roof condition assessment system using deep learning,” J Appl Remote Sens, vol. 12, no. 01, pp. 1–20, Mar. 2018, doi: 10.1117/1.jrs.12.016040.
W. Zhang and M. Xu, “Translate SAR Data into Optical Image Using IHS and Wavelet Transform Integrated Fusion,” Journal of the Indian Society of Remote Sensing, vol. 47, no. 1, pp. 125–137, Jan. 2019, doi: 10.1007/s12524-018-0879-7.
J. Shao, L. Tang, M. Liu, G. Shao, L. Sun, and Q. Qiu, “BDD-net: A general protocol for mapping buildings damaged by a wide range of disasters based on satellite imagery,” Remote Sensing, vol. 12, no. 10. MDPI AG, pp. 1–11, May 01, 2020. doi: 10.3390/rs12101670.
J. Zhao et al., “A rapid public health needs assessment framework for after major earthquakes using high-resolution satellite imagery,” Int J Environ Res Public Health, vol. 15, no. 6, pp. 1–18, Jun. 2018, doi: 10.3390/ijerph15061111.
B. J. Wheeler and H. A. Karimi, “Deep learning-enabled semantic inference of individual building damage magnitude from satellite images,” Algorithms, vol. 13, no. 8, pp. 1–15, Aug. 2020, doi: 10.3390/A13080195.
S. Cotrufo, C. Sandu, F. Giulio Tonolo, and P. Boccardo, “Building damage assessment scale tailored to remote sensing vertical imagery,” Eur J Remote Sens, vol. 51, no. 1, pp. 991–1005, Jan. 2018, doi: 10.1080/22797254.2018.1527662.
W. Liu et al., “SSD: Single Shot MultiBox Detector Wei,” in European Conference on Computer Vision, 2016, vol. 9905, pp. 21–37. doi: 10.1007/978-3-319-46448-0.
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779–788. [Online]. Available: http://pjreddie.com/yolo/
H. Ma, Y. Liu, Y. Ren, and J. Yu, “Detection of collapsed buildings in post-earthquake remote sensing images based on the improved YOLOv3,” Remote Sens (Basel), vol. 12, no. 1, pp. 1–19, Jan. 2020, doi: 10.3390/RS12010044.
F. Nex, D. Duarte, F. G. Tonolo, and N. Kerle, “Structural building damage detection with deep learning: Assessment of a state-of-the-art CNN in operational conditions,” Remote Sens (Basel), vol. 11, no. 23, pp. 1–17, Dec. 2019, doi: 10.3390/rs11232765.
D. Duarte, F. Nex, N. Kerle, and G. Vosselman, “Multi-resolution feature fusion for image classification of building damages with convolutional neural networks,” Remote Sens (Basel), vol. 10, no. 10, pp. 1–26, Oct. 2018, doi: 10.3390/rs10101636.
E. Weber and H. Kané, “Building Disaster Damage Assessment in Satellite Imagery with Multi-Temporal Fusion,” arXiv preprint arXiv:2004.05525, pp. 1–7, Apr. 2020, [Online]. Available: http://arxiv.org/abs/2004.05525
Y. Shen et al., “BDANet: Multiscale Convolutional Neural Network with Cross-Directional Attention for Building Damage Assessment from Satellite Images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, 2022, doi: 10.1109/TGRS.2021.3080580.
H. Xiao, Y. Peng, H. Tan, and P. Li, “Dynamic Cross Fusion Network for Building-Based Damage Assessment,” in Proceedings - IEEE International Conference on Multimedia and Expo, 2021, pp. 1–6. doi: 10.1109/ICME51207.2021.9428414.
H. Chen, E. Nemni, S. Vallecorsa, X. Li, C. Wu, and L. Bromley, “Dual-Tasks Siamese Transformer Framework for Building Damage Assessment,” arXiv preprint arXiv:2201.10953, pp. 1600–1603, Jan. 2022, [Online]. Available: http://arxiv.org/abs/2201.10953
S. Shrestha and L. Vanneschi, “Improved fully convolutional network with conditional random fields for building extraction,” Remote Sens (Basel), vol. 10, no. 7, Jul. 2018, doi: 10.3390/rs10071135.
E. Irwansyah, A. A. S. Gunawan, and Nurhasanah, “Deep Learning Model Comparison for Dense Building Segmentation in the City Using Aerial Imagery Data,” in 2022 International Conference on Science and Technology, ICOSTECH 2022, 2022, pp. 25–27. doi: 10.1109/ICOSTECH54296.2022.9828812.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
