Detecting Environmental Anomalies: Variational Autoencoder- Based Analysis of Air Quality Time Series Data
Keywords:
Anomaly Detection, Variational Autoencoder, Unsupervised Learning, Air QualityAbstract
Air pollution presents a major environmental challenge, necessitating effective methods for rapid detection of pollution episodes to protect public health and economic interests. This study proposes a novel method combining Variational Autoencoders (VAEs) and Random Forest classifiers to identify anomalies in multivariate air quality time series data. The analysis focuses on key pollutants (NO2, PM10, PM2.5, O3, CO, and SO2) and meteorological variables, utilizing data from three monitoring stations over three years. By applying pre-processing techniques and dataset balancing with SMOTE, the hybrid model's performance is evaluated using various metrics. The results highlight the model's robustness in detecting air quality anomalies across different scenarios. Moreover, t-SNE visualizations of the encoded latent space reveal discernible patterns. This study underscores the potential of integrating deep learning with ensemble learning to improve air quality monitoring systems and suggests avenues for future enhancements in broader environmental monitoring applications.
Downloads
References
C. S. Bhosale, et al., "Ambient Air Quality Monitoring with Reference to Particulate Matter (PM10) in Kolhapur City," Nature Environment and Pollution Technology, vol. 22, no. 4, pp. 2029-2037, 2023.
S. K. Guttikunda and B. R. Gurjar, "Role of meteorology in seasonality of air pollution in megacity Delhi, India," Environmental Monitoring and Assessment, vol. 184, pp. 3199-3211, 2012.
W. Jiao, et al., "Community Air Sensor Network (CAIRSENSE) project: evaluation of low-cost sensor performance in a suburban environment in the southeastern United States," Atmospheric Measurement Techniques, vol. 9, no. 11, pp. 5281-5292, 2016.
G. Tancev, "Relevance of drift components and unit-to-unit variability in the predictive maintenance of low-cost electrochemical sensor systems in air quality monitoring," Sensors, vol. 21, no. 9, p. 3298, 2021.
T.-B. Ottosen and P. Kumar, "Outlier detection and gap filling methodologies for low-cost air quality measurements," Environmental Science: Processes & Impacts, vol. 21, no. 4, pp. 701-713, 2019.
F. Rollo, C. Bachechi, and L. Po, "Anomaly detection and repairing for improving air quality monitoring," Sensors, vol. 23, no. 2, p. 640, 2023.
M. Fahim and A. Sillitti, "Anomaly detection, analysis and prediction techniques in IoT environment: A systematic literature review," IEEE Access, vol. 7, pp. 81664-81681, 2019.
D. Park, Y. Hoshi, and C. C. Kemp, "A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder," IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 1544-1551, 2018.
J. Loy-Benitez, S. Heo, and C. Yoo, "Imputing missing indoor air quality data via variational convolutional autoencoders: Implications for ventilation management of subway metro systems," Building and Environment, vol. 182, p. 107135, 2020.
T. Burki, "WHO introduces ambitious new air quality guidelines," The Lancet, vol. 398, no. 10306, p. 1117, 2021.
J. S. Apte, et al., "High-resolution air pollution mapping with Google street view cars: exploiting big data," Environmental Science & Technology, vol. 51, no. 12, pp. 6999-7008, 2017.
L. Spinelle, et al., "Field calibration of a cluster of low-cost commercially available sensors for air quality monitoring. Part B: NO, CO and CO2," Sensors and Actuators B: Chemical, vol. 238, pp. 706-715, 2017.
N. Zimmerman, et al., "A machine learning calibration model using random forests to improve sensor performance for lower-cost air quality monitoring," Atmospheric Measurement Techniques, vol. 11, no. 1, pp. 291-313, 2018.
M. Zaidan, et al., "Dense air quality sensor networks: Validation, analysis, and benefits," IEEE Sensors Journal, vol. 22, no. 23, pp. 23507-23520, 2022.
[15] G. Li and J. J. Jung, "Deep learning for anomaly detection in multivariate time series: Approaches, applications, and challenges," Information Fusion, vol. 91, pp. 93-102, 2023.
X. Teng, Y.-R. Lin, and X. Wen, "Anomaly detection in dynamic networks using multi-view time-series hypersphere learning," in Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp. 827-836, 2017.
Q. Wang, et al., "A novel hybridization of nonlinear grey model and linear ARIMA residual correction for forecasting US shale oil production," Energy, vol. 165, pp. 1320-1331, 2018.
N. Shaadan, et al., "Anomaly detection and assessment of PM10 functional data at several locations in the Klang Valley, Malaysia," Atmospheric Pollution Research, vol. 6, no. 2, pp. 365-375, 2015.
S. Schmidl, et al., "Anomaly detection in time series: a comprehensive evaluation," Proceedings of the VLDB Endowment, vol. 15, no. 9, pp. 1779-1797, 2022.
D. Liu, et al., "Aqeyes: visual analytics for anomaly detection and examination of air quality data," arXiv preprint arXiv:2103.12910, 2021.
B. Actkinson and R. Griffin, "Detecting plumes in mobile air quality monitoring time series with density-based spatial clustering of applications with noise," Atmospheric Measurement Techniques Discussions, pp. 1-22, 2023.
X. Cheng, et al., "Climate modulation of Niño3.4 SST-anomalies on air quality change in southern China: Application to seasonal forecast of haze pollution," Atmospheric Research, vol. 225, pp. 157-164, 2019.
Y. Wei, et al., "LSTM-autoencoder-based anomaly detection for indoor air quality time-series data," IEEE Sensors Journal, vol. 23, no. 4, pp. 3787-3800, 2023.
X. Shu, et al., "Unsupervised dam anomaly detection with spatial–temporal variational autoencoder," Structural Health Monitoring, vol. 22, no. 1, pp. 39-55, 2023.
Z. Li, Y. Sun, L. Yang, Z. Zhao, and X. Chen, "Unsupervised machine anomaly detection using autoencoder and temporal convolutional network," IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1-13, 2022.
Desai, C. Freeman, Z. Wang, and I. Beaver, "TimeVAE: A variational auto-encoder for multivariate time series generation," arXiv preprint arXiv:2111.08095, 2021.
N. Oreshkin, D. Carpov, N. Chapados, and Y. Bengio, "N-BEATS: Neural basis expansion analysis for interpretable time series forecasting," arXiv preprint arXiv:1905.10437, 2019.
O. Rybkin, K. Daniilidis, and S. Levine, "Simple and effective VAE training with calibrated decoders," in International Conference on Machine Learning, pp. 9179-9189, PMLR, 2021.
I. Higgins, L. Matthey, A. Pal, C. P. Burgess, X. Glorot, M. M. Botvinick, S. Mohamed, and A. Lerchner, "beta-VAE: Learning basic visual concepts with a constrained variational framework," ICLR (Poster), vol. 3, 2017.
A. J. Lew and M. J. Buehler, "Encoding and exploring latent design space of optimal material structures via a VAE-LSTM model," Forces in Mechanics, vol. 5, p. 100054, 2021.
M. Memarzadeh, B. Matthews, and I. Avrekh, "Unsupervised anomaly detection in flight data using convolutional variational auto-encoder," Aerospace, vol. 7, no. 8, p. 115, 2020.
S. K. Zhou and R. Chellappa, "From sample similarity to ensemble similarity: Probabilistic distance measures in
reproducing kernel Hilbert space," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 6, pp. 917-929, 2006.
J. Davis and M. Goadrich, "The relationship between Precision-Recall and ROC curves," in Proceedings of the 23rd International Conference on Machine Learning, pp. 233-240, 2006.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
