TY - JOUR
T1 - Incorporation of feature engineering and attention mechanisms into deep learning models to develop an early warning system for harmful algal blooms
AU - Kim, Tae Ho
AU - Shin, Jihoon
AU - Cha, Yoon Kyung
N1 - Publisher Copyright:
© 2023 Elsevier Ltd
PY - 2023/8/15
Y1 - 2023/8/15
N2 - The accurate forecasting of harmful algal blooms (HABs) is hindered due to insufficient monitoring frequency, numerous interacting factors, and large temporal variability in cyanobacteria abundance. In this study, the combined use of multiple feature engineering and attention mechanisms in deep learning (DL) models was explored to improve the performance, temporal resolution, and explainability of HAB forecasts at varying forecast horizons (i.e., 1, 7, and 14 days). Incorporation of feature engineering and attention mechanism into the DL models enabled daily forecasts of HABs without requiring any manual data manipulation. The hybrid DL models were applied to a site in the lower Nakdong River, South Korea, where HABs continue to be a significant water quality problem. Various input features pertaining to meteorological, hydrological, environmental, and biological factors were used to forecast cyanobacteria abundance. Regardless of the feature engineering and attention mechanisms, 1-day forecasts (root mean square error [RMSE] = 0.316–0.405, R2 = 0.985–0.991, mean absolute error [MAE] = 0.241–0.295, symmetric mean absolute percentage error [SMAPE] = 21.998%–22.581%) were much more accurate than 7-day and 14-day forecasts (RMSE = 2.335–2.669, R2 = 0.347–0.500, MAE = 1.830–2.170, SMAPE = 30.256%–34.348%). Among the two feature engineering mechanisms, bidirectional recurrent imputation for time series slightly but consistently outperformed unidirectional recurrent imputation for time series, with the performance difference increasing with an increase in the forecast horizon. Among the two attention mechanisms, although the reverse time attention mechanism (RMSE = 0.316–2.669, R2 = 0.347–0.991, MAE = 0.241–2.156, SMAPE = 22.429%–32.780%) and dual-stage attention-based recurrent neural network (RMSE = 0.382–2.622, R2 = 0.370–0.987, MAE = 0.288–2.170, SMAPE = 21.998%–34.348%) exhibited similar performance, explanations derived by the former had clearer distinctions regarding the relative importance of different input features and time steps. The results of this study reveal that explanations can differ considerably between different attention mechanisms, which necessitate further investigation to ensure the credibility of attention-based DL models.
AB - The accurate forecasting of harmful algal blooms (HABs) is hindered due to insufficient monitoring frequency, numerous interacting factors, and large temporal variability in cyanobacteria abundance. In this study, the combined use of multiple feature engineering and attention mechanisms in deep learning (DL) models was explored to improve the performance, temporal resolution, and explainability of HAB forecasts at varying forecast horizons (i.e., 1, 7, and 14 days). Incorporation of feature engineering and attention mechanism into the DL models enabled daily forecasts of HABs without requiring any manual data manipulation. The hybrid DL models were applied to a site in the lower Nakdong River, South Korea, where HABs continue to be a significant water quality problem. Various input features pertaining to meteorological, hydrological, environmental, and biological factors were used to forecast cyanobacteria abundance. Regardless of the feature engineering and attention mechanisms, 1-day forecasts (root mean square error [RMSE] = 0.316–0.405, R2 = 0.985–0.991, mean absolute error [MAE] = 0.241–0.295, symmetric mean absolute percentage error [SMAPE] = 21.998%–22.581%) were much more accurate than 7-day and 14-day forecasts (RMSE = 2.335–2.669, R2 = 0.347–0.500, MAE = 1.830–2.170, SMAPE = 30.256%–34.348%). Among the two feature engineering mechanisms, bidirectional recurrent imputation for time series slightly but consistently outperformed unidirectional recurrent imputation for time series, with the performance difference increasing with an increase in the forecast horizon. Among the two attention mechanisms, although the reverse time attention mechanism (RMSE = 0.316–2.669, R2 = 0.347–0.991, MAE = 0.241–2.156, SMAPE = 22.429%–32.780%) and dual-stage attention-based recurrent neural network (RMSE = 0.382–2.622, R2 = 0.370–0.987, MAE = 0.288–2.170, SMAPE = 21.998%–34.348%) exhibited similar performance, explanations derived by the former had clearer distinctions regarding the relative importance of different input features and time steps. The results of this study reveal that explanations can differ considerably between different attention mechanisms, which necessitate further investigation to ensure the credibility of attention-based DL models.
KW - Cyanobacteria
KW - Explainable artificial intelligence
KW - Harmful algal bloom
KW - Hybrid deep learning
KW - Recurrent imputation for time series
KW - Reverse time attention mechanism
UR - http://www.scopus.com/inward/record.url?scp=85161279059&partnerID=8YFLogxK
U2 - 10.1016/j.jclepro.2023.137564
DO - 10.1016/j.jclepro.2023.137564
M3 - Article
AN - SCOPUS:85161279059
SN - 0959-6526
VL - 414
JO - Journal of Cleaner Production
JF - Journal of Cleaner Production
M1 - 137564
ER -