In this project, our primary objective was to predict geomagnetic storms based on the DSCOVR (Deep Space Climate Observatory) data spanning the years 2016 to 2023.

Below you can see our forecast visualization with an explanation of the storm's level of danger.

https://storm-prophet.synergist.kiev.ua/

The DSCOVR dataset consisted of Level 1 (raw data) information, encompassing various parameters. The first three columns described the Interplanetary Magnetic Field (IMF) variations and the rest represented the parameters of solar wind flow. These last parameters are usually used to obtain Level 2 features like the speed, the temperature and the density of the solar wind. Because the DSCOVR mission was supposed to end in 2020, the Faraday cups responsible for the flux parameters detection are subject to anomalies, like increased noise or temporal malfunction. With additional data (like wind composition) one could estimate the Level 2 data. This approach, more physical, would consist in the analysis of this Level 2 data to make predictions by solving Magnetohydrodynamic equations or analyzing Birkeland currents. But modeling is always limited by human factors and error propagation in such complex massive systems involving non trivial physical phenomena is significant. Our team thus opted for a purely machine learning approach.

We divided our approach into two categories:

Very Accurate Short-Term Predictions (up to 14 hours per minute): For detailed forecasting needs.

Accurate Long-Term Predictions (up to 30 days per hour): For more extended forecasting horizons.

Our initial strategy involved rigorous data preprocessing. To streamline the data and reduce complexity, we computed specific parameters such as vector sum, its dynamism, arithmetic mean and standard deviation of faraday cup plasma detector performance, condensing the original 54 columns into a more meaningful 8 columns. Additionally, we calculated hourly averages and deviations from minute-level data to better suit our prediction models.

The display below allows one to visualize the correlations between DSCOVR data, at the top (here number magnetic vector dynamics, average and standard deviation of Faraday cup detectors values), and the DST (taken from World Data Center for Geomagnetism, Kyoto), at the bottom (only storms appear clearly).

https://storm-prophet.synergist.kiev.ua/view.php

The cornerstone of our predictive modeling was the Long Short-Term Memory (LSTM) neural network, which excels at handling sequence-to-sequence problems. This choice was critical because we needed to capture the temporal correspondence between different time periods. For instance, storm data originating over several hours could manifest on Earth after a day.

Key features of our LSTM model included:

Sequence-to-Sequence Modeling: This allowed us to capture the temporal relationships in the data, essential for predicting geomagnetic storm levels, which depend on previous time periods.
Data Preprocessing: To reduce complexity, we conducted logical data preprocessing. This involved reducing the number of columns by calculating specific parameters and using feature importance score.
Model Architecture: Our LSTM model architecture included dropout layers to prevent overfitting, dense layers with various activation functions, and regularization techniques such as L2 regularization.
Early Stopping: We implemented early stopping as a technique to mitigate overfitting during the training process. This involved monitoring the validation loss and restoring the best weights when the loss stopped improving.
Optimization: We utilized the Adam optimizer with a specific learning rate to efficiently update the model's weights during training.
Mean Squared Error Loss: To measure the model's performance, we used the mean squared error (MSE) loss function. It quantified the discrepancy between the predicted and actual values, allowing us to assess the accuracy of our predictions.
Permutation Feature Importance: To gain insights into the significance of different input features, we applied permutation feature importance. This technique involved shuffling specific features in the test data, calculating predictions with the shuffled data, and comparing the mean squared error to a baseline performance metric. Features with a significant impact on predictions were identified through this process.
Visualizing Feature Importance: We visualized the results of permutation feature importance using a bar chart. This provided a clear representation of the relative importance of each feature in the model's predictions.
Evaluation Metric: We used the disturbance storm time (DST) as the output variable to assess storm levels. DST values below -50 were indicative of a geomagnetic storm. Our model's performance was evaluated based on the average deviation from the expected DST values.
Epoch Count: On average, we conducted 50 training epochs to fine-tune our LSTM model.

What about the anomalies? According to the furnished challenge description these appear only in 2020 (5 years after DSCOVR launch). Our idea was to train a reverse LSTM model on the years without anomalies (2017-2020), by feeding DST and IMF features to retrieve the Faraday cup detectors measurements and then compare our Faraday cup parameters forecast to the actual noisy data in the period presenting anomalies (2020-2023). This method allowed us to identify the anomalies.

We take additional parameters like standard deviation and average values of detectors and number of non working detectors (over an hour). All the raw data regarding the detectors can be summarized with these features. Here you can see the fitting on the trained data set of these metrics (time in hours).

This first image shows a relatively neat fit of standard deviation over time for 100 first hours from the 2017-2020 dataset. The second one shows a forecast of standard deviation over time for 100 first hours from the 2021-2023 dataset. One can notice that this last fit is less precise, most likely due to anomalies. A closer analysis needs to be performed to identify the exact correlations between the anomalies and geomagnetic activity.

2023 NASA Space Apps Challenge

Storm Prophet