Deep Learning Models for Tracking Solar Eruptive Features

Segmentation and tracking of solar eruptive events based on ground based and remote instrument data is essential in pursuit of better answers about the physical nature of these phenomena. A Data-driven approach appears as an obvious solution; nevertheless, engineering of training sets is a challenging task as obtaining labeled ground truth masks requires a significant amount of manual work. Existing algorithmic tracking software does not allow to automate the process significantly, as each event or set of the images usually requires separate re-configuration of the software. Thus, we have invested significant effort in improving our data-driven, machine learning approach to achieving state-of-the-art segmentation of dim but fast features on the solar surface. Below, we outline the approach used in an upcoming publication (Stepanyuk & Kozarev, under review at JGR Machine Learning).

We developed deeper, 14-layer U-Net-based convolutional neural network (CNN) models, trained on very limited datasets consisting of several hundred base-difference SDO/AIA 193 images. To enhance and generalize the training datasets, we developed and implemented algorithms for extensive geometric data augmentation (right panels in Fig. 1) via shifting and rotation, which increases the generality of the model training. The model training was further protected from overfitting by the use of ‘skip connections’ and dropouts in the network architecture. Finally, we implemented patch learning, in which the input images are split into a large number (>50) of sub-images, passed on to the model for training (left panel in Fig. 1). These enlarge and diversify even further the training dataset, making the model more robust.

Fig. 1 – Training dataset augmentation techniques for solar eruptive feature detection and tracking. Left Panel demonstrates the splitting of input images into separate patches, which are passed to the model for training. Right Panel demonstrates rotation augmentation for two events from the training set. Rotated BD images are shown on the first and third row, with corresponding feature masks (ground truth) in the second and fourth rows, respectively. Columns correspond to random angles chosen by the algorithm during one of the training steps on synthetic data.

We show an application of the current model to a coronal bright front in Fig. 2. In the top row we show the input of three running difference AIA images separated by 48 seconds. The CBF expands away from the source in the south. The front of the circular wave is slightly discernible. The bottom row shows the hybrid model’s output as feature probability. The oval shape of the CBF is clearly visible, split into a northeast and a southwest front. This is best visible on the bottom center image. As this on-disk CBF was not particularly bright, the results of applying the model to these observations shows that there are a number of small features attributed to the front. To reduce this uncertainty and improve the model performance further, we have introduced an additional step in the analysis, in which we pass the resulting set of probability maps through an optical flow algorithm, as outlined below. This velocity estimation based on optical flow algorithms improves the performance of the proposed tracking algorithm.

Fig. 2 – Application of the hybrid CNN segmentation model to the May 10, 2022 CBF. Input running difference AIA 193  ̊Aimages are shown in the upper row. The image segmentation model output is shown in the bottom row. The intensity of each pixel corresponds to the probability that this pixel belongs to the wavefront as determined by the model.

Figure 3 shows the result of applying an optical flow algorithm to the coronal bright front from Fig. 2. The first panel shows an AIA running difference frame for context. In panel II, the CBF shape was captured well by the hybrid CNN model. Nevertheless, some pixel groups were also considered by the CNN as high probability pixels, while they clearly do not belong to the CBF. Such segmentation quality could be enough for human visual interpretation of images, but may be problematic in automated pipelines. Panel III shows the velocity field obtained with the Horn-Schunk (HS) optical flow algorithm. The EUV wave is more clearly distinguished from the static details with similar intensities and shapes. Finally, the thresholded velocity fields as calculated with the HS algorithm are presented in Panel IV in the figure, isolating the feature due to its higher velocities.

Fig. 3 – CBF velocity field calculation. The Horn–Schunk algorithm was applied to the results of the CNN image segmentation model. The initial Running Difference (RD) image and feature mask are presented in panels I and II. Panels III and IV correspond to the unfiltered and thresholded velocity fields.

These images illustrate the idea that velocity field estimation techniques allow to perform image segmentation even with greater precision by distinguishing moving details. Thus, our newly proposed method not only validates the velocity estimation by independent means, but also improves the detection and tracking of CBFs. The proposed approach and models we develop is a step forward towards automatic data-driven segmentation in the heliophysics domain and may contribute to better understanding of the physics behind observed phenomena.