SDO_training

 # ML-Ready Multi-Modal Image Dataset from SDO ## Overview This dataset provides machine learning (ML)-ready solar data curated from NASA’s Solar Dynamics Observatory (SDO), covering observations from **May 13, 2010, to Dec 31, 2024**. It includes Level-1.5 processed data from: **Atmospheric Imaging Assembly (AIA)** and **Helioseismic and Magnetic Imager (HMI)**. The dataset is designed to facilitate large-scale learning applications in heliophysics, such as space weather forecasting, unsupervised representation learning, and scientific foundation model development. ## Dataset Download Instructions To download the data please visit https://registry.opendata.aws/surya-bench/ - Resource type: `S3 Bucket` - Amazon Resource Name (ARN): `arn:aws:s3:::nasa-surya-bench` - AWS CLI Access (No AWS account required): `aws s3 ls --no-sign-request s3://nasa-surya-bench/` --- ## Dataset Structure **Data Variables:** ```text - aia94 (y, x) float32 : AIA 94 Å - aia131 (y, x) float32 : AIA 131 Å - aia171 (y, x) float32 : AIA 171 Å - aia193 (y, x) float32 : AIA 193 Å - aia211 (y, x) float32 : AIA 211 Å - aia304 (y, x) float32 : AIA 304 Å - aia335 (y, x) float32 : AIA 335 Å - aia1600 (y, x) float32 : AIA 1600 Å (UV continuum) - hmi_m (y, x) float32 : HMI LOS Magnetogram - hmi_bx (y, x) float32 : HMI Magnetic Field - x component - hmi_by (y, x) float32 : HMI Magnetic Field - y component - hmi_bz (y, x) float32 : HMI Magnetic Field - z component - hmi_v (y, x) float32 : HMI Doppler Velocity ``` ## Dataset Details | Field | Description | |------------------------|---------------------------------------------| | **Temporal Coverage** | May 13, 2010 – Dec 31, 2024 | | **Data Format** | netCDF (`.nc`), float32 | | **Temporal Granularity**| 12 minutes | | **Data Shape** | `[13, 4096, 4096]` per file | | **Channels** | 13 total (AIA EUV ×8 + HMI magnetograms ×5) | | **Size per File** | ~570 MB | | **Total Size** | ~360TB | --- ## 📦 Downstream Data Repositories All the downstream tasks that uses core-sdo dataset can be found in the [SuryaBench Hugging Face Collections](https://huggingface.co/collections/nasa-ibm-ai4science/suryabench) Each sub-dataset targets a specific task within the heliophysics domain: | Repository | Task Description | |------------|------------------| | [`Surya-bench-solarwind`](https://huggingface.co/datasets/nasa-ibm-ai4science/Surya-bench-solarwind) | Solar wind speed prediction with a 4-day forecast horizon. | | [`surya-bench-flare-forecasting`](https://huggingface.co/datasets/nasa-ibm-ai4science/surya-bench-flare-forecasting) | Binary classification for solar flare occurrence within 24 hours. | | [`surya-bench-ar-segmentation`](https://huggingface.co/datasets/nasa-ibm-ai4science/surya-bench-ar-segmentation) | Pixel-wise segmentation of active regions from solar disk images. | | [`euv_spectra`](https://huggingface.co/datasets/nasa-ibm-ai4science/euv-spectra) | Time-aligned Extreme Ultraviolet (EUV) irradiance spectra from NASA’s SDO/EVE instrument. | | [`surya-bench-coronal-extrapolation`](https://huggingface.co/datasets/nasa-ibm-ai4science/surya-bench-coronal-extrapolation) | Magnetic field extrapolation from photosphere to corona. | | [`ar_emergence`](https://huggingface.co/datasets/nasa-ibm-ai4science/ar_emergence) | Forecasting active region emergence based on historical features. | ## License This dataset is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) License. #### Authors Sujit Roy, Dinesha V Hegde, Johannes Schmude, Rohit Lal, Vishal Gaur corr: [e-mail] # 可直接用于机器学习的SDO多模态图像数据集 ## 概述 本数据集从美国国家航空航天局（National Aeronautics and Space Administration, NASA）的太阳动力学观测台（Solar Dynamics Observatory, SDO）中精选整理得到可直接用于机器学习（Machine Learning, ML）的太阳观测数据，观测时间范围覆盖**2010年5月13日至2024年12月31日**，包含经过1.5级（Level-1.5）处理的**大气成像组件（Atmospheric Imaging Assembly, AIA）**与**日震与磁成像仪（Helioseismic and Magnetic Imager, HMI）**观测数据。 本数据集旨在助力日物理学领域的大规模机器学习应用，例如空间天气预报、无监督表征学习以及科学基础模型研发。 --- ## 数据集结构 **数据变量：** text - aia94 (y, x) float32 : AIA 94埃波段数据 - aia131 (y, x) float32 : AIA 131埃波段数据 - aia171 (y, x) float32 : AIA 171埃波段数据 - aia193 (y, x) float32 : AIA 193埃波段数据 - aia211 (y, x) float32 : AIA 211埃波段数据 - aia304 (y, x) float32 : AIA 304埃波段数据 - aia335 (y, x) float32 : AIA 335埃波段数据 - aia1600 (y, x) float32 : AIA 1600埃波段（紫外连续谱）数据 - hmi_m (y, x) float32 : HMI视线方向磁图（LOS Magnetogram） - hmi_bx (y, x) float32 : HMI磁场x分量 - hmi_by (y, x) float32 : HMI磁场y分量 - hmi_bz (y, x) float32 : HMI磁场z分量 - hmi_v (y, x) float32 : HMI多普勒速度数据 ## 数据集详情 | 字段 | 描述 | |------------------------|---------------------------------------------| | **时间覆盖范围** | 2010年5月13日 — 2024年12月31日 | | **数据格式** | netCDF（`.nc`），float32 精度 | | **时间粒度** | 12分钟 | | **数据形状** | 单个文件为 `[13, 4096, 4096]` | | **总通道数** | 共13通道（8通道AIA极紫外（Extreme Ultraviolet, EUV）数据 + 5通道HMI磁图数据） | | **单文件大小** | 约570 MB | | **总数据量** | 约360 TB | --- ## 备注 一个月的训练数据集已以Parquet格式上传至Hugging Face主分支的并行分支中。 完整数据集存储于亚马逊网络服务（Amazon Web Services, AWS）的S3存储桶中，需注意完整数据集容量超过360 TB。用户可通过以下命令查看完整文件列表： bash aws s3 ls s3://nasa-surya-bench --no-sign-request 若需从AWS S3存储桶中下载单个文件，可参考该链接中的教程：[https://docs.aws.amazon.com/AmazonS3/latest/userguide/download-objects.html]。目前提供了多种从S3下载或同步数据的方案。 ## 作者 Sujit Roy、Dinesha Vasanta Hegde、Johannes Schmude、Amy Lin、Vishal Gaur、Talwinder Singh、Rohit Lal 通讯邮箱：[e-mail]

搜集汇总

背景与挑战

背景概述

该数据集是一个由NASA和IBM合作提供的机器学习就绪多模态太阳观测数据集，源自太阳动力学观测站（SDO），覆盖2010年5月13日至2024年12月31日的AIA和HMI Level-1.5处理数据。它包含13个通道的太阳图像，总规模约360TB，旨在支持太阳物理领域的空间天气预报和基础模型开发等应用。