Overview of Satellite-Derived Surface Solar Radiation Datasets useful for Nowcasting 🌞

[irenelivia]

This issue aims to consolidate and clarify the current set of satellite-derived datasets relevant for solar radiation nowcasting within the MLCast community.

The goal is to:

• Provide a clear overview of available and relevant satellite-derived surface solar radiation datasets
• Clarify differences in data source, processing level, and intended use
• Identify preferred datasets (or complementary roles)
• Serve as a hub linking to dataset-specific sub-issues

This follows recent discussions in the MLCast community about avoiding duplication of preprocessing efforts and aligning on a shared AI-ready data foundation.

---

🧭 Motivation

There is currently some ambiguity across the community regarding:

• Which satellite-derived solar radiation datasets exist
• How they differ in terms of:
  • satellite source (MSG vs MTG)
  • processing method
  • spatial/temporal resolution
  • intended use (climate product vs AI-ready training data)

This issue is intended to resolve that by collecting and structuring the landscape.

---

Summary tables

🟦 1. Operational input datasets

These datasets represent the direct satellite observation space, closest to the operational data that flows into met services through EUMETCAST.

They are intended to be used as:

• inputs (X) to ML models

Dataset	Satellite	Role
MSG SEVIRI L1C	MSG (operational)	Raw multispectral radiances
MTG FCI L1C (proposed)	MTG (operational)	Operational successor to MSG

Related issue on obtaining an ML-ready zarr dataset from MTG FCI level 1c data: #43

🟨 2. Derived geophysical products (target variables) for surface solar radiation (SSR)

These datasets represent physically interpreted variables, derived from satellite observations via retrieval algorithms or radiative transfer models.

They are typically used as:

• training targets
• evaluation / benchmarking datasets
• physical reference products

Dataset	Satellite/Instrument	Variable	Type
MSGCPP	MSG SEVIRI	GHI	Operational surface solar radiation + clearsky radiation product
SARAH-3	MSG SEVIRI	GHI	Climate data record of surface solar radiation, 5 day latency
HANNA	MSG SEVIRI	GHI	High-resolution surface solar radiation demonstrator
DWD SSR	MSG SEVIRI / MTG FCI (emerging)	GHI	Operational surface solar radiation product
LSA-SAF MDSSFTD	MSG SEVIRI	GHI?	Surface solar radiation (land only)

🛰️ Candidate Surface Solar Radiation datasets

1. MSGCPP (MSG Cloud Physical Properties) (KNMI)

• Source: Meteosat Second Generation (MSG)
• Type: Derived product (radiative transfer / physical retrievals)
• Variables: Surface solar radiation, clearsky radiation, etc.
• Resolution: ~3 km, 15 min
• Coverage: Europe (cropped in available datasets)
• Status: Partial Zarr conversion exists (incomplete 👉 Related issue: Adding the MSGCPP dataset #13)
• Notes:
  • Already used in MLCast community (e.g. @mfroelund started transforming some MSGCPP data to zarr 👉 Related issue: Adding the MSGCPP dataset #13
    )
• Archive from late 2024 available on KNMI's open data platform: https://dataplatform.knmi.nl/dataset/msg-cpp-products-1-0

---

2. SARAH-3 (CM SAF)

• Source: MSG-based climate data record
• Type: Surface solar radiation (climate product)
• Resolution: ~0.05° (~5 km), 30 min
• Notes:
  • Long-term climate consistency focus
  • Not optimized for ML nowcasting workflows
  • Useful for climatology / baseline comparisons / emulating surface solar radiation from raw satellite channels
  • More information and archive: https://user.eumetsat.int/catalogue/EO:EUM:DAT:0863

---

3. HANNA (CM SAF demonstrator)

• Source: MSG channels
• Type: High-resolution surface solar radiation
• Resolution: ~0.05° (~1 km), 15 min
• Notes:
  • Very high spatial resolution compared to other SSR products
  • Still a demonstrator product, 2 years available
  • More information: https://www.cmsaf.eu/EN/Products/Demonstrator_Data_Records/Hanna/Hanna_node.htm

---

4. DWD SSR product (MSG + MTG)

• Source: DWD satellite processing chain
• Type: Surface solar radiation
• Notes:
  • Existing MSG-based dataset available internally
  • MTG-based SSR product also exists
  • Not currently available as ML-ready archive
  • Recent (latest two weeks) data available here: https://opendata.dwd.de/weather/satellite/radiation/sis/

---

5 LSA-SAF MDSSFTD (EUMETSAT SAF)

• Source: MSG + Land Surface Analysis SAF
• Type: Surface solar radiation product
• Coverage: Land-only
• Resolution: ~3–5 km, 15 min
• Notes:
  • Used in solar applications (e.g. retraining IrradianceNet @olah-soma, operationally used @geosphere with IrradPhyDNet and DE_330 on euro-cordex domain, used in post-processing for PV production nowcasts)
  • Limited by land-only coverage

❓ Open questions for the community

1. Which derived SSR product(s) should be used as:
   • training targets?
   • evaluation benchmarks?
2. Do we want a community benchmark comparison paper/study across SSR products?

---

👥 Contributors / interested parties

Tagging MLCasters possibly interested in this discussion:
@pdebuyl @ladc @leifdenby @franchg

A quick look at some of the datasets by @irenelivia:

[sidekock]

At a conference at the end of last year ([EUMETNET - E-TREND]), I discussed a number of these datasets with some people (including @ischicker). I don't know exactly who raised the issue, but from what I remember, the SARAH-3 dataset performed quite poorly compared to ground observations, particularly under snow cover. I think an analysis was done by Alberto Carpentieri (https://www.sciencedirect.com/science/article/pii/S0038092X23001810), but it may be with slightly older products. These points could be a good starting point.