Wildfire–Prithvi Research Notes

Purpose: an evidence-bound working document for writing a student symposium abstract and poster. This is not a replacement for the frozen study record or the statistical inference ledger.

Review date: 2026-09-01
Scientific snapshot reviewed: ad954faaf5cfc26e18b689f4f45c0bd808e99744

Poster-readiness decision

READY for a student research symposium/poster, with important caveats.

The repository supports a strong methods-forward poster about detecting a geographic shortcut, redesigning the task, and obtaining preliminary held-out conditional-ranking evidence. It does not support an unqualified claim that Prithvi predicts wildfire risk, detects a causal pre-fire precursor, removes all geographic confounding, or generalizes beyond the studied Morocco cohort.

Current validation evidence:

full test suite: 271 passed, 6 skipped;
dependency check: clean;
source dataset: 934,586 rows × 241 columns, SHA-256 266f5e2e385a4695c30f02446950ffb016c9b12b5140636906788d8a3c508c7e;
pooled embeddings: 104,068 rows × 192 dimensions, SHA-256 86600556533aa183e14431a410d84ced544d4260510cf0d380cbf7f78a152402;
recovery-v2: both external payloads verified with rclone check;
independent repaired-record verdict: RESEARCH RECORD STRONG WITH IMPORTANT CAVEATS.

Recommended abstract framing

Working title

From Geographic Shortcut to Temporal Ranking: Evaluating Frozen Prithvi Representations for Recorded Wildfire Dates

Shorter alternatives:

Shortcut-Aware Wildfire-Date Ranking with Prithvi EO
Can Frozen Prithvi Features Rank Recorded Fire Dates Within Local Areas?
Redesigning Wildfire-Date Evaluation as Coarse-Cell-Conditioned Temporal Ranking

Avoid titles that say “wildfire forecasting,” “early warning,” “risk mapping,” or “causal precursor detection.” Those claims exceed the current evidence.

One-sentence research question

After a shortcut-prone row-classification task was redesigned as label-blind ranking within the same 10-km grid cell and year, could frozen pre-event Prithvi earth-observation scores rank a recorded fire date above recorded no-fire dates, and could a validation-frozen calendar–EO blend improve on EO-only ranking?

One-sentence contribution

The main contribution is an auditable workflow that exposed a geographic shortcut, reconstructed the prediction target as conditional temporal ranking, froze a chronological evaluation, and narrowed its claims after dependence-aware and fine-geography audits.

Preferred story arc

Shortcut discovered → task redesigned → frozen test evaluated → inference repaired → claim narrowed.

That methodological story is stronger and more defensible than presenting the project as a competition for the highest model score.

Study in plain language

The original task classified individual rows as recorded fire or recorded no-fire. It produced high test scores, but latitude and longitude alone nearly matched the main models. The result therefore largely reflected where fires were recorded.

The redesigned study instead compared dates within the same coarse geographic unit. Each risk set contained:

one row dated when a fire was recorded; and
one or more rows dated when no fire was recorded according to the dataset definition.

The model’s task was to rank the recorded fire date above the recorded no-fire dates. This is a conditional recorded-date ranking problem, not estimation of population wildfire probability.

Data and EO pipeline facts

Geography: Morocco.
Source period: 2013-01-31 through 2022-12-23.
Source labels: 467,293 recorded-fire and 467,293 recorded-no-fire rows.
Production EO cohort: 2020 onward.
EO-eligible rows: 104,068 of 108,881.
Imagery: HLS S30 preferred, with L30 fallback.
Sequence: four distinct acquisitions in [D-90, D), at least five days apart; no target-day or post-event imagery.
Chip: 224 × 224 native 30-m pixels.
Bands: Blue, Green, Red, Narrow NIR, SWIR1, SWIR2.
Encoder: frozen ibm-nasa-geospatial/Prithvi-EO-2.0-tiny-TL.
Primary representation: one finite 192-dimensional embedding per eligible row.
Production eo_contract_v2 used granule cloud metadata and six-band spectral-validity/nodata masking. It did not use local pixel-level Fmask.

Prithvi also received acquisition timing and row latitude/longitude metadata. That detail matters when discussing residual fine-location information.

Why the original task was rejected

Evidence class: ROBUSTNESS / DIAGNOSTIC

Diagnostic	PR-AUC	ROC-AUC
Original tabular model A	0.888164	0.903101
Original Prithvi model B	0.893739	0.928278
Coordinates only	0.881665	0.895315

Nearest-neighbor labels agreed 98.05% overall and 99.11% within 1 km. Coordinates alone therefore nearly reproduced the early result. The scientifically important finding was not that the model performed well, but that the original question allowed a severe geographic shortcut.

The redesigned shared-centroid GEO control had within-risk-set concordance 0.500000, exactly chance. Do not plot the original coordinate-only ROC-AUC and redesigned GEO concordance as a before/after performance drop: they are different metrics on different estimands.

Redesigned primary study

Risk-set construction

label-blind fixed-origin 10-km grid;
one recorded fire row and one or more recorded no-fire rows per set;
same cell and same year;
controls within ±14 days of known fires excluded;
347 total modeled risk sets and 1,613 row instances;
chronological split:

Split	Year	Risk sets	Rows
TRAIN	2020	224	1,042
VALIDATION	2021	100	479
TEST	2022	23	92

Use “same coarse cell,” not “same point” or “same exact location.”

Frozen models

A — calendar ranker: timing-safe calendar features.
B — EO ranker: frozen 192-D Prithvi embedding.
C — fixed multimodal blend: 0.4 × p_A + 0.6 × p_B, selected before TEST.
GEO — negative control: shared cell-centroid coordinates, constant within a risk set.
CAL — simple baseline: logistic regression on calendar variables.

No target labels entered EO discovery, frame selection, preprocessing, or Prithvi inference.

Metrics

Top-1: tie-adjusted credit when the recorded fire date has the highest score in its set.
MRR: mean reciprocal rank of the recorded fire date.
Concordance: probability that the model scores the fire date above a control date within the same risk set; ties receive 0.5.

Primary held-out result

Evidence class: PRIMARY HELD-OUT for point estimates

The 2022 TEST was opened once after 2020 training and 2021 validation:

Model	Top-1	MRR	Concordance
A — calendar	0.4388	0.6475	0.6110
B — Prithvi EO	0.4094	0.6406	0.6273
C — fixed blend	0.5145	0.7028	0.6853

All three C–B point differences favor C. The point estimates alone do not show which differences are statistically stable.

Repaired uncertainty

Evidence class: ROBUSTNESS / REPAIR, not a new held-out test

The original bootstrap treated 23 risk sets independently. Reanalysis mapped them to 16 connected events; four reused controls linked two events, producing 15 conservative event-plus-shared-row overlap components. A deterministic 20,000-replicate component bootstrap gave:

C minus B metric	Difference	95% component-clustered CI	Interpretation
Top-1	+0.1051	[0.0000, 0.2375]	touches zero
MRR	+0.0623	[0.0059, 0.1369]	survives narrowly
Concordance	+0.0580	[−0.0052, 0.1412]	inconclusive

All C–A and B–A component-clustered intervals were inconclusive. The safe primary interpretation is therefore:

On a small frozen 2022 TEST, the fixed blend had positive point differences over EO-only across all ranking metrics, but only the MRR difference retained a 95% overlap-component interval wholly above zero.

Do not repeat the historical statement that C significantly beat B on all three metrics.

Fine-geography limitation

Evidence class: ROBUSTNESS / REPAIR

Matching controlled a nominal 10-km cell, not an identical spatial point:

exact-coordinate R1 sets: 0/352;
exact-coordinate case-control pairs: 0/1,656;
median case-control distance: 5.3835 km;
only 53 sets from 46 events kept every control within 5 km.

The corrected representation-distance analysis retained only 330 cross-component shared-endpoint pairs across six cells. Its correlations are descriptive only; no valid dependence-preserving permutation was available.

Safe conclusion:

Coarse geography is controlled, but exact geography is not. Fine-location confounding remains plausible and unresolved.

This does not prove the EO association is geographic. It prevents attributing it solely to temporal or biophysical wildfire processes.

Optional exploratory findings

Evidence class: POST-HOC EXPLORATORY

These results belong in the poster body or discussion, not in the abstract unless explicitly labeled secondary.

Low-capacity EO head

Model	Concordance
B0 — original pooled EO baseline	0.5300
B2 — pooled embedding + learned EO head	0.5976

Historical B2–B0 difference: +0.0676 [0.0208, 0.1171]. This remains conditional on post-hoc model selection and event-only rather than conservative overlap-component inference.

Nested fusion sensitivity

C11 was selected in all five fusion-layer outer folds:

Contrast	Concordance difference	95% overlap-component CI
Nested fusion − A	+0.0571	[0.0145, 0.0961]
Nested fusion − CAL	+0.0459	[−0.0226, 0.0989]
Nested fusion − B2	+0.0110	[−0.0103, 0.0342]

The fusion-layer choice was nested, but its saved A/B2 base scores were not regenerated under a fully outer-nested scheme. The correct conclusion is an exploratory association over A—not established superiority over CAL or B2.

Negative findings worth preserving

Naive early fusion underperformed the original A/B row models.
The 13-set fresh-event attempt was underpowered and did not reproduce the primary C–B advantage.
Richer B3–B8 spatial/temporal architectures did not reliably beat B2.
C11 did not establish a concordance gain over B2 or CAL.
The 0–15-day exploratory C11–A difference was exactly zero with a wide CI.

Including at least one negative result strengthens the symposium story by showing that the analysis was not filtered to positive outcomes.

Abstract sentence bank

Background

Foundation-model representations may support wildfire analysis, but spatially structured labels can let models succeed by recognizing location rather than short-term conditions.

Objective

We tested whether frozen pre-event Prithvi representations could rank a recorded fire date above recorded no-fire dates within the same coarse geographic cell and year, alone and in a validation-fixed blend with calendar scores.

Methods

After coordinate-only diagnostics exposed a spatial shortcut, we constructed 347 retrospective, label-blind temporal risk sets in Morocco within 10-km cells, trained on 2020, selected models on 2021, and evaluated once on 23 2022 risk sets from 16 connected events.

Primary result

The fixed blend achieved Top-1 0.5145, MRR 0.7028, and concordance 0.6853; relative to EO-only, only the repaired MRR difference remained above zero under a 15-component bootstrap (+0.0623, 95% CI 0.0059–0.1369).

Limitation

Shared cell centroids controlled coarse geography, but candidates did not share exact coordinates, leaving fine-location confounding unresolved.

Takeaway

The study provides preliminary shortcut-aware evidence for conditional recorded-date ranking and demonstrates why task design and dependence-aware inference are essential before interpreting high wildfire-model scores.

Optional exploratory sentence

In post-hoc event-grouped analyses, a low-capacity head improved decoding of pooled Prithvi embeddings, whereas richer EO architectures and fusion did not establish superiority over the strongest simple comparators.

Conservative abstract skeleton

Aim for 180–230 words. Replace bracketed transitions, not the scientific content:

Background: Foundation-model representations may support wildfire analysis, but spatially structured labels can reward models for recognizing location rather than short-term conditions. Objective: We asked whether frozen pre-event Prithvi representations could rank a recorded fire date above recorded no-fire dates within the same coarse geographic cell and year. Methods: Coordinate-only diagnostics first exposed a strong spatial shortcut (ROC-AUC 0.8953), motivating 347 retrospective, label-blind temporal risk sets in Morocco within 10-km cells. Calendar-only, EO-only, and a validation-fixed blend were trained on 2020, selected on 2021, and evaluated once on 23 2022 risk sets from 16 connected events. Results: The blend achieved Top-1 0.5145, MRR 0.7028, and concordance 0.6853. Compared with EO-only, point differences favored the blend across all metrics, but only MRR remained above zero in a 15-component bootstrap (+0.0623, 95% CI 0.0059–0.1369); Top-1 touched zero and concordance was inconclusive. Conclusion: The shared-centroid geography control was at chance, but candidates did not share exact coordinates. These results support preliminary conditional-ranking evidence and a shortcut-aware evaluation workflow, not calibrated wildfire risk, causal precursor detection, or external generalization.

Claim ledger for abstract and poster

ID	Evidence class	Permitted claim	Abstract use	Do not infer
D-1	Robustness / diagnostic	Original row task had a strong geographic shortcut	Required	Coordinate score is a temporal forecast
P-1	Primary held-out	C had the strongest 2022 TEST point metrics	Required	C beat A significantly
R-1	Robustness / repair	Only C–B MRR retained a positive component CI	Required	All C–B metrics were significant
R-2	Robustness / repair	Shared centroid neutralized coarse-cell ranking only	Required limitation	All geography was removed
E-1	Post-hoc exploratory	B2 improved on B0 under historical event bootstrap	Optional	Confirmed architecture superiority
E-2	Post-hoc exploratory	Nested fusion was positive versus A	Optional	It beat CAL or B2
L-1	Limitation	Exact-location confounding remains unresolved	Required	Fine geography explains the result
L-2	Limitation	Conditional ranking is not calibrated population risk	Required	Absolute fire probability
L-3	Limitation	Morocco-only, 2020–2022 evidence	Required	External or prospective generalization

Terms for a multidisciplinary audience

Earth observation (EO): satellite measurements of the Earth’s surface.
HLS: harmonized Landsat and Sentinel-2 imagery.
Prithvi: a pretrained geospatial foundation model used here as a frozen feature encoder.
Risk set: one recorded fire date compared with one or more recorded no-fire dates in the same coarse cell and year.
Held out: not used to train or select the model.
Concordance: fraction of within-set case-control comparisons ranked in the correct order.
MRR: a score that rewards placing the recorded fire date near the top of its set.
Overlap component: a conservative group of risk sets linked by the same event or a reused row; resampling these groups avoids treating linked sets as independent.

Claims that must not appear

“Prithvi predicts wildfire risk.”
“The model provides an early-warning system.”
“Same-cell matching removed all geography.”
“Prithvi detected a purely temporal or causal biophysical precursor.”
“Fusion significantly beat EO on all three primary metrics.”
“Fusion established a gain over calendar model A.”
“C11 or nested fusion beat CAL or B2.”
“Richer spatial-token architectures beat B2.”
“EO improves prediction at 0–15 days.”
“The findings generalize across years, outside Morocco, or prospectively.”
“The scores are calibrated probabilities of wildfire.”
“Production v2 used local pixel-level Fmask.”

Questions a symposium reviewer may ask

Why not present the original high ROC-AUC models?

Coordinates alone nearly matched them, so those scores could not distinguish short-term wildfire information from static geography.

Did the redesign eliminate geography?

It neutralized the shared 10-km-cell centroid as a ranking feature. It did not hold exact coordinates fixed; fine-location confounding remains unresolved.

Why only 23 TEST risk sets?

The chronological 2022 holdout was intentionally frozen and small. The analysis therefore reports component-clustered uncertainty and treats conclusions as preliminary.

Did fusion beat EO?

Its point estimates were higher on all three primary metrics. After conservative clustering, only the MRR difference had a 95% CI wholly above zero.

Did Prithvi add information beyond the calendar baseline?

The primary B–A intervals were inconclusive. Later fusion was positive versus A but not statistically superior to CAL or B2, and those later analyses were post-hoc.

Is this operational forecasting?

No. The study used a retrospective HLS archive and evaluates conditional recorded-date ranking, not deployment latency or calibrated future risk.

What would make the claim stronger?

An exact-point or fixed-chip-center EO comparison, fully nested evaluation on an untouched cohort, and external/prospective testing in another geography.

Future evidence needed

Pure temporal EO attribution: exact-point/fixed-center comparisons or a location-metadata/chip-center ablation.
Confirmatory multimodal value: fully nested base and fusion models on an untouched cohort.
Transportability and calibration: independent geography, prospective sampling, and known control ascertainment/prevalence.

Evidence hierarchy and source map

Use sources in this order when wording claims:

reports/POSTER_SAFE_RESULTS.md
reports/RESEARCH_GAUNTLET_REPAIRED.md
reports/STATISTICAL_INFERENCE_LEDGER.md
the specific append-only repair report
PRIMARY_STUDY_STEP22.md for frozen design and point estimates
RESULTS.md only with its appended errata and claim-status sections

The primary document is intentionally historical: its original risk-set bootstrap wording is superseded by the overlap-component repair.

Source	SHA-256 at review
`reports/POSTER_SAFE_RESULTS.md`	`1163485940747a9d6102bee7a457f29806cb8a1be16b7ea769f110c5aabee53b`
`reports/RESEARCH_GAUNTLET_REPAIRED.md`	`350f6a2a77166fab6a4c2167a84af251e1d8b121234de57f91abce5a23602cd3`
`reports/STATISTICAL_INFERENCE_LEDGER.md`	`16c7c8fb56ddf925d64ec098e43e8d054a76cb0e9d820820b48801e94b1a9e6e`
`reports/step22_event_clustered_reanalysis.md`	`c61e0f716d737ae66e7431608bdea0a14b83cc227fdb2de191d7ee743a6e9b0f`
`reports/fine_scale_geography_audit.md`	`baacac9922f639163b2545ff5c839c0adf7f481533f0fdf9dbf68b1db3b77db0`
`reports/step29_nested_selection_reanalysis.md`	`3bc6c128df831716774b4c514af89a2d6e159969069ea771f10054c98366037e`
`reports/recovery_v2_verification.md`	`c90d6a997e39261ff21a938310f784d2c802ec48060223a0bf2d9a6db4540959`
`PRIMARY_STUDY_STEP22.md`	`19da7b1f3986a131cbcf4cc316f405912796400ccae9622f8175fae246c680d4`

The repaired reports and later exploratory artifacts are preserved in the checksum-bound recovery-v2 package. At review time, they were not all committed to the Git tree, so the Git commit alone is not the complete scientific record.

Final abstract checklist

Bottom line for an abstract

The safest and strongest abstract is not “Prithvi predicts wildfire.” It is:

A spatial-shortcut audit changed the scientific question. After a geography-conditioned temporal redesign, a fixed calendar–Prithvi blend showed preliminary held-out ranking improvements, while conservative dependence and fine-location audits narrowed the claim to one supported reciprocal-rank contrast and an evaluation-design contribution.

FilesExpand file tree

Research_notes.md

Latest commit

History