Skip to content

521: derived-drought-historical-monthly uncertainty q02 #564

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
EnisGerxhalija wants to merge 244 commits into
base: main
Choose a base branch
from
+6,478 −0
Open

521: derived-drought-historical-monthly uncertainty q02 #564

Changes from all commits
Commits
Show all changes
244 commits
Select commit Hold shift + click to select a range
f7f4a49
Playing around with dataset
EnisGerxhalija Oct 31, 2025
91cec6b
Experimenting with dataset
EnisGerxhalija Nov 5, 2025
d615c6e
Function to make SPI
EnisGerxhalija Nov 5, 2025
d6208b7
SPI Conversion
EnisGerxhalija Nov 6, 2025
34fb1a7
Writing helper functions
EnisGerxhalija Nov 7, 2025
7076a29
Comparing SPIs
EnisGerxhalija Nov 10, 2025
dbb90d6
Adjusting for historical zero precipitation
EnisGerxhalija Nov 11, 2025
8babb9b
More historical zero precipitation work
EnisGerxhalija Nov 11, 2025
d94d811
Fixed accumulation period issue
EnisGerxhalija Nov 13, 2025
e44a8df
Finalised reanalysis, onto ensemble
EnisGerxhalija Nov 14, 2025
93c686b
quality check
EnisGerxhalija Nov 17, 2025
4afebe9
Ensemble work
EnisGerxhalija Nov 19, 2025
60544f1
Plotting SPI Uncertainty across Europe
EnisGerxhalija Nov 20, 2025
3bfdaba
Plots of precipitation against spi
EnisGerxhalija Nov 21, 2025
7975dc0
SPI against Precipitation plots, downloaded Ensemble data
EnisGerxhalija Nov 21, 2025
1ad3076
Input data monthly mean
EnisGerxhalija Nov 27, 2025
9ad4c65
Monthly means recalculation
EnisGerxhalija Nov 28, 2025
d1e7ea2
Calculating SPEI index
EnisGerxhalija Dec 8, 2025
7710601
Checked SPEI, and SPI ensemble
EnisGerxhalija Dec 9, 2025
3366150
Calculating SPI Unc
EnisGerxhalija Dec 10, 2025
b9e01f4
Applying ensemble data
EnisGerxhalija Dec 15, 2025
24105bc
More ensemble application
EnisGerxhalija Dec 17, 2025
5328330
General logistic fit
EnisGerxhalija Dec 18, 2025
a60d436
Calculating quality indicators
EnisGerxhalija Dec 19, 2025
ac2e0c0
Adding quality flags
EnisGerxhalija Dec 22, 2025
2d1b15d
Probability ensemble forecasting
EnisGerxhalija Dec 24, 2025
b50644d
Diagnosing WSL issue
EnisGerxhalija Dec 29, 2025
55151c8
Reproducibility notebook
EnisGerxhalija Dec 30, 2025
8114e06
Reviewing reproducibility notebook
EnisGerxhalija Dec 31, 2025
05dc4e7
Remove invalid / duplicate files
burggraaff Dec 31, 2025
af8b5c8
Tweak author section
burggraaff Dec 31, 2025
0454841
Merge branch 'main' into 521_drought
burggraaff Dec 31, 2025
5c92b30
Pushing changes to pull
EnisGerxhalija Jan 2, 2026
1afd73d
Merge branch '521_drought' of github.com:ecmwf-projects/c3s2-eqc-qual…
EnisGerxhalija Jan 2, 2026
c347e55
Adding Oli suggestions
EnisGerxhalija Jan 3, 2026
69a37d7
Using MARS data
EnisGerxhalija Jan 6, 2026
d4b912f
Restore .gitignore
burggraaff Jan 7, 2026
aeb38a0
Add info about using full ERA5 CDS entry
burggraaff Jan 7, 2026
9c91809
Clear outputs
burggraaff Jan 7, 2026
602d9f4
Rework initial ERA5 download to match Atlas/AgERA5 style
burggraaff Jan 7, 2026
f0c1f7f
Minor changes to accumulation example (not function)
burggraaff Jan 7, 2026
e7ae056
Minor refactor of accumulation
burggraaff Jan 7, 2026
3f10ef4
New fitting function based on GroupBy
burggraaff Jan 7, 2026
442fbc3
Update variable names -- SPI calculation works again
burggraaff Jan 7, 2026
af165f9
New computing function for SPI/SPEI
burggraaff Jan 7, 2026
ac010e8
Minor (hopefully not micro) optimisations re chunking/float size
burggraaff Jan 7, 2026
b7e16b9
Optimising functions
EnisGerxhalija Jan 8, 2026
75ceb15
Optimising code for SPEI calculation
EnisGerxhalija Jan 9, 2026
2a9af9b
Separate uncertainty notebook
burggraaff Jan 12, 2026
556977d
Calcualting ensemble members for spi
EnisGerxhalija Jan 12, 2026
52979da
Merge branch '521_drought' of github.com:ecmwf-projects/c3s2-eqc-qual…
EnisGerxhalija Jan 12, 2026
154250f
Minor textual changes
burggraaff Jan 13, 2026
f22b5ae
Remove unused code
burggraaff Jan 13, 2026
4bfe0d7
Refactor creation of number dimension
burggraaff Jan 13, 2026
5fc58e9
Refactor geospatial mean+spread plot
burggraaff Jan 13, 2026
cfe496e
Downloading ensemble data troubleshoot
EnisGerxhalija Jan 14, 2026
8517532
Merge branch '521_drought' of github.com:ecmwf-projects/c3s2-eqc-qual…
EnisGerxhalija Jan 14, 2026
9a6d177
Consistency notebook for review
EnisGerxhalija Jan 15, 2026
5a1e1a3
Feedback
burggraaff Jan 16, 2026
889ed0c
Adding glue reviewing feedback
EnisGerxhalija Jan 20, 2026
b1d08b9
Reviewing feedback
EnisGerxhalija Jan 22, 2026
5be49a7
Downloading data from different regions
EnisGerxhalija Jan 22, 2026
f04042e
Zero precipitation quality check
EnisGerxhalija Jan 25, 2026
1f0362d
Completed quality flag comparison
EnisGerxhalija Jan 26, 2026
5bb560c
Fixed confusion matrix problem
EnisGerxhalija Jan 27, 2026
fce186e
Patching heads, fixing quality plot
EnisGerxhalija Jan 27, 2026
ef07051
Final Enis version
EnisGerxhalija Jan 28, 2026
2f99357
Merge branch 'main' into 521_drought
burggraaff Jan 29, 2026
ee33d0b
Undo run
burggraaff Jan 29, 2026
cc4b8c0
Reorganise helper functions: Data pre-processing
burggraaff Jan 29, 2026
5557d7e
Reorganise helper functions further
burggraaff Jan 29, 2026
07ba58a
Cleanup: General setup
burggraaff Jan 29, 2026
3db281d
Notebook titles
burggraaff Jan 29, 2026
e0cf139
Notebook & section titles
burggraaff Jan 29, 2026
51eab3e
Cleanup: References
burggraaff Jan 29, 2026
c85227a
SPI: Download ERA5 tp
burggraaff Jan 29, 2026
12234e4
SPI: Accumulation periods + Use consistent dimension names
burggraaff Jan 29, 2026
4edcd98
Fit Gamma distribution
burggraaff Feb 2, 2026
df8732d
Compute CDF for SPI
burggraaff Feb 2, 2026
649ccda
Adjust for 0 precipitation
burggraaff Feb 2, 2026
3b8db9d
Calculate SPI
burggraaff Feb 2, 2026
a6ca8d2
Download ERA5-Drought
burggraaff Feb 2, 2026
6070a2f
Handle variable names without hyphens
burggraaff Feb 3, 2026
d0e5c3d
Time series comparison (differs from EG)
burggraaff Feb 3, 2026
2a1950b
Reorganise visualisation subsection
burggraaff Feb 3, 2026
1a2d7f2
Use global ERA5 download in case study - Fixes previous disparity
burggraaff Feb 3, 2026
4e2a826
Merge branch 'main' into 521_drought
burggraaff Feb 3, 2026
138a168
Reorganise SPI comparisons
burggraaff Feb 3, 2026
7345c2e
SPEI: Calculate water balance
burggraaff Feb 3, 2026
a857c3e
Finish reproducing SPEI
burggraaff Feb 4, 2026
a5c0e39
SPEI time series comparison
burggraaff Feb 4, 2026
3207bee
Confusion matrices
burggraaff Feb 4, 2026
d2fe555
Adjust time series comparison plot size and line
burggraaff Feb 4, 2026
9341749
Introduction
burggraaff Feb 4, 2026
9a07bd8
Comparison table
burggraaff Feb 4, 2026
f375b25
Reshuffle comparison cells
burggraaff Feb 5, 2026
7154780
Regional comparison
burggraaff Feb 5, 2026
c0349e8
Load pzero
burggraaff Feb 5, 2026
5244980
Download quality flags
burggraaff Feb 5, 2026
48e1524
Plot flagged months (partial)
burggraaff Feb 6, 2026
f7d727a
Remove redundant functions for cds downloads
burggraaff Feb 6, 2026
5f8031e
Finalise quality flag geospatial plot
burggraaff Feb 6, 2026
f44fcde
Quantitative comparison of p_zero flags
burggraaff Feb 6, 2026
191ab79
Shapiro-Wilk normality test
burggraaff Feb 6, 2026
c14807e
Merge branch 'main' into 521_drought
burggraaff Feb 9, 2026
e797d28
Move reproducibility notebook to q01 (part 1)
burggraaff Feb 9, 2026
a8ce1c7
Move reproducibility notebook to q01 (part 2)
burggraaff Feb 9, 2026
fc16860
Cleanup: Intro / Methodology
burggraaff Feb 9, 2026
8ade68f
Reference for accumulation periods
burggraaff Feb 9, 2026
26b8f56
Consistently use en dashes, following Keune+25
burggraaff Feb 9, 2026
41310a6
Remove unused imports
burggraaff Feb 9, 2026
53bea00
Calculate fraction of absolute difference over threshold
burggraaff Feb 9, 2026
3bccaec
SPI time series conclusions
burggraaff Feb 9, 2026
df14a30
writing ffp
EnisGerxhalija Feb 9, 2026
20c6ac0
deleting latest file
EnisGerxhalija Feb 9, 2026
301142d
SPI analysis - First draft
burggraaff Feb 9, 2026
d1bc645
SPEI comparison
burggraaff Feb 9, 2026
b0a8d1f
Remove bar charts
burggraaff Feb 9, 2026
725ff65
SPEI analysis
burggraaff Feb 9, 2026
9100b12
Update glue labels to new filename
burggraaff Feb 9, 2026
a21d7eb
Setup for ensemble
burggraaff Feb 10, 2026
4972bd3
Uncertainty notebook SPI ensemble
EnisGerxhalija Feb 10, 2026
f91477b
Pull changes from Oli, Merge branch '521_drought' of github.com:ecmwf…
EnisGerxhalija Feb 10, 2026
a91886d
Final Enis draft of ffp.
EnisGerxhalija Feb 10, 2026
59f453e
Enis pushing final changes to repro
EnisGerxhalija Feb 11, 2026
fc35be2
Remove FFP file, now hosted separately
burggraaff Feb 11, 2026
d70c38a
Clear outputs
burggraaff Feb 11, 2026
ab01b89
Load and preprocess ensemble data
burggraaff Feb 11, 2026
56fa276
Full ensemble comparison
burggraaff Feb 11, 2026
4e54fb3
Consistently hyphenate SPI and SPEI
burggraaff Feb 11, 2026
71fd724
Set example_region_size to 12 for final plots
burggraaff Feb 12, 2026
2546a2f
Refactoring uncertainty code
EnisGerxhalija Feb 12, 2026
351d85b
GISCO borders (earthkit-plots 0.6.0)
burggraaff Feb 12, 2026
e012646
Merge branch '521_drought_rep' into 521_drought
burggraaff Feb 12, 2026
4a5060f
Consistent GISCO borders and black coastlines
burggraaff Feb 12, 2026
85a46d7
Adding quality masking
EnisGerxhalija Feb 13, 2026
9b362a5
Pushing quality check changes Merge branch '521_drought' of github.co…
EnisGerxhalija Feb 13, 2026
c33aa7e
Fixed bar gaps
EnisGerxhalija Feb 13, 2026
865c8ba
Adding parametric insurance section
EnisGerxhalija Feb 15, 2026
6166d7b
Creating payout function
EnisGerxhalija Feb 16, 2026
0d1490a
Add payout figure
EnisGerxhalija Feb 16, 2026
a85904f
Refactoring code + writeup
EnisGerxhalija Feb 17, 2026
90ce428
Fixed ensemble request
EnisGerxhalija Feb 17, 2026
a893216
Fixed loading reanalysis data
EnisGerxhalija Feb 18, 2026
f34c052
Function for payout plot
EnisGerxhalija Feb 18, 2026
7e310dc
Payout section for OB review
EnisGerxhalija Feb 18, 2026
897a353
Newest version for OB to review
EnisGerxhalija Feb 18, 2026
28c0397
Payout difference fixed
EnisGerxhalija Feb 18, 2026
e3fcf14
Fixed box plot outlier issue
EnisGerxhalija Feb 18, 2026
f37a178
Outlier colour now green
EnisGerxhalija Feb 18, 2026
7db3daf
Drop in intro text + references
burggraaff Feb 18, 2026
32a205b
Tweak title, use case, questions
burggraaff Feb 18, 2026
e34cbaa
Introduce parametric insurance
burggraaff Feb 18, 2026
31b4ba8
Merge branch '521_drought_unc01' into 521_drought
burggraaff Feb 18, 2026
ab7fff5
Clear outputs
burggraaff Feb 19, 2026
c09989e
Cleanup: Data loading
burggraaff Feb 19, 2026
4426889
Split off uncertainty q02 notebook
burggraaff Feb 19, 2026
eab2b76
Download and preprocess p0 mask
burggraaff Feb 19, 2026
a292e6c
Download all quality flags
burggraaff Feb 19, 2026
76cb967
Remove consistency notebook from uncertainty branch
burggraaff Feb 19, 2026
0da528e
Time series without quality flags
burggraaff Feb 19, 2026
28e68db
Time series with quality flags
burggraaff Feb 19, 2026
5cff063
Indemnity function
burggraaff Feb 19, 2026
968e7d9
Fixing introduction
EnisGerxhalija Feb 20, 2026
90bd9e7
Merge branch '521_drought' of github.com:ecmwf-projects/c3s2-eqc-qual…
EnisGerxhalija Feb 20, 2026
57cb649
Merge branch 'main' into 521_drought_unc01
burggraaff Feb 20, 2026
b4c1d18
Remove defunct file
burggraaff Feb 20, 2026
137e476
Insurance intro
burggraaff Feb 20, 2026
2326bf1
Tweak description of insurance, assign metadata
burggraaff Feb 20, 2026
d9ac615
Fixed code up to ensemble categorisation
EnisGerxhalija Feb 20, 2026
d0c6e69
Now fixed categorising error
EnisGerxhalija Feb 20, 2026
9a69c57
Indemnity plot
burggraaff Feb 20, 2026
187ffa2
Remove unused imports and functions
burggraaff Feb 20, 2026
7715c5b
Update TOC
burggraaff Feb 20, 2026
7484068
Expand key resources
burggraaff Feb 20, 2026
1de2e76
Merge branch '521_drought_unc01' into 521_drought
burggraaff Feb 20, 2026
9548213
Revert "Fixing introduction"
burggraaff Feb 20, 2026
ef358c2
Update introduction (q01) with some copypaste (q02)
burggraaff Feb 21, 2026
c731fcf
Methodology
burggraaff Feb 21, 2026
cd8cf91
Minor code cleanup
burggraaff Feb 21, 2026
d49565a
Copy over key resources
burggraaff Feb 21, 2026
009d7e0
Reorder references
burggraaff Feb 21, 2026
e4ac77f
Restore gridlines by default
burggraaff Feb 21, 2026
ba88196
Minor textual tweaks
burggraaff Feb 23, 2026
d557788
Added reanalysis data
EnisGerxhalija Feb 23, 2026
a631394
Added reanalysis download
EnisGerxhalija Feb 23, 2026
6e7da05
Fixed erroneous reanalysis flag request
EnisGerxhalija Feb 23, 2026
33540b9
Categorising drought by recorded year
EnisGerxhalija Feb 23, 2026
e4fc769
Adding category probability code
EnisGerxhalija Feb 23, 2026
81abb79
Code for OB review
EnisGerxhalija Feb 23, 2026
b4c5024
Refactor getting states from Natural Earth
burggraaff Feb 23, 2026
0ef3a9c
Tweak quality assessment questions
burggraaff Feb 23, 2026
5b7ed45
Setup: Pick country and find drought events
burggraaff Feb 23, 2026
0c23d8f
Download SPI and SPEI for region + dates
burggraaff Feb 24, 2026
3b6cbe6
Download quality flags
burggraaff Feb 24, 2026
321984b
Restructure
burggraaff Feb 24, 2026
f1d2121
Display SPI / SPEI
burggraaff Feb 24, 2026
8dfa4c3
Reduce ensemble mask along number dimension
burggraaff Feb 24, 2026
84e1adb
Remove q01 notebook from q02 branch
burggraaff Feb 24, 2026
563f9db
Undo accidental changes to .gitignore
burggraaff Feb 24, 2026
e855b76
Merge branch 'main' into 521_drought
burggraaff Feb 24, 2026
922b41f
Reanalysis and Ensemble Uncertainty plots
EnisGerxhalija Feb 24, 2026
d35a83b
Fixed probability style
EnisGerxhalija Feb 24, 2026
0376f12
Fixed plot styling
EnisGerxhalija Feb 24, 2026
840c168
Clearing figures
EnisGerxhalija Feb 24, 2026
37a5afd
Change EM-DAT filename again
burggraaff Feb 24, 2026
09e7c22
Added threshold statistics
EnisGerxhalija Feb 24, 2026
bdaf60d
Plot SPI and SPEI
burggraaff Feb 24, 2026
cfa867b
Wipe outputs
burggraaff Feb 24, 2026
4238487
Merge branch '521_drought' into 521_drought_unc02
burggraaff Feb 24, 2026
25ea48a
Cleanup: Unused functions
burggraaff Feb 24, 2026
d56bb46
Cleanup: Unused imports
burggraaff Feb 24, 2026
f2d4210
Let earthkit handle state borders (but off by default)
burggraaff Feb 24, 2026
2bc5c4c
Plot SPI / SPEI categorisation
burggraaff Feb 24, 2026
345f8c2
Quantify reanalysis vs ensemble match/mismatch
burggraaff Feb 25, 2026
212341a
Introduction
burggraaff Feb 25, 2026
3f4b819
Methodology
burggraaff Feb 25, 2026
49324e8
Analysis: SPI and SPEI
burggraaff Feb 25, 2026
02f42a8
Finish text
burggraaff Feb 25, 2026
15c7f31
Figures and tables
burggraaff Feb 25, 2026
861e175
Remove progress bars
burggraaff Feb 25, 2026
cdf9b36
Tweak quality assessment statement
burggraaff Feb 25, 2026
418c20f
Tweak text, table caption, text in figures
burggraaff Feb 25, 2026
ecb9441
Fix Steinemann+06 link
burggraaff Feb 25, 2026
28ef846
Change match/mismatch table to a plot
burggraaff Feb 25, 2026
383e119
New version for internal review, with match/mismatch plot
burggraaff Feb 25, 2026
514ec2b
Finish internal review - Back to tables
burggraaff Feb 26, 2026
00f48eb
Remove stderr outputs
burggraaff Feb 26, 2026
f4ee7e2
Merge branch 'main' into 521_drought_unc02_rev
burggraaff May 29, 2026
411324b
Update intro: ECMWF now provide...
burggraaff May 29, 2026
f118050
Alphabetise references
burggraaff May 29, 2026
84c8416
Fix earthkit 404 (not found in the system)
burggraaff Jun 4, 2026
b9deefe
. after date
burggraaff Jun 4, 2026
698af44
Merge branch 'main' into 521_drought_unc02_rev
burggraaff Jun 4, 2026
d581625
Extreme precipitation -> Wet periods
burggraaff Jun 4, 2026
ea1bc1c
Clear outputs
burggraaff Jun 4, 2026
7cbc877
Update number -> realization dimension (untested)
burggraaff Jun 4, 2026
938f13d
Adapt to updated ekp Styles; temp fix
burggraaff Jun 4, 2026
6289342
Adjust to ensemble counting from 1, not 0
burggraaff Jun 4, 2026
b7f1817
Revert "Adapt to updated ekp Styles; temp fix"
burggraaff Jun 12, 2026
dddbcbe
Remove quality flag sentence
burggraaff Jun 12, 2026
1a916c8
Rephrase description of quality flags
burggraaff Jun 12, 2026
669651a
Execute notebook
burggraaff Jun 12, 2026
73ccd5b
Merge branch 'main' into 521_drought_unc02
burggraaff Jun 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Loading