Skip to content

Commit 1b866a4

Browse files
committed
training fix, docs overhaul
1 parent 48a9c69 commit 1b866a4

19 files changed

Lines changed: 105 additions & 98 deletions

.github/workflows/docs.yml

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ on:
44
branches:
55
- main
66
paths:
7-
- "paper/**"
7+
- "docs/**"
88
workflow_dispatch: # allow manual runs
99

1010
permissions:
@@ -31,7 +31,6 @@ jobs:
3131
pip install --upgrade pip
3232
pip install mkdocs-material mkdocstrings[python] pymdown-extensions
3333
34-
# Make your package importable for mkdocstrings (ok if no setup.py/pyproject)
3534
- name: Make repo importable
3635
run: |
3736
pip install -e . || true

docs/architecture.md

Lines changed: 6 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,10 @@
11
# Architecture
22

3-
This document outlines how ESA OpenSR organises its super-resolution GAN, the major components that make up the model, and how
4-
each piece interacts during training and inference.
3+
This document outlines how ESA OpenSR organises its super-resolution GAN, the major components that make up the model, and how each piece interacts during training and inference.
54

65
## SRGAN Lightning module
76

8-
`opensr_srgan/model/SRGAN.py` defines `SRGAN_model`, a `pytorch_lightning.LightningModule` that encapsulates the full adversarial workflow.
9-
The module is initialised from a YAML configuration file and provides the following responsibilities:
7+
`opensr_srgan/model/SRGAN.py` defines `SRGAN_model`, a `pytorch_lightning.LightningModule` that encapsulates the full adversarial workflow. The module is initialised from a YAML configuration file and provides the following responsibilities:
108

119
* **Configuration ingestion.** Uses OmegaConf to load hyperparameters, dataset choices, and logging options. Convenience helpers
1210
such as `_pretrain_check()` and `_compute_adv_loss_weight()` translate config values into runtime behaviour.
@@ -47,8 +45,7 @@ The generator zoo lives under `opensr_srgan/model/generators/` and can be select
4745
* **Advanced variants (`SRGAN_advanced.py`).** Provides additional block implementations and compatibility aliases exposed in
4846
`__init__.py` for backwards compatibility.
4947

50-
Common traits across generators include configurable input channel counts (`Model.in_bands`), support for upscaling factors from
51-
2× to 8×, and residual scaling to stabilise deeper networks.
48+
Common traits across generators include configurable input channel counts (`Model.in_bands`), support for upscaling factors from 2× to 8×, and residual scaling to stabilise deeper networks.
5249

5350
## Discriminator options
5451

@@ -59,13 +56,11 @@ Common traits across generators include configurable input channel counts (`Mode
5956
* **PatchGAN discriminator (`patchgan.py`).** Operates on local patches, which can improve high-frequency fidelity when training
6057
with large images. The depth is controlled by `n_blocks` and defaults to three layers.
6158

62-
Both discriminators use LeakyReLU activations and strided convolutions to progressively downsample the input until a real/fake
63-
logit map is produced.
59+
Both discriminators use LeakyReLU activations and strided convolutions to progressively downsample the input until a real/fake logit map is produced.
6460

6561
## Loss suite and metrics
6662

67-
`opensr_srgan/model/loss` contains the perceptual and pixel-based criteria applied to the generator outputs. The primary entry point is
68-
`GeneratorContentLoss`, which supports:
63+
`opensr_srgan/model/loss` contains the perceptual and pixel-based criteria applied to the generator outputs. The primary entry point is `GeneratorContentLoss`, which supports:
6964

7065
* **L1 reconstruction** over all spectral bands.
7166
* **Spectral Angle Mapper (SAM)** to preserve spectral signatures.
@@ -95,5 +90,4 @@ These helpers allow the generator to operate in a normalised space while still r
9590
5. When exported, `predict_step()` can be called directly or wrapped in a Lightning `Trainer.predict()` loop for large-scale
9691
inference.
9792

98-
This modular design keeps the research workflow flexible: swap components with configuration changes, extend the factories with
99-
new architectures, or plug in custom losses without touching the training loop itself.
93+
This modular design keeps the research workflow flexible: swap components with configuration changes, extend the factories with new architectures, or plug in custom losses without touching the training loop itself.

docs/assets/adv_loss_warmup.png

20.6 KB
Loading

docs/assets/discr_x_prob.png

19.7 KB
Loading

docs/assets/discr_y_prob.png

23.3 KB
Loading
20.2 KB
Loading

docs/assets/lr_scheduler.png

17.6 KB
Loading

docs/assets/pretrain_phase.png

16.1 KB
Loading

docs/configuration.md

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
# Configuration
22

3-
ESA OpenSR relies on YAML files to control every aspect of the training pipeline. This page documents the available keys and how
4-
they influence the underlying code. Use `opensr_srgan/configs/config_20m.yaml` and `opensr_srgan/configs/config_10m.yaml` as starting points.
3+
ESA OpenSR relies on YAML files to control every aspect of the training pipeline. This page documents the available keys and how they influence the underlying code. Use `opensr_srgan/configs/config_20m.yaml` and `opensr_srgan/configs/config_10m.yaml` as starting points.
54

65
## File structure
76

@@ -119,16 +118,14 @@ performing sweeps:
119118

120119
### Discriminator presets
121120

122-
Tune discriminator depth to match the generator capacity—too shallow and adversarial loss underfits, too deep and the
123-
training loop destabilises. These starting points mirror the architectures bundled with the repo:
121+
Tune discriminator depth to match the generator capacity—too shallow and adversarial loss underfits, too deep and the training loop destabilises. These starting points mirror the architectures bundled with the repo:
124122

125123
| Discriminator type | Recommended depth parameter | Additional notes |
126124
| --- | --- | --- |
127125
| `standard` | `n_blocks = 8` | Mirrors the original SRGAN CNN with alternating stride-1/stride-2 blocks before the dense head.】 |
128126
| `patchgan` | `n_blocks = 3` | Maps to the 3-layer PatchGAN (a.k.a. `n_layers`); increase to 4–5 for larger crops or when the generator is particularly sharp. |
129127

130-
When adjusting these presets, scale generator and discriminator together and monitor adversarial loss ramps defined in
131-
`Training.Losses` to keep training stable.
128+
When adjusting these presets, scale generator and discriminator together and monitor adversarial loss ramps defined in `Training.Losses` to keep training stable.
132129

133130
## Optimisers
134131

@@ -170,5 +167,4 @@ available for experiments that prefer a steady rise.
170167
* **Override selectively.** When launching through scripts or notebooks, you can load a base config and override specific fields at
171168
runtime using `OmegaConf.merge`.
172169

173-
With a clear understanding of these fields, you can rapidly iterate on architectures, datasets, and training strategies without
174-
modifying the underlying code.
170+
With a clear understanding of these fields, you can rapidly iterate on architectures, datasets, and training strategies without modifying the underlying code.

docs/data.md

Lines changed: 10 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,37 +1,31 @@
11
# Data
22

3-
The training stack ships with a single, self-contained example dataset that you can download in seconds to verify that the
4-
pipeline works end to end. This page explains how to fetch the sample data, how it is structured, and what you need to do when
3+
The training stack ships with a single, self-contained example dataset that you can download in seconds to verify that the pipeline works end to end. This page explains how to fetch the sample data, how it is structured, and what you need to do when
54
you are ready to plug in your own collections.
65

76
## Example dataset
87

9-
The example dataset is a small Sentinel-2 crop bundle hosted on the Hugging Face Hub. Each `.npz` archive contains a
10-
high-resolution chip stored under the key `hr`. Low-resolution counterparts are generated on the fly by bicubic
11-
interpolation inside the dataset class.
8+
The example dataset is a small Sentinel-2 crop bundle hosted on the Hugging Face Hub. Each `.npz` archive contains a high-resolution chip stored under the key `hr`. Low-resolution counterparts are generated on the fly by bicubic interpolation inside the dataset class.
129

1310
* **Scale factor:** 4× upsampling between the generated LR inputs and provided HR targets.
1411
* **Splits:** All files except the final 20 samples are used for training; the last 20 form the validation split.
1512
* **Normalisation:** Values above 1.5 are assumed to be Sentinel-2 reflectance and are normalised by `1/10000`.
1613

1714
### Downloading the files
1815

19-
`opensr_srgan/data/example_data/download_example_dataset.py` exposes a helper that downloads and extracts the archive
20-
into `example_dataset/` relative to your working directory. Run it from a Python shell or a small script:
16+
`opensr_srgan/data/example_data/download_example_dataset.py` exposes a helper that downloads and extracts the archive into `example_dataset/` relative to your working directory. Run it from a Python shell or a small script:
2117

2218
```python
2319
from opensr_srgan.data.example_data.download_example_dataset import get_example_dataset
2420

2521
get_example_dataset()
2622
```
2723

28-
The helper pulls `example_dataset.zip` from the [`simon-donike/SR-GAN`](https://huggingface.co/simon-donike/SR-GAN)
29-
repository, extracts it, and removes the temporary archive once the copy completes.
24+
The helper pulls `example_dataset.zip` from the [`simon-donike/SR-GAN`](https://huggingface.co/simon-donike/SR-GAN) repository, extracts it, and removes the temporary archive once the copy completes.
3025

3126
### Directory layout
3227

33-
After extraction the folder contains `.npz` chips named `hr_*.npz`. No further preparation is required. Simply point the
34-
configuration to the dataset by setting:
28+
After extraction the folder contains `.npz` chips named `hr_*.npz`. No further preparation is required. Simply point the configuration to the dataset by setting:
3529

3630
```yaml
3731
Data:
@@ -43,16 +37,11 @@ training and validation dataloaders.
4337

4438
## Adding new datasets
4539

46-
When you are ready to move beyond the bundled sample data you can register a custom dataset. The repository uses a single
47-
factory function—`opensr_srgan.data.dataset_selector.select_dataset`—to keep the training script agnostic of individual
40+
When you are ready to move beyond the bundled sample data you can register a custom dataset. The repository uses a single factory function `opensr_srgan.data.dataset_selector.select_dataset` to keep the training script agnostic of individual
4841
collections. To integrate a new source:
4942

50-
1. **Implement a dataset class.** Create a `torch.utils.data.Dataset` that returns `(lr, hr)` tensors and performs any
51-
normalisation your sensor requires. Place the implementation somewhere under `opensr_srgan/data/`.
52-
2. **Update the selector.** Add a new `elif` branch to `select_dataset` that imports your dataset class, instantiates the
53-
training and validation splits, and returns them.
54-
3. **Expose configuration hooks.** Introduce a new `Data.dataset_type` key (for example `MySensor`) and any additional fields
55-
you need (paths, augmentation flags, scale factors, …). Document the required entries in your configuration file.
43+
1. **Implement a dataset class.** Create a `torch.utils.data.Dataset` that returns `(lr, hr)` tensors and performs any normalisation your sensor requires. Place the implementation somewhere under `opensr_srgan/data/`.
44+
2. **Update the selector.** Add a new `elif` branch to `select_dataset` that imports your dataset class, instantiates the training and validation splits, and returns them.
45+
3. **Expose configuration hooks.** Introduce a new `Data.dataset_type` key (for example `MyDataset`) and any additional fields you need (paths, augmentation flags, scale factors, …). Document the required entries in your configuration file.
5646

57-
Following this pattern keeps `opensr_srgan.train` untouched: once the selector knows about your dataset you can launch
58-
training through the CLI or the Python API without further changes.
47+
Following this pattern keeps `opensr_srgan.train` untouched: once the selector knows about your dataset you can launch training through the CLI or the Python API without further changes.

0 commit comments

Comments
 (0)