You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/architecture.md
+6-12Lines changed: 6 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,12 +1,10 @@
1
1
# Architecture
2
2
3
-
This document outlines how ESA OpenSR organises its super-resolution GAN, the major components that make up the model, and how
4
-
each piece interacts during training and inference.
3
+
This document outlines how ESA OpenSR organises its super-resolution GAN, the major components that make up the model, and how each piece interacts during training and inference.
5
4
6
5
## SRGAN Lightning module
7
6
8
-
`opensr_srgan/model/SRGAN.py` defines `SRGAN_model`, a `pytorch_lightning.LightningModule` that encapsulates the full adversarial workflow.
9
-
The module is initialised from a YAML configuration file and provides the following responsibilities:
7
+
`opensr_srgan/model/SRGAN.py` defines `SRGAN_model`, a `pytorch_lightning.LightningModule` that encapsulates the full adversarial workflow. The module is initialised from a YAML configuration file and provides the following responsibilities:
10
8
11
9
***Configuration ingestion.** Uses OmegaConf to load hyperparameters, dataset choices, and logging options. Convenience helpers
12
10
such as `_pretrain_check()` and `_compute_adv_loss_weight()` translate config values into runtime behaviour.
@@ -47,8 +45,7 @@ The generator zoo lives under `opensr_srgan/model/generators/` and can be select
47
45
***Advanced variants (`SRGAN_advanced.py`).** Provides additional block implementations and compatibility aliases exposed in
48
46
`__init__.py` for backwards compatibility.
49
47
50
-
Common traits across generators include configurable input channel counts (`Model.in_bands`), support for upscaling factors from
51
-
2× to 8×, and residual scaling to stabilise deeper networks.
48
+
Common traits across generators include configurable input channel counts (`Model.in_bands`), support for upscaling factors from 2× to 8×, and residual scaling to stabilise deeper networks.
52
49
53
50
## Discriminator options
54
51
@@ -59,13 +56,11 @@ Common traits across generators include configurable input channel counts (`Mode
59
56
***PatchGAN discriminator (`patchgan.py`).** Operates on local patches, which can improve high-frequency fidelity when training
60
57
with large images. The depth is controlled by `n_blocks` and defaults to three layers.
61
58
62
-
Both discriminators use LeakyReLU activations and strided convolutions to progressively downsample the input until a real/fake
63
-
logit map is produced.
59
+
Both discriminators use LeakyReLU activations and strided convolutions to progressively downsample the input until a real/fake logit map is produced.
64
60
65
61
## Loss suite and metrics
66
62
67
-
`opensr_srgan/model/loss` contains the perceptual and pixel-based criteria applied to the generator outputs. The primary entry point is
68
-
`GeneratorContentLoss`, which supports:
63
+
`opensr_srgan/model/loss` contains the perceptual and pixel-based criteria applied to the generator outputs. The primary entry point is `GeneratorContentLoss`, which supports:
69
64
70
65
***L1 reconstruction** over all spectral bands.
71
66
***Spectral Angle Mapper (SAM)** to preserve spectral signatures.
@@ -95,5 +90,4 @@ These helpers allow the generator to operate in a normalised space while still r
95
90
5. When exported, `predict_step()` can be called directly or wrapped in a Lightning `Trainer.predict()` loop for large-scale
96
91
inference.
97
92
98
-
This modular design keeps the research workflow flexible: swap components with configuration changes, extend the factories with
99
-
new architectures, or plug in custom losses without touching the training loop itself.
93
+
This modular design keeps the research workflow flexible: swap components with configuration changes, extend the factories with new architectures, or plug in custom losses without touching the training loop itself.
Copy file name to clipboardExpand all lines: docs/configuration.md
+4-8Lines changed: 4 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,6 @@
1
1
# Configuration
2
2
3
-
ESA OpenSR relies on YAML files to control every aspect of the training pipeline. This page documents the available keys and how
4
-
they influence the underlying code. Use `opensr_srgan/configs/config_20m.yaml` and `opensr_srgan/configs/config_10m.yaml` as starting points.
3
+
ESA OpenSR relies on YAML files to control every aspect of the training pipeline. This page documents the available keys and how they influence the underlying code. Use `opensr_srgan/configs/config_20m.yaml` and `opensr_srgan/configs/config_10m.yaml` as starting points.
5
4
6
5
## File structure
7
6
@@ -119,16 +118,14 @@ performing sweeps:
119
118
120
119
### Discriminator presets
121
120
122
-
Tune discriminator depth to match the generator capacity—too shallow and adversarial loss underfits, too deep and the
123
-
training loop destabilises. These starting points mirror the architectures bundled with the repo:
121
+
Tune discriminator depth to match the generator capacity—too shallow and adversarial loss underfits, too deep and the training loop destabilises. These starting points mirror the architectures bundled with the repo:
|`standard`|`n_blocks = 8`| Mirrors the original SRGAN CNN with alternating stride-1/stride-2 blocks before the dense head.】 |
128
126
|`patchgan`|`n_blocks = 3`| Maps to the 3-layer PatchGAN (a.k.a. `n_layers`); increase to 4–5 for larger crops or when the generator is particularly sharp. |
129
127
130
-
When adjusting these presets, scale generator and discriminator together and monitor adversarial loss ramps defined in
131
-
`Training.Losses` to keep training stable.
128
+
When adjusting these presets, scale generator and discriminator together and monitor adversarial loss ramps defined in `Training.Losses` to keep training stable.
132
129
133
130
## Optimisers
134
131
@@ -170,5 +167,4 @@ available for experiments that prefer a steady rise.
170
167
***Override selectively.** When launching through scripts or notebooks, you can load a base config and override specific fields at
171
168
runtime using `OmegaConf.merge`.
172
169
173
-
With a clear understanding of these fields, you can rapidly iterate on architectures, datasets, and training strategies without
174
-
modifying the underlying code.
170
+
With a clear understanding of these fields, you can rapidly iterate on architectures, datasets, and training strategies without modifying the underlying code.
The training stack ships with a single, self-contained example dataset that you can download in seconds to verify that the
4
-
pipeline works end to end. This page explains how to fetch the sample data, how it is structured, and what you need to do when
3
+
The training stack ships with a single, self-contained example dataset that you can download in seconds to verify that the pipeline works end to end. This page explains how to fetch the sample data, how it is structured, and what you need to do when
5
4
you are ready to plug in your own collections.
6
5
7
6
## Example dataset
8
7
9
-
The example dataset is a small Sentinel-2 crop bundle hosted on the Hugging Face Hub. Each `.npz` archive contains a
10
-
high-resolution chip stored under the key `hr`. Low-resolution counterparts are generated on the fly by bicubic
11
-
interpolation inside the dataset class.
8
+
The example dataset is a small Sentinel-2 crop bundle hosted on the Hugging Face Hub. Each `.npz` archive contains a high-resolution chip stored under the key `hr`. Low-resolution counterparts are generated on the fly by bicubic interpolation inside the dataset class.
12
9
13
10
***Scale factor:** 4× upsampling between the generated LR inputs and provided HR targets.
14
11
***Splits:** All files except the final 20 samples are used for training; the last 20 form the validation split.
15
12
***Normalisation:** Values above 1.5 are assumed to be Sentinel-2 reflectance and are normalised by `1/10000`.
16
13
17
14
### Downloading the files
18
15
19
-
`opensr_srgan/data/example_data/download_example_dataset.py` exposes a helper that downloads and extracts the archive
20
-
into `example_dataset/` relative to your working directory. Run it from a Python shell or a small script:
16
+
`opensr_srgan/data/example_data/download_example_dataset.py` exposes a helper that downloads and extracts the archive into `example_dataset/` relative to your working directory. Run it from a Python shell or a small script:
21
17
22
18
```python
23
19
from opensr_srgan.data.example_data.download_example_dataset import get_example_dataset
24
20
25
21
get_example_dataset()
26
22
```
27
23
28
-
The helper pulls `example_dataset.zip` from the [`simon-donike/SR-GAN`](https://huggingface.co/simon-donike/SR-GAN)
29
-
repository, extracts it, and removes the temporary archive once the copy completes.
24
+
The helper pulls `example_dataset.zip` from the [`simon-donike/SR-GAN`](https://huggingface.co/simon-donike/SR-GAN) repository, extracts it, and removes the temporary archive once the copy completes.
30
25
31
26
### Directory layout
32
27
33
-
After extraction the folder contains `.npz` chips named `hr_*.npz`. No further preparation is required. Simply point the
34
-
configuration to the dataset by setting:
28
+
After extraction the folder contains `.npz` chips named `hr_*.npz`. No further preparation is required. Simply point the configuration to the dataset by setting:
35
29
36
30
```yaml
37
31
Data:
@@ -43,16 +37,11 @@ training and validation dataloaders.
43
37
44
38
## Adding new datasets
45
39
46
-
When you are ready to move beyond the bundled sample data you can register a custom dataset. The repository uses a single
47
-
factory function—`opensr_srgan.data.dataset_selector.select_dataset`—to keep the training script agnostic of individual
40
+
When you are ready to move beyond the bundled sample data you can register a custom dataset. The repository uses a single factory function `opensr_srgan.data.dataset_selector.select_dataset` to keep the training script agnostic of individual
48
41
collections. To integrate a new source:
49
42
50
-
1. **Implement a dataset class.** Create a `torch.utils.data.Dataset` that returns `(lr, hr)` tensors and performs any
51
-
normalisation your sensor requires. Place the implementation somewhere under `opensr_srgan/data/`.
52
-
2. **Update the selector.** Add a new `elif` branch to `select_dataset` that imports your dataset class, instantiates the
53
-
training and validation splits, and returns them.
54
-
3. **Expose configuration hooks.** Introduce a new `Data.dataset_type` key (for example `MySensor`) and any additional fields
55
-
you need (paths, augmentation flags, scale factors, …). Document the required entries in your configuration file.
43
+
1. **Implement a dataset class.** Create a `torch.utils.data.Dataset` that returns `(lr, hr)` tensors and performs any normalisation your sensor requires. Place the implementation somewhere under `opensr_srgan/data/`.
44
+
2. **Update the selector.** Add a new `elif` branch to `select_dataset` that imports your dataset class, instantiates the training and validation splits, and returns them.
45
+
3. **Expose configuration hooks.** Introduce a new `Data.dataset_type` key (for example `MyDataset`) and any additional fields you need (paths, augmentation flags, scale factors, …). Document the required entries in your configuration file.
56
46
57
-
Following this pattern keeps `opensr_srgan.train` untouched: once the selector knows about your dataset you can launch
58
-
training through the CLI or the Python API without further changes.
47
+
Following this pattern keeps `opensr_srgan.train` untouched: once the selector knows about your dataset you can launch training through the CLI or the Python API without further changes.
0 commit comments