data: add India agriculture raster and vector layer configs

- Add india_worldcover (ESA WorldCover, resampled 10x) and india_soc_change
  (SOC prediction change 2018-2023) to raster_config.yaml
- Add WorldCover resampling preprocessing cell to 01_raster_layers.ipynb
- Add resample_factor option to WorldCereal workflow in 04_layers_step1.ipynb
  for large countries that exceed Mapbox 500KB tile limit
- Document data directory structure in LAYER_PROCESSING_GUIDELINES.md

Author: SofiaAldabet

data-processing/LAYER_PROCESSING_GUIDELINES.md

@@ -49,6 +49,20 @@ uv sync
 source .venv/bin/activate
 ```
 
+### Data directory
+
+Raw inputs and processed outputs live in `data-processing/data/`, which is gitignored. Create it manually on first clone:
+
+```
+data-processing/data/
+├── raw/          # Original files from data providers, organised by theme/country
+│   └── {Theme}/{Country}/...
+└── processed/    # Pipeline outputs (COG, MBTiles, APNGs), organised the same way
+    └── {Theme}/{Country}/{Rasters|Vectors|APNGs}/...
+```
+
+All config paths (`base_path`, `input_file`, `output_file`, `vector_file`) are relative to `data-processing/src/`, so they use `../data/raw/...` and `../data/processed/...`.
+
 ### Environment variables
 
 Copy `.env.example` to `.env` and fill in credentials:

data-processing/notebooks/01_raster_layers.ipynb

@@ -18,7 +18,7 @@
     "\n",
     "The source of truth for all raster layer definitions is `raster_config.yaml`. Most layers run straight from config with no manual steps required and are not documented here.\n",
     "\n",
-    "This notebook documents only layers that needed **custom preprocessing** before running the standard processor \u2014 merging tiles, resampling, clipping to boundaries, assigning a missing CRS, etc. These steps are kept for reproducibility and reference.\n",
+    "This notebook documents only layers that needed **custom preprocessing** before running the standard processor — merging tiles, resampling, clipping to boundaries, assigning a missing CRS, etc. These steps are kept for reproducibility and reference.\n",
     "\n",
     "To process layers that require no preprocessing, use the **working cell** at the bottom of this notebook."
    ],
@@ -298,7 +298,7 @@
     "        output_file = input_file.parent / f\"0007MZ_GIL_L4_AGB_{suffix}.tiff\"\n",
     "        with rio.open(output_file, \"w\", **meta) as dst:\n",
     "            dst.write(src.read(i), 1)\n",
-    "        print(f\"{suffix} ({description}) \u2192 {output_file}\")"
+    "        print(f\"{suffix} ({description}) → {output_file}\")"
    ]
   },
   {
@@ -319,7 +319,7 @@
     "    input_file = raw_dir / f\"0007MZ_GIL_L4_AGB_{suffix}.tiff\"\n",
     "    output_file = raw_dir / f\"0007MZ_GIL_L4_AGB_{suffix}_resampled.tiff\"\n",
     "    resample_raster(input_file, output_file, scale_factor=3, resampling_method=Resampling.average)\n",
-    "    print(f\"{suffix} resampled \u2192 {output_file}\")"
+    "    print(f\"{suffix} resampled → {output_file}\")"
    ]
   },
   {
@@ -337,6 +337,18 @@
     ")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "source": "#### India - Agriculture (WorldCover)",
+   "metadata": {}
+  },
+  {
+   "cell_type": "code",
+   "source": "# Resample WorldCover tile from 10m to ~100m (36000x36000 → 3600x3600).\n# 5x was still too dense — styled RGBA tiles exceed Mapbox 500KB limit at z8.\n# Categorical data — use nearest resampling to preserve land cover classes.\nfrom rasterio.enums import Resampling\n\ninput_file = Path(\n    \"../data/raw/Agriculture/India/GDA-AGRI_UC1-India/ESA_WORLDCOVER/\"\n    \"ESA_WorldCover_10m_2021_v200_N30E075_Map/ESA_WorldCover_10m_2021_v200_N30E075_Map.tif\"\n)\noutput_file = Path(\n    \"../data/raw/Agriculture/India/GDA-AGRI_UC1-India/ESA_WORLDCOVER/\"\n    \"ESA_WorldCover_10m_2021_v200_N30E075_Map/ESA_WorldCover_10m_2021_v200_N30E075_Map_resampled.tif\"\n)\n\nresample_raster(input_file, output_file, scale_factor=10, resampling_method=Resampling.nearest)\nprint(f\"Resampled → {output_file}\")",
+   "metadata": {},
+   "execution_count": null,
+   "outputs": []
+  },
   {
    "cell_type": "markdown",
    "id": "new-section-header",
@@ -350,7 +362,7 @@
     "1. Add the layer definition to `raster_config.yaml`\n",
     "2. Edit `layer_keys` below and run\n",
     "3. If preprocessing was needed, move those cells to the **Preprocessing records** section above with a markdown header explaining what was done\n",
-    "4. Revert this cell before committing \u2014 do not accumulate processed layer keys here"
+    "4. Revert this cell before committing — do not accumulate processed layer keys here"
    ],
    "outputs": [],
    "execution_count": null
@@ -361,7 +373,7 @@
    "id": "new-working-cell",
    "metadata": {},
    "outputs": [],
-   "source": "# WORKING CELL \u2014 edit layer_keys, run, then revert. Do not commit with real values.\nprocess_raster_layers(\n    config_path=config_path,\n    layer_keys=[\"your_layer_key_here\"],\n    upload_layer_keys=[],\n)"
+   "source": "# WORKING CELL — edit layer_keys, run, then revert. Do not commit with real values.\nprocess_raster_layers(\n    config_path=config_path,\n    layer_keys=[\"your_layer_key_here\"],\n    upload_layer_keys=[],\n)"
   }
  ],
  "metadata": {

data-processing/notebooks/04_layers_step1.ipynb

@@ -4,7 +4,25 @@
    "cell_type": "markdown",
    "id": "ec48b40a",
    "metadata": {},
-   "source": "# Extract layers for step 1\n\nEvery story in the Impact Sphere includes a Step 1 layer — a national or regional overview that provides geographic context for the story. This layer always covers the full country or region, even when the story itself focuses on a specific city or area within it (e.g., a story about Nairobi still uses a Kenya-wide Step 1 layer; a story about West Africa uses a regional mosaic).\n\nThis notebook extracts Step 1 layers from one of four global datasets:\n- **WorldCover** — ESA global land cover map (10 m)\n- **WSF Building Area** — DLR World Settlement Footprint 3D building area\n- **Global Surface Water (GSW)** — JRC surface water occurrence\n- **WorldCereal** — ESA global cropland classification\n\nMore sources may be added in the future.\n\n## Workflows\n\nEach data source section provides two workflows:\n- **Single country** — clip or download the dataset for one country, then style and process\n- **Multi-country region** — clip or download per country (each clipped to its GADM boundary), merge into a regional mosaic, then style and process"
+   "source": [
+    "# Extract layers for step 1\n",
+    "\n",
+    "Every story in the Impact Sphere includes a Step 1 layer — a national or regional overview that provides geographic context for the story. This layer always covers the full country or region, even when the story itself focuses on a specific city or area within it (e.g., a story about Nairobi still uses a Kenya-wide Step 1 layer; a story about West Africa uses a regional mosaic).\n",
+    "\n",
+    "This notebook extracts Step 1 layers from one of four global datasets:\n",
+    "- **WorldCover** — ESA global land cover map (10 m)\n",
+    "- **WSF Building Area** — DLR World Settlement Footprint 3D building area\n",
+    "- **Global Surface Water (GSW)** — JRC surface water occurrence\n",
+    "- **WorldCereal** — ESA global cropland classification\n",
+    "\n",
+    "More sources may be added in the future.\n",
+    "\n",
+    "## Workflows\n",
+    "\n",
+    "Each data source section provides two workflows:\n",
+    "- **Single country** — clip or download the dataset for one country, then style and process\n",
+    "- **Multi-country region** — clip or download per country (each clipped to its GADM boundary), merge into a regional mosaic, then style and process"
+   ]
   },
   {
    "cell_type": "markdown",
@@ -22,7 +40,22 @@
    "id": "a25ef8df",
    "metadata": {},
    "outputs": [],
-   "source": "import os\nimport sys\nfrom glob import glob\nfrom pathlib import Path\n\nimport rasterio as rio\nfrom rasterio.merge import merge\n\nsys.path.append(\"../src\")\n\nfrom data_processing.download_step1 import download_worldcover_for_country, download_gsw_for_country\nfrom data_processing.raster_processor import RasterProcessor\nfrom data_processing.utils import clip_raster_to_country_and_create_cog"
+   "source": [
+    "import os\n",
+    "import sys\n",
+    "from glob import glob\n",
+    "from pathlib import Path\n",
+    "\n",
+    "import rasterio as rio\n",
+    "from rasterio.enums import Resampling\n",
+    "from rasterio.merge import merge\n",
+    "\n",
+    "sys.path.append(\"../src\")\n",
+    "\n",
+    "from data_processing.download_step1 import download_worldcover_for_country, download_gsw_for_country\n",
+    "from data_processing.raster_processor import RasterProcessor\n",
+    "from data_processing.utils import clip_raster_to_country_and_create_cog, resample_raster"
+   ]
   },
   {
    "cell_type": "markdown",
@@ -525,17 +558,7 @@
    "id": "5d89ee8c",
    "metadata": {},
    "outputs": [],
-   "source": [
-    "# WORKING CELL — set country, run, then revert.\n",
-    "country = \"your_country\"\n",
-    "\n",
-    "# Clip raster to country and create COG\n",
-    "raster_file = \"../data/processed/WorldCereal/2021_tc-annual_temporarycrops_classification_0.004deg.tif\"\n",
-    "output_dir = \"../data/processed/WorldCereal/Clipped/\"\n",
-    "\n",
-    "cog_file = clip_raster_to_country_and_create_cog(raster_file, country, output_dir=output_dir)\n",
-    "print(f\"Created COG: {cog_file}\")"
-   ]
+   "source": "# WORKING CELL — set country, run, then revert.\ncountry = \"your_country\"\n\n# Clip raster to country and create COG\nraster_file = \"../data/processed/WorldCereal/2021_tc-annual_temporarycrops_classification_0.004deg.tif\"\noutput_dir = \"../data/processed/WorldCereal/Clipped/\"\n\ncog_file = clip_raster_to_country_and_create_cog(raster_file, country, output_dir=output_dir)\nprint(f\"Created COG: {cog_file}\")"
   },
   {
    "cell_type": "code",
@@ -545,13 +568,23 @@
    "outputs": [],
    "source": [
     "# Style and process raster\n",
+    "# For large countries (e.g. India), resample first to avoid Mapbox 500KB tile limit at low zoom.\n",
+    "# Set resample_factor > 1 to downsample; 1 = no resampling.\n",
+    "resample_factor = 1\n",
+    "\n",
     "print(f\"Processing WorldCereal for {country}...\")\n",
     "BASE_PATH = Path(\"../data/processed/WorldCereal/\")\n",
     "input_file = BASE_PATH / f\"Clipped/2021_tc-annual_temporarycrops_classification_0.004deg_{country.replace(' ', '_')}.tif\"\n",
     "style_file = BASE_PATH / \"temporarycrops.qml\"\n",
     "output_file = BASE_PATH / f\"Rasters/WorldCereal_{country.replace(' ', '_')}.tif\"\n",
     "layer_name = f\"{country.replace(' ', '_')}_WorldCereal\"\n",
     "\n",
+    "if resample_factor > 1:\n",
+    "    resampled_file = input_file.parent / f\"{input_file.stem}_resampled{input_file.suffix}\"\n",
+    "    resample_raster(input_file, resampled_file, scale_factor=resample_factor, resampling_method=Resampling.nearest)\n",
+    "    input_file = resampled_file\n",
+    "    print(f\"Resampled by {resample_factor}x → {resampled_file}\")\n",
+    "\n",
     "RasterProcessor(\n",
     "    input_file,\n",
     "    output_file=output_file,\n",
@@ -673,7 +706,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "esa_gda_env",
+   "display_name": "esa-gda",
    "language": "python",
    "name": "python3"
   },
@@ -687,7 +720,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.23"
+   "version": "3.13.11"
   }
  },
  "nbformat": 4,

data-processing/src/raster_config.yaml

@@ -761,4 +761,21 @@ layers:
     style_file: "Pop_Accessibility_Classes.qml"
     output_file: "../data/processed/Health/Bhutan/Rasters/Bhutan_PopAccess_Hospitals.tif"
     layer_name: "Bhutan_PopAccess_Hospitals"
-    max_zoom: 13
+    max_zoom: 13
+
+  # India - Agriculture
+  india_worldcover:
+    base_path: "../data/raw/Agriculture/India/GDA-AGRI_UC1-India/ESA_WORLDCOVER/ESA_WorldCover_10m_2021_v200_N30E075_Map"
+    input_file: "ESA_WorldCover_10m_2021_v200_N30E075_Map_resampled.tif"
+    style_file: "../landcover.qml"
+    output_file: "../data/processed/Agriculture/India/Rasters/India_Worldcover.tif"
+    layer_name: "India_Worldcover"
+    max_zoom: 11
+
+  india_soc_change:
+    base_path: "../data/raw/Agriculture/India/GDA-AGRI_UC1-India/PREDICTIONS"
+    input_file: "SOC_prediction_Himachal-Pradesh-croplands_change-2018-2023.tif"
+    style_file: "SOC_change.qml"
+    output_file: "../data/processed/Agriculture/India/Rasters/India_SOC_Change.tif"
+    layer_name: "India_SOC_Change"
+    max_zoom: 14