|
| 1 | +--- |
| 2 | +title: "Evaluating your downstream task with Tessera embeddings" |
| 3 | +date: 2026-09-09 |
| 4 | +tags: [python, research] |
| 5 | +author: "Srinivasan Keshav" |
| 6 | +description: "tessera-eval turns a ground-truth shapefile into an honest accuracy estimate for Tessera embeddings — cross-validation, learning curves, and spatial splits." |
| 7 | +--- |
| 8 | + |
| 9 | +*tessera-eval turns a ground-truth shapefile into an honest accuracy estimate for Tessera embeddings — cross-validation, learning curves, and spatial splits.* |
| 10 | + |
| 11 | +<figure class="cover-image"> |
| 12 | + <img src="/blog/tessera-eval-austria-cover.jpg" alt="A false-colour TESSERA embedding map of farmland northeast of Vienna, Austria, showing sharp-edged agricultural fields in blues, greens and purples set against a magenta backdrop of built-up areas and roads" /> |
| 13 | + <figcaption>TESSERA 2.0 embeddings over the fields northeast of Vienna, Austria — the same area used in the crop-classification example below. Credit: Tessera Zarr Explorer, University of Cambridge; basemap Sentinel-2 cloudless by EOX, contains modified Copernicus Sentinel data.</figcaption> |
| 14 | +</figure> |
| 15 | + |
| 16 | +[Tessera](https://geotessera.org) compresses a year of Sentinel-1 and Sentinel-2 imagery into a single 128-dimensional embedding for every 10 m pixel, which serves as a general-purpose summary of what that patch of ground looked like and how it changed. For an ecologist the practical question is: are these embeddings good enough for my task, and which ML model should I use? The `tessera-eval` Python package helps to answer that. |
| 17 | + |
| 18 | +Assuming that your ground truth is in a shapefile, the basic idea is that each labelled pixel is matched with a corresponding embedding vector; your labels plus those vectors then become a standard supervised-learning problem. `tessera-eval` handles the details, fetching and mosaicking the Tessera tiles, sampling pixels from your polygons, and scoring models. It supports both classification (habitat class, land cover) and regression (canopy height, biomass, percent cover). |
| 19 | + |
| 20 | +It supports: |
| 21 | + |
| 22 | +- **k-fold cross-validation** — a robust accuracy estimate (macro-F1, or R²/RMSE/MAE for regression) with the fold-to-fold spread. |
| 23 | +- **Learning curves** — accuracy versus training-set size, so you can see whether collecting more labels would still help. |
| 24 | +- **Confusion matrices**, and predicted-vs-actual scatters for regression. |
| 25 | +- **Spatial train/test splits** — hold out a whole region instead of random pixels, so nearby look-alike pixels can't inflate the score. |
| 26 | +- **Spatial-context models** — MLPs over a 3×3 or 5×5 neighbourhood, alongside per-pixel k-NN, random forest, XGBoost and MLP. |
| 27 | + |
| 28 | +Here is a minimal run. [`austria_crops.geojson`](https://raw.githubusercontent.com/ucam-eo/tessera-eval/main/examples/austria_crops.geojson) is 349 field parcels near Vienna labelled by crop type: |
| 29 | + |
| 30 | +```python |
| 31 | +import geopandas as gpd |
| 32 | +from geotessera import GeoTessera |
| 33 | +from tessera_eval import load_embeddings_for_shapefile, run_kfold_cv |
| 34 | + |
| 35 | +gdf = gpd.read_file("austria_crops.geojson").to_crs(4326) # polygons with a "crop" column |
| 36 | + |
| 37 | +# One embedding per 10 m pixel inside your polygons (downloads tiles on first run) |
| 38 | +vectors, labels, class_names, stats = load_embeddings_for_shapefile( |
| 39 | + gdf, field="crop", year=2024, gt_instance=GeoTessera() |
| 40 | +) |
| 41 | +print(f"{len(labels):,} labelled pixels across {len(class_names)} classes") |
| 42 | + |
| 43 | +# 5-fold cross-validation, random forest on the raw embeddings |
| 44 | +for event in run_kfold_cv(vectors, labels, ["rf"], k=5, task="classification"): |
| 45 | + if event["type"] == "aggregate": |
| 46 | + m = event["models"]["rf"] |
| 47 | + print(f"macro-F1: {m['mean_f1']:.3f} ± {m['std_f1']:.3f}") |
| 48 | +``` |
| 49 | + |
| 50 | +That prints macro-F1: 0.796 ± 0.004, a reasonable result for 10-way crop classification from embeddings alone. Replace ["rf"] with ["nn", "rf", "xgboost", "mlp"] to compare all four per-pixel models at once. For a continuous target (canopy height, biomass, percent cover), pass task="regression" with the regressor names — ["nn_reg", "rf_reg", "xgboost_reg", "mlp_reg"] to get R²/RMSE/MAE instead of F1. One caveat: random k-fold on satellite data is optimistic because neighbouring pixels are correlated. For a realistic comparison, use a spatial split. |
| 51 | + |
| 52 | +Install with `pip install "tessera-eval[geotessera,xgboost]"`. A follow-up post will cover the command-line interface, which runs the same evaluations without writing any Python. |
| 53 | + |
| 54 | +To find out more, here is the link to the [git repo](https://github.com/ucam-eo/tessera-eval). |
| 55 | + |
| 56 | +--- |
| 57 | + |
| 58 | +*This post was first published on [Keshav's own blog](https://svr-sk818-web.cl.cam.ac.uk/keshav/blog/posts/2026-09-09-evaluating-downstream-tasks-with-tessera-embeddings.html) on 9 September 2026.* |
| 59 | + |
| 60 | +<style> |
| 61 | + .cover-image { |
| 62 | + margin: 0 0 2rem 0; |
| 63 | + width: 100%; |
| 64 | + } |
| 65 | + |
| 66 | + .cover-image img { |
| 67 | + width: 100%; |
| 68 | + height: auto; |
| 69 | + max-height: 520px; |
| 70 | + object-fit: cover; |
| 71 | + object-position: center center; |
| 72 | + display: block; |
| 73 | + border-radius: 4px; |
| 74 | + } |
| 75 | + |
| 76 | + figure { |
| 77 | + margin: 0; |
| 78 | + } |
| 79 | + |
| 80 | + figcaption { |
| 81 | + font-size: 12px; |
| 82 | + color: var(--text-muted); |
| 83 | + margin-top: 6px; |
| 84 | + line-height: 1.5; |
| 85 | + font-style: italic; |
| 86 | + } |
| 87 | +</style> |
0 commit comments