-
Couldn't load subscription status.
- Fork 134
Update to DEA Notebooks Template #1386
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Changes from 12 commits
ef6a30e
085d2d2
e1e5bc2
23bc08b
ffaf0d4
c8036fe
76f8cbd
edeb657
5810b83
9839ca2
c094892
2633d6d
f13c0e0
af4acbc
b1bd13b
293af5e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| { | ||
| "name": "dea-notebooks", | ||
| "image": "mcr.microsoft.com/devcontainers/base:ubuntu", | ||
| "features": { | ||
| "ghcr.io/devcontainers/features/python:1": { | ||
| "version": "3.11", | ||
| "installJupyterlab": true | ||
| } | ||
| }, | ||
| "postCreateCommand": "pip install -r requirements.txt", | ||
| "postStartCommand": "nohup jupyter lab --ip 0.0.0.0 --port 8888 --no-browser --NotebookApp.token='' > /tmp/jupyter.log 2>&1 &", | ||
| "forwardPorts": [8888], | ||
| "portsAttributes": { | ||
| "8888": { | ||
| "label": "JupyterLab", | ||
| "onAutoForward": "openBrowser" | ||
| } | ||
| }, | ||
| "hostRequirements": { | ||
| "cpus": 4, | ||
| "memory": "16gb", | ||
| "storage": "15gb" | ||
| }, | ||
| "customizations": { | ||
| "vscode": { | ||
| "extensions": [ | ||
| "ms-python.python", | ||
| "ms-toolsai.jupyter", | ||
| "charliermarsh.ruff" | ||
| ] | ||
| } | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,13 +1,82 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "<!-- DEA Notebook Header -->\n", | ||
| "<table style=\"width:100%; border:1px solid GrayText; border-radius:12px; background:Canvas; font-family:'Segoe UI',Roboto,sans-serif; line-height:1.6; color:CanvasText;\">\n", | ||
| "<tr>\n", | ||
| "<td style=\"padding:20px; vertical-align:top; width:80%;\">\n", | ||
| "\n", | ||
| "<h1 style=\"margin-top:0; color:CanvasText;\">Descriptive title that follows notebook filename</h1>\n", | ||
| "\n", | ||
| "<p>\n", | ||
| "🔎 Visit <a href=\"https://knowledge.dea.ga.gov.au\" style=\"color:inherit;\">DEA's Knowledge Hub</a> for detailed technical information about the datasets and concepts used in this notebook.\n", | ||
| "</p>\n", | ||
| "\n", | ||
| "<p>\n", | ||
| "<span style=\"display:inline-block; font-size:12px; background:rgba(59,130,246,0.15); border-radius:12px; padding:3px 10px; margin-right:6px;\">🟢 Beginner</span>\n", | ||
| "<span style=\"display:inline-block; font-size:12px; background:rgba(16,185,129,0.15); border-radius:12px; padding:3px 10px; margin-right:6px;\">⚡ Runtime: ~10 mins</span>\n", | ||
| "<span style=\"display:inline-block; font-size:12px; background:rgba(107,114,128,0.15); border-radius:12px; padding:3px 10px;\">🗓️ Updated: 2025-08</span>\n", | ||
| "</p>\n", | ||
| "\n", | ||
| "<!-- Products -->\n", | ||
| "<div style=\"margin-bottom:12px; padding:12px; background:rgba(59,130,246,0.08); border:1px solid rgb(59,130,246); border-radius:8px;\">\n", | ||
| "<b>📦 Products used</b><br>\n", | ||
| "<a href=\"https://knowledge.dea.ga.gov.au/products/ga_s2am_ard_3\">ga_s2am_ard_3</a>, \n", | ||
| "<a href=\"https://knowledge.dea.ga.gov.au/products/ga_s2bm_ard_3\">ga_s2bm_ard_3</a>, \n", | ||
| "<a href=\"https://knowledge.dea.ga.gov.au/products/ga_s2cm_ard_3\">ga_s2cm_ard_3</a>, \n", | ||
| "<a href=\"https://knowledge.dea.ga.gov.au/products/ga_ls5t_ard_3\">ga_ls5t_ard_3</a>, \n", | ||
| "<a href=\"https://knowledge.dea.ga.gov.au/products/ga_ls7e_ard_3\">ga_ls7e_ard_3</a>, \n", | ||
| "<a href=\"https://knowledge.dea.ga.gov.au/products/ga_ls8c_ard_3\">ga_ls8c_ard_3</a>\n", | ||
| "</div>\n", | ||
| "\n", | ||
| "<!-- Special requirements -->\n", | ||
| "<div style=\"margin-bottom:12px; padding:12px; background:rgba(59,130,246,0.08); border:1px solid rgb(59,130,246); border-radius:8px;\">\n", | ||
| "<b>⚙️ Special requirements</b><br>\n", | ||
| "Optional notes, e.g. If running on the NCI, ensure that <code>module load otps</code> is run prior to launching this notebook.\n", | ||
| "</div>\n", | ||
| "\n", | ||
| "<!-- Prerequisites -->\n", | ||
| "<div style=\"margin-bottom:12px; padding:12px; background:rgba(59,130,246,0.08); border:1px solid rgb(59,130,246); border-radius:8px;\">\n", | ||
| "<b>📖 Prerequisites</b><br>\n", | ||
| "Optional list of prerequisite notebooks or background knowledge.\n", | ||
| "</div>\n", | ||
| "\n", | ||
| "<!-- How to run -->\n", | ||
| "<div style=\"margin-bottom:12px; padding:14px; background:rgba(16,185,129,0.1); border:1px solid rgb(16,185,129); border-radius:8px;\">\n", | ||
| "<b>🚀 How you can run this notebook</b>\n", | ||
| "\n", | ||
| "- <b>DEA Sandbox:</b> Eligible users can sign up for the <a href=\"https://sandbox.dea.ga.gov.au/\" style=\"color:inherit;\">DEA Sandbox</a> that provides free compute and a managed Python environment.\n", | ||
| "\n", | ||
| "- <b>Personal computer:</b> This notebook loads DEA data using <code>odc-stac</code>. Data streams across the internet, so can be run locally after installing the required <a href=\"https://pypi.org/project/dea-tools/\" style=\"color:inherit;\">DEA Tools</a> Python package, e.g. <code>pip install dea-tools[notebooks]</code>.\n", | ||
| "\n", | ||
| "\n", | ||
| "- <b>Run this notebook online:</b> Click on the badge below to launch a GitHub Codespaces instance to run the notebook from your browser. See this [wiki page]() for more information.\n", | ||
| "\n", | ||
| " <a href=\"https://github.com/codespaces/new?hide_repo_select=true&ref=template&repo=GeoscienceAustralia%2Fdea-notebooks\" target=\"_blank\">\n", | ||
| " <img src=\"https://img.shields.io/badge/Codespaces-Open-blue?logo=github\" alt=\"Open in GitHub Codespaces\" style=\"height:25px;\">\n", | ||
| " </a>\n", | ||
| "</div>\n", | ||
| "\n", | ||
| "</td>\n", | ||
| "\n", | ||
| "<!-- Logo on the right -->\n", | ||
| "<td style=\"padding:20px; text-align:right; vertical-align:top; width:20%;\">\n", | ||
| " <img src=\"https://raw.githubusercontent.com/GeoscienceAustralia/dea-notebooks/refs/heads/develop/Supplementary_data/dea_logo.jpg\" alt=\"DEA logo\" style=\"width:220px; max-width:80%; height:auto;\">\n", | ||
| "</td>\n", | ||
| "</tr>\n", | ||
| "</table>\n" | ||
| ] | ||
| }, | ||
| { | ||
| "attachments": {}, | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### General advice (delete this cell before submitting for review)\n", | ||
| "\n", | ||
| "* Update the **Compatibility** line depending on whether the notebook is compatible with the `NCI` and `DEA Sandbox` environments (e.g. \"Notebook currently compatible with the `DEA Sandbox` environment only\" or \"Notebook currently compatible with the `NCI` environment only\")\n", | ||
| "* When adding **Products used**, embed the hyperlink to that specific product on the DEA Explorer using the `[product_name](product url)` syntax.\n", | ||
| "* When writing in Markdown cells, start each sentence on a **new line**. This makes it easy to see changes through git commits.\n", | ||
| "* Use Australian English in markdown cells and code comments.\n", | ||
|
|
@@ -32,25 +101,6 @@ | |
| "---\n" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Descriptive title that follows notebook filename <img align=\"right\" src=\"./Supplementary_data/dea_logo.jpg\">\n", | ||
| "\n", | ||
| "* **[Sign up to the DEA Sandbox](https://app.sandbox.dea.ga.gov.au/)** to run this notebook interactively from a browser\n", | ||
| "* **Compatibility:** Notebook currently compatible with both the `NCI` and `DEA Sandbox` environments\n", | ||
| "* **Products used:** \n", | ||
| "[ga_s2am_ard_3](https://explorer.dea.ga.gov.au/products/ga_s2am_ard_3), \n", | ||
| "[ga_s2bm_ard_3](https://explorer.dea.ga.gov.au/products/ga_s2bm_ard_3),\n", | ||
| "[ga_s2cm_ard_3](https://explorer.dea.ga.gov.au/products/ga_s2cm_ard_3),\n", | ||
| "[ga_ls5t_ard_3](https://explorer.dea.ga.gov.au/products/ga_ls5t_ard_3),\n", | ||
| "[ga_ls7e_ard_3](https://explorer.dea.ga.gov.au/products/ga_ls7e_ard_3),\n", | ||
| "[ga_ls8c_ard_3](https://explorer.dea.ga.gov.au/products/ga_ls8c_ard_3)\n", | ||
| "* **Special requirements:** An _optional_ description of any special requirements, e.g. If running on the [NCI](https://nci.org.au/), ensure that `module load otps` is run prior to launching this notebook\n", | ||
| "* **Prerequisites:** An _optional_ list of any notebooks that should be run or content that should be understood prior to launching this notebook\n" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
|
|
@@ -80,16 +130,7 @@ | |
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Getting started\n", | ||
| "\n", | ||
| "Provide any particular instructions that the user might need, e.g. To run this analysis, run all the cells in the notebook, starting with the \"Load packages\" cell. " | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Load packages\n", | ||
| "## Load packages\n", | ||
| "Import Python packages that are used for the analysis.\n", | ||
| "\n", | ||
| "Use standard import commands; some are shown below. \n", | ||
|
|
@@ -98,51 +139,69 @@ | |
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 1, | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "%matplotlib inline\n", | ||
| "\n", | ||
| "import datacube\n", | ||
| "import matplotlib.pyplot as plt\n", | ||
| "import pystac_client\n", | ||
| "import odc.stac\n", | ||
| "\n", | ||
| "import numpy as np\n", | ||
| "import pandas as pd\n", | ||
| "import xarray as xr\n", | ||
| "import matplotlib.pyplot as plt\n", | ||
| "\n", | ||
| "import sys\n", | ||
| "sys.path.insert(1, '../Tools/')\n", | ||
| "from dea_tools.plotting import rgb\n" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Connect to the datacube\n", | ||
| "## Connect to DEA's Explorer STAC API\n", | ||
| "\n", | ||
| "Connect to the datacube so we can access DEA data.\n", | ||
| "The `app` parameter is a unique name for the analysis which is based on the notebook file name." | ||
| "This allows us to search for data through DEA's STAC catalog" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 2, | ||
| "metadata": { | ||
| "jupyter": { | ||
| "source_hidden": true | ||
| } | ||
| }, | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "dc = datacube.Datacube(app='DEA_notebooks_template')" | ||
| "catalog = pystac_client.Client.open(\"https://explorer.dea.ga.gov.au/stac\")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### Analysis parameters\n", | ||
| "## Configure AWS access\n", | ||
| "\n", | ||
| "To load data via STAC, we must configure appropriate access to data stored on DEA’s Amazon S3 buckets. This can be done with the `odc.stac.configure_rio` function. The configuration below must be used when loading any DEA data through the STAC API.\n", | ||
| "\n", | ||
| "If opening a dask client, e.g. through `dea_tools.dask.create_local_dask_cluster()`, then this step is not required as the function handles this." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "odc.stac.configure_rio(\n", | ||
| " cloud_defaults=True,\n", | ||
| " aws={\"aws_unsigned\": True},\n", | ||
| ")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Analysis parameters\n", | ||
| "\n", | ||
| "An *optional* section to inform the user of any parameters they'll need to configure to run the notebook:\n", | ||
| "\n", | ||
|
|
@@ -152,7 +211,7 @@ | |
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 3, | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
|
|
@@ -174,7 +233,7 @@ | |
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 4, | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
|
|
@@ -192,7 +251,7 @@ | |
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 5, | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
|
|
@@ -210,7 +269,7 @@ | |
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 6, | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
|
|
@@ -232,24 +291,16 @@ | |
| "**Contact:** If you need assistance, please post a question on the [Open Data Cube Discord chat](https://discord.com/invite/4hhBQVas5U) or on the [GIS Stack Exchange](https://gis.stackexchange.com/questions/ask?tags=open-data-cube) using the `open-data-cube` tag (you can view previously asked questions [here](https://gis.stackexchange.com/questions/tagged/open-data-cube)).\n", | ||
| "If you would like to report an issue with this notebook, you can file one on [GitHub](https://github.com/GeoscienceAustralia/dea-notebooks).\n", | ||
| "\n", | ||
| "**Last modified:** September 2021\n", | ||
| "**Last modified:** September 2025\n", | ||
|
||
| "\n", | ||
| "**Compatible datacube version:** " | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 7, | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [ | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "1.8.19\n" | ||
| ] | ||
| } | ||
| ], | ||
| "outputs": [], | ||
| "source": [ | ||
| "print(datacube.__version__)" | ||
|
||
| ] | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| ## dea-tools will install most required packages | ||
| dea-tools[notebooks]>0.4 | ||
| dask-labextension | ||
| jupyter-resource-usage |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Has the wiki page link been left blank for now on purpose, or should this link to a docs page for codespaces?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a placeholder for now