You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CHANGELOG.md
+19Lines changed: 19 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,6 +6,25 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
6
and this project adheres to [PEP 440](https://www.python.org/dev/peps/pep-0440/)
7
7
and uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
8
8
9
+
## [0.3.0]
10
+
RAiDER package was refactored to expose the main functionality as a Python library, including the `prepareWeatherModel`
11
+
and `tropo_delay` functions, as well as anciliarry functions needed for defining AOIs, look vectors, etc.
12
+
13
+
### New/Updated Features
14
+
+ Python library access to main functions for accessing weather model data and calculating delays
15
+
+ Slant delay calculation through projection is supported for cubes with orbit files
16
+
+ Upgrade dem-stitcher to [`>=2.3.1`](https://github.com/ACCESS-Cloud-Based-InSAR/dem-stitcher/blob/dev/CHANGELOG.md#231) so that the updated urls for the GLO-30 DEM are used.
17
+
+`raider.py ++calcDelaysGUNW GUNWFILE` is enabled as a placeholder only.
18
+
+ Upgraded ISCE3 to `>=v0.9.0` to fix a conda build issue as described in [#425](https://github.com/dbekaert/RAiDER/issues/425)
19
+
+ Allow user to specify --download_only or download_only=True in the configure file
20
+
+ Added documentation for the Python library interface.
21
+
+ Added some unit tests.
22
+
+ Fixed some bugs and tweaked the CLI.
23
+
+ Added unit tests, docstrings, initial API reference
24
+
+__main__ file to allow calls to different functionality. `raider.py ++process downloadGNSS ...` can now perform the functionality of `raiderDownloadGNSS.py ...
25
+
26
+
27
+
9
28
## [0.2.0]
10
29
11
30
RAiDER package was refactored to use a configure file (yaml) to parse parameters. In addition, ocker container images
@@ -13,15 +14,18 @@ Copyright (c) 2019-2022, California Institute of Technology ("Caltech"). All rig
13
14
THIS IS RESEARCH CODE PROVIDED TO YOU "AS IS" WITH NO WARRANTIES OF CORRECTNESS. USE AT YOUR OWN RISK.
14
15
15
16
## Contents
16
-
-[1. Getting Started](#1-getting-started)
17
-
-[Installing With Conda](#installing-with-conda)
18
-
-[Using the Docker Image](#using-the-docker-image)
19
-
-[Installing from Source](#installing-from-source)
20
-
-[2. Setup of third party weather model access](#2-setup-of-third-party-weather-model-access)
21
-
-[3. Running RAiDER and Documentation](#3-running-raider-and-documentation)
22
-
-[4. Citing](#4-citation)
23
-
-[5. Contributors](#5-contributors)
17
+
18
+
1.[Getting Started](#1-getting-started)
19
+
-[Installing With Conda](#installing-with-conda)
20
+
-[Using the Docker Image](#using-the-docker-image)
21
+
-[Installing from Source](#installing-from-source)
22
+
2.[Setup of third party weather model access](#2-setup-of-third-party-weather-model-access)
23
+
3.[Running RAiDER and Documentation](#3-running-raider-and-documentation)
24
+
4.[Citing](#4-citation)
25
+
5.[Development](#5-development)
26
+
-[Contributors](#contributors)
24
27
------
28
+
25
29
## 1. Getting Started
26
30
27
31
RAiDER has been tested on the following systems:
@@ -31,6 +35,7 @@ RAiDER has been tested on the following systems:
31
35
RAiDER does **not** currently run on arm64 processors on Mac. We will update this note once the build becomes available.
32
36
33
37
### Installing With Conda
38
+
34
39
RAiDER is available on [conda-forge](https://anaconda.org/conda-forge/raider). __[Conda](https://docs.conda.io/en/latest/index.html)__ is a cross-platform way to use Python that allows you to setup and use "virtual environments." These can help to keep dependencies for different sets of code separate. We recommend using [Miniforge](https://github.com/conda-forge/miniforge), a conda environment manager that uses conda-forge as its default code repo. Alternatively,see __[here](https://docs.anaconda.com/anaconda/install/)__ for help installing Anaconda and __[here](https://docs.conda.io/en/latest/miniconda.html)__ for installing Miniconda.
35
40
36
41
Installing RAiDER:
@@ -40,6 +45,7 @@ conda activate RAiDER
40
45
```
41
46
42
47
### Using the Docker image
48
+
43
49
RAiDER provides a [docker container image](https://docs.docker.com/get-started/) with all the necessary dependencies pre-installed. To get the latest released version:
44
50
```
45
51
docker pull ghcr.io/dbekaert/raider:latest
@@ -64,32 +70,43 @@ cd work
64
70
```
65
71
For more docker run options, see: <https://docs.docker.com/engine/reference/run/>.
66
72
67
-
### Installing from source
68
-
You can also install RAiDER directly from source. Doing so is recommended for those who would like to [contribute to the source code](https://github.com/dbekaert/RAiDER/blob/dev/CONTRIBUTING.md), which we heartily encourage! For more details on installing from source see [here](https://github.com/dbekaert/RAiDER/blob/dev/Installing_from_source.md).
69
-
```
70
-
git clone https://github.com/dbekaert/RAiDER.git
71
-
cd RAiDER
72
-
conda create -f environment.yml
73
-
conda activate RAiDER
74
-
python -m pip install -e .
75
-
```
73
+
76
74
------
77
75
## 2. Setup of third party weather model access
78
-
RAiDER has the ability to download weather models from third-parties; some of which require license agreements. See [here](WeatherModels.md) for details.
76
+
77
+
RAiDER has the ability to download weather models from third-parties; some of which require license agreements. See [here](https://github.com/dbekaert/RAiDER/blob/dev/docs/WeatherModels.md) for details.
79
78
80
79
------
81
80
## 3. Running RAiDER and Documentation
81
+
82
82
For detailed documentation, examples, and Jupyter notebooks see the [RAiDER-docs repository](https://github.com/dbekaert/RAiDER-docs).
83
-
We welcome contributions of other examples on how to leverage the RAiDER (see [here](https://github.com/dbekaert/RAiDER/blob/master/CONTRIBUTING.md) for instructions).
83
+
We welcome contributions of other examples on how to leverage the RAiDER (see [here](https://github.com/dbekaert/RAiDER/blob/dev/CONTRIBUTING.md) for instructions).
84
84
``` raiderDelay.py -h ``` provides a help menu and list of example commands to get started.
85
-
The RAiDER scripts are highly modulized in Python and allows for building your own processing workflow.
85
+
The RAiDER scripts are highly modularized in Python and allows for building your own processing workflow.
86
86
87
87
------
88
88
## 4. Citation
89
89
TODO
90
90
91
91
------
92
-
## 5. Contributors
92
+
## 5. Development
93
+
94
+
Contributions are welcome and heartily encourage! See our [contributing guide](https://github.com/dbekaert/RAiDER/blob/dev/CONTRIBUTING.md).
95
+
96
+
### Development install
97
+
For development, we recommend installing directly from source.
98
+
```
99
+
git clone https://github.com/dbekaert/RAiDER.git
100
+
cd RAiDER
101
+
conda create -f environment.yml
102
+
conda activate RAiDER
103
+
python -m pip install -e .
104
+
```
105
+
For more details on installing from source see [here](https://github.com/dbekaert/RAiDER/blob/dev/docs/Installing_from_source.md).
RAiDER has built-in support for a number of different weather models. RAiDER provides all of the interfacing to data servers required to access data for the different weather models, although some weather models require a license agreement and accounts to be set-up. Instructions for accessing data, including license-limited data, are provided below. It is the user's responsibility to accept license agreements for whatever model is desired.
2
+
RAiDER has built-in support for a number of different weather models. RAiDER provides all the interfacing to data servers required to access data for the different weather models, although some weather models require a license agreement and accounts to be set-up. Instructions for accessing data, including license-limited data, are provided below. It is the user's responsibility to accept license agreements for whatever model is desired.
3
3
4
4
In addition, RAiDER provides functionality for adding additional weather models. See the [RAiDER-docs repository](https://github.com/dbekaert/RAiDER-docs) page on how to do this. We would love to expand the suite of supported models, and welcome any contributions. Please see the contributing guidelines or reach out through an issue ticket for help.
ERA-5/ERA-I products require access to the ESA Copernicus servers. GMAO and MERRA-2 products require access to the NASA Earthdata servers. If you are unable to download products, ensure that you have registered and have downloaded the public API key, and accepted/added the license/application for type of product you wish to download as detailed below.
16
17
17
18
------
18
19
19
-
## 1. NOAA weather models (HRRR)
20
-
High-resolution rapid refresh (HRRR) weather model data products are generated by __[NOAA](https://rapidrefresh.noaa.gov/hrrr/)__ for the coninental US (CONUS) but not archived beyond three days. However a public __[archive](home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr_FAQ.html)__ is available at the University of Utah. This archive does not require a license agreement. This model has the highest spatial resolution available in RAiDER, with a horizontal grid spacing of about 3 km, and is provided in a Lambert conformal conic projection.
21
-
22
-
[return to table of content](#contents)
23
-
20
+
## 2. NOAA weather models (HRRR)
21
+
High-resolution rapid refresh (HRRR) weather model data products are generated by __[NOAA](https://rapidrefresh.noaa.gov/hrrr/)__ for the coninental US (CONUS) but not archived beyond three days. However, a public __[archive](home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr_FAQ.html)__ is available at the University of Utah. This archive does not require a license agreement. This model has the highest spatial resolution available in RAiDER, with a horizontal grid spacing of about 3 km, and is provided in a Lambert conformal conic projection.
The Copernicus Climate Data Store (CDS) provides access to the European Centre for Medium-Range Weather Forecasts (__[ECMWF](https://www.ecmwf.int/)__) provides a number of different weather models, including ERA5 and ERA5T reanalysis models.
29
27
30
28
The ECMWF provides access to both reanalysis and real-time prediction models. You can read more information about their reanalysis models __[here](https://www.ecmwf.int/en/research/climate-reanalysis)__ and real-time model __[here](https://www.ecmwf.int/en/forecasts/datasets/catalogue-ecmwf-real-time-products)__. ECMWF models are global, with horizontal resolution of about 30 km for ERA-I, ERA-5, and ERA-5T, and 6 km for Hi-RES. All of these models come in a global projection (EPSG 4326, WGS-84).
@@ -34,41 +32,47 @@ The ECMWF provides access to both reanalysis and real-time prediction models. Yo
34
32
2. Confirm your email, etc.
35
33
3. Install the public API key and client as instructed __[here](https://cds.climate.copernicus.eu/api-how-to)__:
36
34
37
-
a. Copy the URL and API key from the webpage into a file in your home directory name ~/.cdsapirc
38
-
url: https://cds.climate.copernicus.eu/api/v2
39
-
key: your_key_here
40
-
__**Note**: the key represents the API key obtained upon the registration of CDS API, and should be replaced with the user's own information.__
41
-
42
-
b. Install the CDS API using pip:
43
-
pip install cdsapi
44
-
___**Note**: this step has been included in the conda install of RAiDER, thus can be omitted if one uses the recommended conda install of RAiDER___
35
+
a. Copy the URL and API key from the webpage into a file in your home directory name `~/.cdsapirc`
36
+
37
+
url: https://cds.climate.copernicus.eu/api/v2
38
+
key: your_key_here
39
+
40
+
**Note**: the key represents the API key obtained upon the registration of CDS API, and should be replaced with the user's own information.
41
+
42
+
b. Install the CDS API using pip
43
+
44
+
pip install cdsapi
45
+
46
+
**Note**: this step has been included in the conda install of RAiDER, thus can be omitted if one uses the recommended conda install of RAiDER
47
+
45
48
4. You must accept the [license](https://cds.climate.copernicus.eu/cdsapp/#!/terms/licence-to-use-copernicus-products) for each product you wish to download.
46
49
47
50
48
51
### Accessing ERAI, HRES
49
52
50
53
ECMWF requires a license agreement to be able to access, download, and use their products. Instructions for completing this process is below.
51
54
52
-
1. Create an account on the ECMWF servers __[here](https://accounts.ecmwf.int/auth/realms/ecmwf/protocol/openid-connect/auth?response_type=code&scope=openid%20email&client_id=apache-www&state=sBYlpcTRPhat8d6uuM9swLCxuP8&redirect_uri=https%3A%2F%2Fwww.ecmwf.int%2Foidc.cgi&nonce=RyEzBUy4m6oo_HxRQEmJxbc5jrKY4KFZd1Usgi8cpnM)__. The ERA-I model is open-access, while HRES requires a special liscence agreement.
55
+
1. Create an account on the ECMWF servers __[here](https://accounts.ecmwf.int/auth/realms/ecmwf/protocol/openid-connect/auth?response_type=code&scope=openid%20email&client_id=apache-www&state=sBYlpcTRPhat8d6uuM9swLCxuP8&redirect_uri=https%3A%2F%2Fwww.ecmwf.int%2Foidc.cgi&nonce=RyEzBUy4m6oo_HxRQEmJxbc5jrKY4KFZd1Usgi8cpnM)__. The ERA-I model is open-access, while HRES requires a special licence agreement.
53
56
2. Confirm your email, etc.
54
57
3. Install the public API key and client as instructed __[here](https://confluence.ecmwf.int/display/WEBAPI/Access+ECMWF+Public+Datasets#AccessECMWFPublicDatasets-key)__:
55
58
56
-
a. Copy the URL and API key from the webpage into a file in your home directory name ~/.ecmwfapirc
57
-
{
58
-
"url" : "https://api.ecmwf.int/v1",
59
-
"key" : your key here,
60
-
"email" : your email here
61
-
}
59
+
a. Copy the URL and API key from the webpage into a file in your home directory name `~/.ecmwfapirc`
60
+
61
+
{
62
+
"url" : "https://api.ecmwf.int/v1",
63
+
"key" : your key here,
64
+
"email" : your email here
65
+
}
62
66
63
-
__**Note**: the email that is used to register the user account, and the key represents the API key obtained upon the registration of ECMWF API, and should be replaced with the user's own information.__
67
+
**Note**: the email that is used to register the user account, and the key represents the API key obtained upon the registration of ECMWF API, and should be replaced with the user's own information.
64
68
65
-
b. Install the ECMWF API using pip:
66
-
```pip install ecmwf-api-client```
67
-
___**Note**: this step has been included in the conda install of RAiDER, thus can be omitted if one uses the recommended conda install of RAiDER___
69
+
b. Install the ECMWF API using pip:
68
70
69
-
[return to table of content](#contents)
71
+
pip install ecmwf-api-client`
70
72
71
-
### 3. NASA weather models (GMAO, MERRA2)
73
+
**Note**: this step has been included in the conda install of RAiDER, thus can be omitted if one uses the recommended conda install of RAiDER
74
+
75
+
## 4. NASA weather models (GMAO, MERRA2)
72
76
73
77
1. The Global Modeling and Assimilation Office (__[GMAO](https://www.nccs.nasa.gov/services/data-collections/coupled-products/geos5-forecast#:~:text=The%20Global%20Modeling%20and%20Assimilation,near%2Dreal%2Dtime%20production.)__) at NASA generates reanalysis weather models. GMAO products can also be accessed without a license agreement through the pyDAP interface implemented in RAiDER. GMAO has a horizontal grid spacing of approximately 33 km, and its projection is EPSG code 4326 (WGS-84).
74
78
@@ -85,17 +89,14 @@ Reference: __[The Modern-Era Retrospective Analysis for Research and Application
85
89
login <USERNAME>
86
90
password <PASSWORD>
87
91
88
-
__**Note**: the username and password represent the user's username and password.__
92
+
**Note**: the username and password represent the user's username and password.
89
93
90
94
4. Add the application `NASA GESDISC DATA ARCHIVE` by clicking on the `Applications->Authorized Apps` on the menu after logging into your Earthdata profile, and then scrolling down to the application `NASA GESDISC DATA ARCHIVE` to approve it. _This seems not required for GMAO for now, but recommended to do so for all OpenDAP-based weather models._
91
95
5. Install the OpenDAP using pip:
92
96
93
97
pip install pydap==3.2.1
94
98
95
99
96
-
___**Note**: this step has been included in the conda install of RAiDER, thus can be omitted if one uses the recommended conda install of RAiDER___
100
+
**Note**: this step has been included in the conda install of RAiDER, thus can be omitted if one uses the recommended conda install of RAiDER
97
101
98
-
___**Note**: PyDAP v3.2.1 is required for now (thus specified in the above pip install command) because the latest v3.2.2 (as of now) has a known [bug](https://colab.research.google.com/drive/1f_ss1Oa3VzgAOd_p8sgekdnLVE5NW6s5) in accessing and slicing the GMAO data. This bug is expected to be fixed in newer versions of PyDAP.___
99
-
100
-
[return to table of content](#contents)
101
-
102
+
**Note**: PyDAP v3.2.1 is required for now (thus specified in the above pip install command) because the latest v3.2.2 (as of now) has a known [bug](https://colab.research.google.com/drive/1f_ss1Oa3VzgAOd_p8sgekdnLVE5NW6s5) in accessing and slicing the GMAO data. This bug is expected to be fixed in newer versions of PyDAP.
0 commit comments