|
1 | | -# Description |
2 | | -This repository contains scripts to process necessary geospatial datasets |
3 | | -and implement efficient zonal statistics on given ESRI Shapefiles. The |
4 | | -general usage of the script (i.e., `./extract-gis.sh`) is as follows: |
5 | | - |
6 | | -```console |
7 | | -Usage: |
8 | | - extract-gis [options...] |
9 | | - |
10 | | -Script options: |
11 | | - -d, --dataset Geospatial dataset of interest |
12 | | - -i, --dataset-dir=DIR The source path of the dataset file(s) |
13 | | - -r, --crs=INT The EPSG code of interest; optional |
14 | | - [defaults to 4326] |
15 | | - -v, --variable=var1[,var2[...]] If applicable, variables to process |
16 | | - -o, --output-dir=DIR Writes processed files to DIR |
17 | | - -s, --start-date=DATE If applicable, start date of the geospatial |
18 | | - data; optional |
19 | | - -e, --end-date=DATE If applicable, end date of the geospatial |
20 | | - data; optional |
21 | | - -l, --lat-lims=REAL,REAL Latitude's upper and lower bounds; optional |
22 | | - -n, --lon-lims=REAL,REAL Longitude's upper and lower bounds; optional |
23 | | - -f, --shape-file=PATH Path to the ESRI '.shp' file; optional |
24 | | - -F, --fid=STR Column name representing elements of the |
25 | | - ESRI Shapefile to report statistics; optional |
26 | | - defaults to the first column |
27 | | - -j, --submit-job Submit the data extraction process as a job |
28 | | - on the SLURM system; optional |
29 | | - -t, --print-geotiff=BOOL Extract the subsetted GeoTIFF file; optional |
30 | | - [defaults to 'true'] |
31 | | - -a, --stat=stat1[,stat2[...]] If applicable, extract the statistics of |
32 | | - interest, currently available options are: |
33 | | - 'min';'max';'mean';'majority';'minority'; |
34 | | - 'median';'quantile';'variety';'variance'; |
35 | | - 'stdev';'coefficient_of_variation';'frac'; |
36 | | - 'coords'; 'count'; 'sum'; optional |
37 | | - -U, --include-na Include NA values in generated statistics; |
38 | | - optional |
39 | | - -q, --quantile=q1[,q2[...]] Quantiles of interest to be produced if 'quantile' |
40 | | - is included in the '--stat' argument. The values |
41 | | - must be comma delimited float numbers between |
42 | | - 0 and 1; optional [defaults to every 5th quantile] |
43 | | - -p, --prefix=STR Prefix prepended to the output files |
44 | | - -b, --parsable Parsable SLURM message mainly used |
45 | | - for chained job submissions |
46 | | - -c, --cache=DIR Path of the cache directory; optional |
47 | | - -E, --email=STR E-mail when job starts, ends, and |
48 | | - fails; optional |
49 | | - -u, --account Digital Research Alliance of Canada's sponsor's |
50 | | - account name; optional, defaults to 'rpp-kshook' |
51 | | - -L, --lib-path Path to the shared libraries needed; optional, |
52 | | - see the source code for the default path |
53 | | - -V, --version Show version |
54 | | - -h, --help Show this screen and exit |
55 | | - |
56 | | -For bug reports, questions, and discussions open an issue |
57 | | -at https://github.com/kasra-keshavarz/gistool/issues |
58 | | -``` |
59 | | - |
60 | | - |
61 | | -# Available Datasets |
62 | | -|**#**|Dataset |Time Scale |CRS |DOI |Description | |
63 | | -|-----|--------------------------------------------|----------------------|-----|-------------------------------|---------------------| |
64 | | -|**1**|MODIS |2000 - 2021 | |10.5067/MODIS/MCD12Q1.006 |[link](modis) | |
65 | | -|**2**|MERIT Hydro |Not Applicable (N/A) |4326 |10.1029/2019WR024873 |[link](merit_hydro) | |
66 | | -|**3**|Soil Grids (v1) |Not Applicable (N/A) |4326 |10.1371/journal.pone.0169748 |[link](soil_grids) | |
67 | | -|**4**|Landsat NALCMS |2010 and 2015 |4326 |10.3390/rs9111098 |[link](landsat) | |
68 | | -|**5**|Global Depth to Bedrock |Not Applicable (N/A) | |10.1002/2016MS000686 |[link](depth_to_bedrock) | |
69 | | -|**6**|USDA Soil Class |Not Applicable (N/A) |4326 |10.4211/hs.1361509511e44adfba814f6950c6e742|[link](soil_class)| |
70 | | -|**7**|Global Soil Dataset for Earth System Modelling (GSDE)|Not Applicable (N/A)|4326 |10.1002/2013MS000293 |[link](GSDE) | |
71 | | - |
72 | | -# General Example |
73 | | -As an example, follow the code block below. Please remember that you MUST have access to Graham cluster with Digital Alliance of Canada. Also, remember to generate a [Personal Access Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) with GitHub in advance. Enter the following codes in your Graham shell as a test case: |
74 | | - |
75 | | -```console |
76 | | -foo@bar:~$ git clone https://github.com/kasra-keshavarz/gistool.git # clone the repository |
77 | | -foo@bar:~$ cd ./gistool/ # always move to the repository's directory |
78 | | -foo@bar:~$ ./extract-gis.sh -h # view the usage message |
79 | | -foo@bar:~$ ./extract-gis.sh \ |
80 | | - --dataset="merit-hydro" \ |
81 | | - --dataset-dir="/project/rpp-kshook/CompHydCore/merit_hydro/raw_data/" \ |
82 | | - --output-dir="$HOME/scratch/merit-hydro-test" \ |
83 | | - --lat-lims="45,47" \ |
84 | | - --lon-lims="-120,-117" \ |
85 | | - --print-geotiff=true \ |
86 | | - --variable="elv,hnd" \ |
87 | | - --prefix="merit_test_"; |
88 | | -``` |
89 | | -See the [example](./example) directory for real-world scripts for each geospatial dataset included in this repository. |
90 | | - |
91 | | - |
92 | | -# Logs |
93 | | -The datasets logs are generated under the `$HOME/.gistool` directory, |
94 | | -only in cases where jobs are submitted to clusters' schedulers. If |
95 | | -processing is not submitted as a job, then the logs are printed on screen. |
96 | | - |
97 | | - |
98 | | -# `--lib-path` options |
99 | | -Currently, on Graham HPC, the following options are available: |
100 | | -```console |
101 | | -/project/rpp-kshook/Climate_Forcing_Data/assets/r-envs/ # default, rpp-kshook allocation |
102 | | -/project/rrg-mclark/lib # rrg-mclark allocation |
103 | | -``` |
104 | | - |
105 | | - |
106 | | -# New Datasets |
107 | | -If you are considering any new dataset to be added to the data repository, |
108 | | -and subsequently the associated scripts added here, you can open a new |
109 | | -ticket on the **Issues** tab of the current repository. Or, you can make |
110 | | -a [Pull Request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) |
111 | | -on this repository with your own script. |
112 | | - |
113 | | - |
114 | | -# Support |
115 | | -Please open a new ticket on the **Issues** tab of the current repository in case of any problem. |
116 | | - |
| 1 | +# Documentation |
| 2 | +The relevant documentation is located on [Readthedocs](https://gistool.readthedocs.io/en/latest/) website. |
117 | 3 |
|
118 | 4 | # License |
119 | | -Geospatial Dataset Processing Workflow<br> |
| 5 | +Geospatial Data Processing Workflow - gistool <br> |
120 | 6 | Copyright (C) 2022-2023, University of Saskatchewan<br> |
121 | 7 | Copyright (C) 2023-2024, University of Calgary<br> |
| 8 | +Copyright (C) 2022-2024, datatool developers |
122 | 9 |
|
123 | 10 | This program is free software: you can redistribute it and/or modify |
124 | 11 | it under the terms of the GNU General Public License as published by |
|
0 commit comments