diff --git a/README.md b/README.md index 31edcac..d96b71a 100644 --- a/README.md +++ b/README.md @@ -50,6 +50,7 @@ artifacts. - [Spun up initial conditions for soil, snow, canopy, with no flux at the bottom](https://github.com/CliMA/ClimaArtifacts/tree/main/saturated_land_ic) - [CLM surface data for the year 2000](https://github.com/CliMA/ClimaArtifacts/tree/main/clm_data) - [Soil texture parameters needed for the full soil model; derived from SoilGrids](https://github.com/CliMA/ClimaArtifacts/tree/main/soilgrids) +- [Soil crganic carbon densities for the soil co2 model; derived from SoilGrids](https://github.com/CliMA/ClimaArtifacts/tree/main/soilgrids_ocd) - [Soil parameters needed for Richards equation; global at 1km resolution from S. Gupta et al 2022, 2024](https://github.com/CliMA/ClimaArtifacts/tree/main/soil_params_Gupta2020_2022) - [Foliage clumping index, derived from MODIS data for 2006](https:////github.com/CliMA/ClimaArtifacts/tree/main/modis_clumping_index) diff --git a/soilgrids_ocd/LICENSE b/soilgrids_ocd/LICENSE new file mode 100644 index 0000000..1020133 --- /dev/null +++ b/soilgrids_ocd/LICENSE @@ -0,0 +1,395 @@ +Attribution 4.0 International + +======================================================================= + +Creative Commons Corporation ("Creative Commons") is not a law firm and +does not provide legal services or legal advice. Distribution of +Creative Commons public licenses does not create a lawyer-client or +other relationship. Creative Commons makes its licenses and related +information available on an "as-is" basis. Creative Commons gives no +warranties regarding its licenses, any material licensed under their +terms and conditions, or any related information. Creative Commons +disclaims all liability for damages resulting from their use to the +fullest extent possible. + +Using Creative Commons Public Licenses + +Creative Commons public licenses provide a standard set of terms and +conditions that creators and other rights holders may use to share +original works of authorship and other material subject to copyright +and certain other rights specified in the public license below. The +following considerations are for informational purposes only, are not +exhaustive, and do not form part of our licenses. + + Considerations for licensors: Our public licenses are + intended for use by those authorized to give the public + permission to use material in ways otherwise restricted by + copyright and certain other rights. Our licenses are + irrevocable. Licensors should read and understand the terms + and conditions of the license they choose before applying it. + Licensors should also secure all rights necessary before + applying our licenses so that the public can reuse the + material as expected. Licensors should clearly mark any + material not subject to the license. This includes other CC- + licensed material, or material used under an exception or + limitation to copyright. More considerations for licensors: + wiki.creativecommons.org/Considerations_for_licensors + + Considerations for the public: By using one of our public + licenses, a licensor grants the public permission to use the + licensed material under specified terms and conditions. If + the licensor's permission is not necessary for any reason--for + example, because of any applicable exception or limitation to + copyright--then that use is not regulated by the license. Our + licenses grant only permissions under copyright and certain + other rights that a licensor has authority to grant. Use of + the licensed material may still be restricted for other + reasons, including because others have copyright or other + rights in the material. A licensor may make special requests, + such as asking that all changes be marked or described. + Although not required by our licenses, you are encouraged to + respect those requests where reasonable. More considerations + for the public: + wiki.creativecommons.org/Considerations_for_licensees + +======================================================================= + +Creative Commons Attribution 4.0 International Public License + +By exercising the Licensed Rights (defined below), You accept and agree +to be bound by the terms and conditions of this Creative Commons +Attribution 4.0 International Public License ("Public License"). To the +extent this Public License may be interpreted as a contract, You are +granted the Licensed Rights in consideration of Your acceptance of +these terms and conditions, and the Licensor grants You such rights in +consideration of benefits the Licensor receives from making the +Licensed Material available under these terms and conditions. + + +Section 1 -- Definitions. + + a. Adapted Material means material subject to Copyright and Similar + Rights that is derived from or based upon the Licensed Material + and in which the Licensed Material is translated, altered, + arranged, transformed, or otherwise modified in a manner requiring + permission under the Copyright and Similar Rights held by the + Licensor. For purposes of this Public License, where the Licensed + Material is a musical work, performance, or sound recording, + Adapted Material is always produced where the Licensed Material is + synched in timed relation with a moving image. + + b. Adapter's License means the license You apply to Your Copyright + and Similar Rights in Your contributions to Adapted Material in + accordance with the terms and conditions of this Public License. + + c. Copyright and Similar Rights means copyright and/or similar rights + closely related to copyright including, without limitation, + performance, broadcast, sound recording, and Sui Generis Database + Rights, without regard to how the rights are labeled or + categorized. For purposes of this Public License, the rights + specified in Section 2(b)(1)-(2) are not Copyright and Similar + Rights. + + d. Effective Technological Measures means those measures that, in the + absence of proper authority, may not be circumvented under laws + fulfilling obligations under Article 11 of the WIPO Copyright + Treaty adopted on December 20, 1996, and/or similar international + agreements. + + e. Exceptions and Limitations means fair use, fair dealing, and/or + any other exception or limitation to Copyright and Similar Rights + that applies to Your use of the Licensed Material. + + f. Licensed Material means the artistic or literary work, database, + or other material to which the Licensor applied this Public + License. + + g. Licensed Rights means the rights granted to You subject to the + terms and conditions of this Public License, which are limited to + all Copyright and Similar Rights that apply to Your use of the + Licensed Material and that the Licensor has authority to license. + + h. Licensor means the individual(s) or entity(ies) granting rights + under this Public License. + + i. Share means to provide material to the public by any means or + process that requires permission under the Licensed Rights, such + as reproduction, public display, public performance, distribution, + dissemination, communication, or importation, and to make material + available to the public including in ways that members of the + public may access the material from a place and at a time + individually chosen by them. + + j. Sui Generis Database Rights means rights other than copyright + resulting from Directive 96/9/EC of the European Parliament and of + the Council of 11 March 1996 on the legal protection of databases, + as amended and/or succeeded, as well as other essentially + equivalent rights anywhere in the world. + + k. You means the individual or entity exercising the Licensed Rights + under this Public License. Your has a corresponding meaning. + + +Section 2 -- Scope. + + a. License grant. + + 1. Subject to the terms and conditions of this Public License, + the Licensor hereby grants You a worldwide, royalty-free, + non-sublicensable, non-exclusive, irrevocable license to + exercise the Licensed Rights in the Licensed Material to: + + a. reproduce and Share the Licensed Material, in whole or + in part; and + + b. produce, reproduce, and Share Adapted Material. + + 2. Exceptions and Limitations. For the avoidance of doubt, where + Exceptions and Limitations apply to Your use, this Public + License does not apply, and You do not need to comply with + its terms and conditions. + + 3. Term. The term of this Public License is specified in Section + 6(a). + + 4. Media and formats; technical modifications allowed. The + Licensor authorizes You to exercise the Licensed Rights in + all media and formats whether now known or hereafter created, + and to make technical modifications necessary to do so. The + Licensor waives and/or agrees not to assert any right or + authority to forbid You from making technical modifications + necessary to exercise the Licensed Rights, including + technical modifications necessary to circumvent Effective + Technological Measures. For purposes of this Public License, + simply making modifications authorized by this Section 2(a) + (4) never produces Adapted Material. + + 5. Downstream recipients. + + a. Offer from the Licensor -- Licensed Material. Every + recipient of the Licensed Material automatically + receives an offer from the Licensor to exercise the + Licensed Rights under the terms and conditions of this + Public License. + + b. No downstream restrictions. You may not offer or impose + any additional or different terms or conditions on, or + apply any Effective Technological Measures to, the + Licensed Material if doing so restricts exercise of the + Licensed Rights by any recipient of the Licensed + Material. + + 6. No endorsement. Nothing in this Public License constitutes or + may be construed as permission to assert or imply that You + are, or that Your use of the Licensed Material is, connected + with, or sponsored, endorsed, or granted official status by, + the Licensor or others designated to receive attribution as + provided in Section 3(a)(1)(A)(i). + + b. Other rights. + + 1. Moral rights, such as the right of integrity, are not + licensed under this Public License, nor are publicity, + privacy, and/or other similar personality rights; however, to + the extent possible, the Licensor waives and/or agrees not to + assert any such rights held by the Licensor to the limited + extent necessary to allow You to exercise the Licensed + Rights, but not otherwise. + + 2. Patent and trademark rights are not licensed under this + Public License. + + 3. To the extent possible, the Licensor waives any right to + collect royalties from You for the exercise of the Licensed + Rights, whether directly or through a collecting society + under any voluntary or waivable statutory or compulsory + licensing scheme. In all other cases the Licensor expressly + reserves any right to collect such royalties. + + +Section 3 -- License Conditions. + +Your exercise of the Licensed Rights is expressly made subject to the +following conditions. + + a. Attribution. + + 1. If You Share the Licensed Material (including in modified + form), You must: + + a. retain the following if it is supplied by the Licensor + with the Licensed Material: + + i. identification of the creator(s) of the Licensed + Material and any others designated to receive + attribution, in any reasonable manner requested by + the Licensor (including by pseudonym if + designated); + + ii. a copyright notice; + + iii. a notice that refers to this Public License; + + iv. a notice that refers to the disclaimer of + warranties; + + v. a URI or hyperlink to the Licensed Material to the + extent reasonably practicable; + + b. indicate if You modified the Licensed Material and + retain an indication of any previous modifications; and + + c. indicate the Licensed Material is licensed under this + Public License, and include the text of, or the URI or + hyperlink to, this Public License. + + 2. You may satisfy the conditions in Section 3(a)(1) in any + reasonable manner based on the medium, means, and context in + which You Share the Licensed Material. For example, it may be + reasonable to satisfy the conditions by providing a URI or + hyperlink to a resource that includes the required + information. + + 3. If requested by the Licensor, You must remove any of the + information required by Section 3(a)(1)(A) to the extent + reasonably practicable. + + 4. If You Share Adapted Material You produce, the Adapter's + License You apply must not prevent recipients of the Adapted + Material from complying with this Public License. + + +Section 4 -- Sui Generis Database Rights. + +Where the Licensed Rights include Sui Generis Database Rights that +apply to Your use of the Licensed Material: + + a. for the avoidance of doubt, Section 2(a)(1) grants You the right + to extract, reuse, reproduce, and Share all or a substantial + portion of the contents of the database; + + b. if You include all or a substantial portion of the database + contents in a database in which You have Sui Generis Database + Rights, then the database in which You have Sui Generis Database + Rights (but not its individual contents) is Adapted Material; and + + c. You must comply with the conditions in Section 3(a) if You Share + all or a substantial portion of the contents of the database. + +For the avoidance of doubt, this Section 4 supplements and does not +replace Your obligations under this Public License where the Licensed +Rights include other Copyright and Similar Rights. + + +Section 5 -- Disclaimer of Warranties and Limitation of Liability. + + a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE + EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS + AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF + ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, + IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, + WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR + PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, + ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT + KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT + ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. + + b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE + TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, + NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, + INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, + COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR + USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN + ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR + DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR + IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. + + c. The disclaimer of warranties and limitation of liability provided + above shall be interpreted in a manner that, to the extent + possible, most closely approximates an absolute disclaimer and + waiver of all liability. + + +Section 6 -- Term and Termination. + + a. This Public License applies for the term of the Copyright and + Similar Rights licensed here. However, if You fail to comply with + this Public License, then Your rights under this Public License + terminate automatically. + + b. Where Your right to use the Licensed Material has terminated under + Section 6(a), it reinstates: + + 1. automatically as of the date the violation is cured, provided + it is cured within 30 days of Your discovery of the + violation; or + + 2. upon express reinstatement by the Licensor. + + For the avoidance of doubt, this Section 6(b) does not affect any + right the Licensor may have to seek remedies for Your violations + of this Public License. + + c. For the avoidance of doubt, the Licensor may also offer the + Licensed Material under separate terms or conditions or stop + distributing the Licensed Material at any time; however, doing so + will not terminate this Public License. + + d. Sections 1, 5, 6, 7, and 8 survive termination of this Public + License. + + +Section 7 -- Other Terms and Conditions. + + a. The Licensor shall not be bound by any additional or different + terms or conditions communicated by You unless expressly agreed. + + b. Any arrangements, understandings, or agreements regarding the + Licensed Material not stated herein are separate from and + independent of the terms and conditions of this Public License. + + +Section 8 -- Interpretation. + + a. For the avoidance of doubt, this Public License does not, and + shall not be interpreted to, reduce, limit, restrict, or impose + conditions on any use of the Licensed Material that could lawfully + be made without permission under this Public License. + + b. To the extent possible, if any provision of this Public License is + deemed unenforceable, it shall be automatically reformed to the + minimum extent necessary to make it enforceable. If the provision + cannot be reformed, it shall be severed from this Public License + without affecting the enforceability of the remaining terms and + conditions. + + c. No term or condition of this Public License will be waived and no + failure to comply consented to unless expressly agreed to by the + Licensor. + + d. Nothing in this Public License constitutes or may be interpreted + as a limitation upon, or waiver of, any privileges and immunities + that apply to the Licensor or You, including from the legal + processes of any jurisdiction or authority. + + +======================================================================= + +Creative Commons is not a party to its public +licenses. Notwithstanding, Creative Commons may elect to apply one of +its public licenses to material it publishes and in those instances +will be considered the “Licensor.” The text of the Creative Commons +public licenses is dedicated to the public domain under the CC0 Public +Domain Dedication. Except for the limited purpose of indicating that +material is shared under a Creative Commons public license or as +otherwise permitted by the Creative Commons policies published at +creativecommons.org/policies, Creative Commons does not authorize the +use of the trademark "Creative Commons" or any other trademark or logo +of Creative Commons without its prior written consent including, +without limitation, in connection with any unauthorized modifications +to any of its public licenses or any other arrangements, +understandings, or agreements concerning use of licensed material. For +the avoidance of doubt, this paragraph does not form part of the +public licenses. + +Creative Commons may be contacted at creativecommons.org. \ No newline at end of file diff --git a/soilgrids_ocd/Manifest.toml b/soilgrids_ocd/Manifest.toml new file mode 100644 index 0000000..daa757f --- /dev/null +++ b/soilgrids_ocd/Manifest.toml @@ -0,0 +1,541 @@ +# This file is machine-generated - editing it directly is not advised + +julia_version = "1.12.5" +manifest_format = "2.0" +project_hash = "944e4f8173cccfdf7bffa1f18d203a2db9e8ef0d" + +[[deps.ArgTools]] +uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f" +version = "1.1.2" + +[[deps.ArtifactUtils]] +deps = ["Downloads", "Git", "HTTP", "Pkg", "ProgressLogging", "SHA", "TOML", "gh_cli_jll"] +git-tree-sha1 = "d111d8f4f6158a083869cb987adb957cd7330e6c" +uuid = "8b73e784-e7d8-4ea5-973d-377fed4e3bce" +version = "0.2.5" + +[[deps.Artifacts]] +uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33" +version = "1.11.0" + +[[deps.Base64]] +uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f" +version = "1.11.0" + +[[deps.BitFlags]] +git-tree-sha1 = "0691e34b3bb8be9307330f88d1a3c3f25466c24d" +uuid = "d1d4a3ce-64b1-5f1a-9ba4-7e7e69966f35" +version = "0.1.9" + +[[deps.Blosc_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Lz4_jll", "Zlib_jll", "Zstd_jll"] +git-tree-sha1 = "535c80f1c0847a4c967ea945fca21becc9de1522" +uuid = "0b7ba130-8d10-5ba8-a3d6-c5182647fed9" +version = "1.21.7+0" + +[[deps.Bzip2_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl"] +git-tree-sha1 = "1b96ea4a01afe0ea4090c5c8039690672dd13f2e" +uuid = "6e34b625-4abd-537c-b88f-471c36dfa7a0" +version = "1.0.9+0" + +[[deps.CFTime]] +deps = ["Dates", "Printf"] +git-tree-sha1 = "b2c9f2d3b8014323c0a8f2b401b68fa6bb08ed06" +uuid = "179af706-886a-5703-950a-314cd64e0468" +version = "0.2.8" + +[[deps.ClimaArtifactsHelper]] +deps = ["ArtifactUtils", "Downloads", "Pkg", "REPL", "SHA"] +path = "../ClimaArtifactsHelper.jl" +uuid = "6ffa2572-8378-4377-82eb-ea11db28b255" +version = "0.0.1" + +[[deps.CodecZlib]] +deps = ["TranscodingStreams", "Zlib_jll"] +git-tree-sha1 = "962834c22b66e32aa10f7611c08c8ca4e20749a9" +uuid = "944b1d66-785c-5afd-91f1-9de20f533193" +version = "0.7.8" + +[[deps.CommonDataModel]] +deps = ["CFTime", "DataStructures", "Dates", "DiskArrays", "Preferences", "Printf", "Statistics"] +git-tree-sha1 = "bf07704e843daabd2cb2bb1404571656f80bce16" +uuid = "1fbeeb36-5f17-413c-809b-666fb144f157" +version = "0.4.3" + +[[deps.CompilerSupportLibraries_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae" +version = "1.3.0+1" + +[[deps.ConcurrentUtilities]] +deps = ["Serialization", "Sockets"] +git-tree-sha1 = "21d088c496ea22914fe80906eb5bce65755e5ec8" +uuid = "f0e56b4a-5159-44fe-b623-3e5288b988bb" +version = "2.5.1" + +[[deps.ConstructionBase]] +git-tree-sha1 = "b4b092499347b18a015186eae3042f72267106cb" +uuid = "187b0558-2788-49d3-abe0-74a17ed4e7c9" +version = "1.6.0" + + [deps.ConstructionBase.extensions] + ConstructionBaseIntervalSetsExt = "IntervalSets" + ConstructionBaseLinearAlgebraExt = "LinearAlgebra" + ConstructionBaseStaticArraysExt = "StaticArrays" + + [deps.ConstructionBase.weakdeps] + IntervalSets = "8197267c-284f-5f27-9208-e0e47529a953" + LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e" + StaticArrays = "90137ffa-7385-5640-81b9-e52037218182" + +[[deps.DataStructures]] +deps = ["OrderedCollections"] +git-tree-sha1 = "e86f4a2805f7f19bec5129bc9150c38208e5dc23" +uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8" +version = "0.19.4" + +[[deps.Dates]] +deps = ["Printf"] +uuid = "ade2ca70-3891-5945-98fb-dc099432e06a" +version = "1.11.0" + +[[deps.DiskArrays]] +deps = ["ConstructionBase", "LRUCache", "Mmap", "OffsetArrays"] +git-tree-sha1 = "e5d9ce1b751ddf9bcd9d36b51249dce8ea73cd55" +uuid = "3c3547ce-8d99-4f5e-a174-61eb10b00ae3" +version = "0.4.19" + +[[deps.Downloads]] +deps = ["ArgTools", "FileWatching", "LibCURL", "NetworkOptions"] +uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6" +version = "1.7.0" + +[[deps.ExceptionUnwrapping]] +deps = ["Test"] +git-tree-sha1 = "d36f682e590a83d63d1c7dbd287573764682d12a" +uuid = "460bff9d-24e4-43bc-9d9f-a8973cb893f4" +version = "0.1.11" + +[[deps.Expat_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl"] +git-tree-sha1 = "27af30de8b5445644e8ffe3bcb0d72049c089cf1" +uuid = "2e619515-83b5-522b-bb60-26c02a35a201" +version = "2.7.3+0" + +[[deps.FileWatching]] +uuid = "7b1f6079-737a-58dc-b8bc-7a2ca5c1b5ee" +version = "1.11.0" + +[[deps.Git]] +deps = ["Git_LFS_jll", "Git_jll", "JLLWrappers", "OpenSSH_jll"] +git-tree-sha1 = "824a1890086880696fc908fe12a17bcf61738bd8" +uuid = "d7ba0133-e1db-5d97-8f8c-041e4b3a1eb2" +version = "1.5.0" + +[[deps.Git_LFS_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl"] +git-tree-sha1 = "bb8471f313ed941f299aa53d32a94ab3bee08844" +uuid = "020c3dae-16b3-5ae5-87b3-4cb189e250b2" +version = "3.7.0+0" + +[[deps.Git_jll]] +deps = ["Artifacts", "Expat_jll", "JLLWrappers", "LibCURL_jll", "Libdl", "Libiconv_jll", "OpenSSL_jll", "PCRE2_jll", "Zlib_jll"] +git-tree-sha1 = "dc34a3e3d96b4ed305b641e626dc14c12b7824b8" +uuid = "f8c6e375-362e-5223-8a59-34ff63f689eb" +version = "2.53.0+0" + +[[deps.HDF5_jll]] +deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "LazyArtifacts", "LibCURL_jll", "Libdl", "MPICH_jll", "MPIPreferences", "MPItrampoline_jll", "MicrosoftMPI_jll", "OpenMPI_jll", "OpenSSL_jll", "TOML", "Zlib_jll", "libaec_jll"] +git-tree-sha1 = "e94f84da9af7ce9c6be049e9067e511e17ff89ec" +uuid = "0234f1f7-429e-5d53-9886-15a909be8d59" +version = "1.14.6+0" + +[[deps.HTTP]] +deps = ["Base64", "CodecZlib", "ConcurrentUtilities", "Dates", "ExceptionUnwrapping", "Logging", "LoggingExtras", "MbedTLS", "NetworkOptions", "OpenSSL", "PrecompileTools", "Random", "SimpleBufferStream", "Sockets", "URIs", "UUIDs"] +git-tree-sha1 = "51059d23c8bb67911a2e6fd5130229113735fc7e" +uuid = "cd3eb016-35fb-5094-929b-558a96fad6f3" +version = "1.11.0" + +[[deps.Hwloc_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "XML2_jll", "Xorg_libpciaccess_jll"] +git-tree-sha1 = "baaaebd42ed9ee1bd9173cfd56910e55a8622ee1" +uuid = "e33a78d0-f292-5ffc-b300-72abe9b543c8" +version = "2.13.0+1" + +[[deps.InteractiveUtils]] +deps = ["Markdown"] +uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240" +version = "1.11.0" + +[[deps.JLLWrappers]] +deps = ["Artifacts", "Preferences"] +git-tree-sha1 = "0533e564aae234aff59ab625543145446d8b6ec2" +uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210" +version = "1.7.1" + +[[deps.JuliaSyntaxHighlighting]] +deps = ["StyledStrings"] +uuid = "ac6e5ff7-fb65-4e79-a425-ec3bc9c03011" +version = "1.12.0" + +[[deps.LRUCache]] +git-tree-sha1 = "5519b95a490ff5fe629c4a7aa3b3dfc9160498b3" +uuid = "8ac3fa9e-de4c-5943-b1dc-09c6b5f20637" +version = "1.6.2" +weakdeps = ["Serialization"] + + [deps.LRUCache.extensions] + SerializationExt = ["Serialization"] + +[[deps.LazyArtifacts]] +deps = ["Artifacts", "Pkg"] +uuid = "4af54fe1-eca0-43a8-85a7-787d91b784e3" +version = "1.11.0" + +[[deps.LibCURL]] +deps = ["LibCURL_jll", "MozillaCACerts_jll"] +uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21" +version = "0.6.4" + +[[deps.LibCURL_jll]] +deps = ["Artifacts", "LibSSH2_jll", "Libdl", "OpenSSL_jll", "Zlib_jll", "nghttp2_jll"] +uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0" +version = "8.15.0+0" + +[[deps.LibGit2]] +deps = ["LibGit2_jll", "NetworkOptions", "Printf", "SHA"] +uuid = "76f85450-5226-5b5a-8eaa-529ad045b433" +version = "1.11.0" + +[[deps.LibGit2_jll]] +deps = ["Artifacts", "LibSSH2_jll", "Libdl", "OpenSSL_jll"] +uuid = "e37daf67-58a4-590a-8e99-b0245dd2ffc5" +version = "1.9.0+0" + +[[deps.LibSSH2_jll]] +deps = ["Artifacts", "Libdl", "OpenSSL_jll"] +uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8" +version = "1.11.3+1" + +[[deps.Libdl]] +uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb" +version = "1.11.0" + +[[deps.Libiconv_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl"] +git-tree-sha1 = "be484f5c92fad0bd8acfef35fe017900b0b73809" +uuid = "94ce4f54-9a6c-5748-9c1c-f9c7231a4531" +version = "1.18.0+0" + +[[deps.LinearAlgebra]] +deps = ["Libdl", "OpenBLAS_jll", "libblastrampoline_jll"] +uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e" +version = "1.12.0" + +[[deps.Logging]] +uuid = "56ddb016-857b-54e1-b83d-db4d58db5568" +version = "1.11.0" + +[[deps.LoggingExtras]] +deps = ["Dates", "Logging"] +git-tree-sha1 = "f00544d95982ea270145636c181ceda21c4e2575" +uuid = "e6f89c97-d47a-5376-807f-9c37f3926c36" +version = "1.2.0" + +[[deps.Lz4_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl"] +git-tree-sha1 = "191686b1ac1ea9c89fc52e996ad15d1d241d1e33" +uuid = "5ced341a-0733-55b8-9ab6-a4889d929147" +version = "1.10.1+0" + +[[deps.MPICH_jll]] +deps = ["Artifacts", "CompilerSupportLibraries_jll", "Hwloc_jll", "JLLWrappers", "LazyArtifacts", "Libdl", "MPIPreferences", "TOML"] +git-tree-sha1 = "9341048b9f723f2ae2a72a5269ac2f15f80534dc" +uuid = "7cb0a576-ebde-5e09-9194-50597f1243b4" +version = "4.3.2+0" + +[[deps.MPIPreferences]] +deps = ["Libdl", "Preferences"] +git-tree-sha1 = "8e98d5d80b87403c311fd51e8455d4546ba7a5f8" +uuid = "3da0fdf6-3ccc-4f1b-acd9-58baa6c99267" +version = "0.1.12" + +[[deps.MPItrampoline_jll]] +deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "LazyArtifacts", "Libdl", "MPIPreferences", "TOML"] +git-tree-sha1 = "36c2d142e7d45fb98b5f83925213feb3292ca348" +uuid = "f1f71cc9-e9ae-5b93-9b94-4fe0e1ad3748" +version = "5.5.5+0" + +[[deps.Markdown]] +deps = ["Base64", "JuliaSyntaxHighlighting", "StyledStrings"] +uuid = "d6f4376e-aef5-505a-96c1-9c027394607a" +version = "1.11.0" + +[[deps.MbedTLS]] +deps = ["Dates", "MbedTLS_jll", "MozillaCACerts_jll", "NetworkOptions", "Random", "Sockets"] +git-tree-sha1 = "8785729fa736197687541f7053f6d8ab7fc44f92" +uuid = "739be429-bea8-5141-9913-cc70e7f3736d" +version = "1.1.10" + +[[deps.MbedTLS_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl"] +git-tree-sha1 = "ff69a2b1330bcb730b9ac1ab7dd680176f5896b8" +uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1" +version = "2.28.1010+0" + +[[deps.MicrosoftMPI_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "bc95bf4149bf535c09602e3acdf950d9b4376227" +uuid = "9237b28f-5490-5468-be7b-bb81f5f5e6cf" +version = "10.1.4+3" + +[[deps.Mmap]] +uuid = "a63ad114-7e13-5084-954f-fe012c677804" +version = "1.11.0" + +[[deps.MozillaCACerts_jll]] +uuid = "14a3606d-f60d-562e-9121-12d972cd8159" +version = "2025.11.4" + +[[deps.NCDatasets]] +deps = ["CFTime", "CommonDataModel", "DataStructures", "Dates", "DiskArrays", "NetCDF_jll", "NetworkOptions", "Printf"] +git-tree-sha1 = "791095699024db9dbb95bc48c0507cb0d1938369" +uuid = "85f8d34a-cbdd-5861-8df4-14fed0d494ab" +version = "0.14.14" + + [deps.NCDatasets.extensions] + NCDatasetsMPIExt = "MPI" + + [deps.NCDatasets.weakdeps] + MPI = "da04e1cc-30fd-572f-bb4f-1f8673147195" + +[[deps.NetCDF_jll]] +deps = ["Artifacts", "Blosc_jll", "Bzip2_jll", "HDF5_jll", "JLLWrappers", "LazyArtifacts", "LibCURL_jll", "Libdl", "MPICH_jll", "MPIPreferences", "MPItrampoline_jll", "MicrosoftMPI_jll", "OpenMPI_jll", "TOML", "XML2_jll", "Zlib_jll", "Zstd_jll", "libaec_jll", "libzip_jll"] +git-tree-sha1 = "d574803b6055116af212434460adf654ce98e345" +uuid = "7243133f-43d8-5620-bbf4-c2c921802cf3" +version = "401.900.300+0" + +[[deps.NetworkOptions]] +uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908" +version = "1.3.0" + +[[deps.OffsetArrays]] +git-tree-sha1 = "117432e406b5c023f665fa73dc26e79ec3630151" +uuid = "6fe1bfb0-de20-5000-8ca7-80f57d26f881" +version = "1.17.0" + + [deps.OffsetArrays.extensions] + OffsetArraysAdaptExt = "Adapt" + + [deps.OffsetArrays.weakdeps] + Adapt = "79e6a3ab-5dfb-504d-930d-738a2a938a0e" + +[[deps.OpenBLAS_jll]] +deps = ["Artifacts", "CompilerSupportLibraries_jll", "Libdl"] +uuid = "4536629a-c528-5b80-bd46-f80d51c5b363" +version = "0.3.29+0" + +[[deps.OpenMPI_jll]] +deps = ["Artifacts", "CompilerSupportLibraries_jll", "Hwloc_jll", "JLLWrappers", "LazyArtifacts", "Libdl", "MPIPreferences", "TOML", "Zlib_jll"] +git-tree-sha1 = "2f3d05e419b6125ffe06e55784102e99325bdbe2" +uuid = "fe0851c0-eecd-5654-98d4-656369965a5c" +version = "5.0.10+0" + +[[deps.OpenSSH_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "OpenSSL_jll", "Zlib_jll"] +git-tree-sha1 = "301412a644646fdc0ad67d0a87487466b491e53d" +uuid = "9bd350c2-7e96-507f-8002-3f2e150b4e1b" +version = "10.2.1+0" + +[[deps.OpenSSL]] +deps = ["BitFlags", "Dates", "MozillaCACerts_jll", "NetworkOptions", "OpenSSL_jll", "Sockets"] +git-tree-sha1 = "1d1aaa7d449b58415f97d2839c318b70ffb525a0" +uuid = "4d8831e6-92b7-49fb-bdf8-b643e874388c" +version = "1.6.1" + +[[deps.OpenSSL_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "458c3c95-2e84-50aa-8efc-19380b2a3a95" +version = "3.5.4+0" + +[[deps.OrderedCollections]] +git-tree-sha1 = "05868e21324cede2207c6f0f466b4bfef6d5e7ee" +uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d" +version = "1.8.1" + +[[deps.PCRE2_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "efcefdf7-47ab-520b-bdef-62a2eaa19f15" +version = "10.44.0+1" + +[[deps.Pkg]] +deps = ["Artifacts", "Dates", "Downloads", "FileWatching", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "Random", "SHA", "TOML", "Tar", "UUIDs", "p7zip_jll"] +uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f" +version = "1.12.1" +weakdeps = ["REPL"] + + [deps.Pkg.extensions] + REPLExt = "REPL" + +[[deps.PrecompileTools]] +deps = ["Preferences"] +git-tree-sha1 = "07a921781cab75691315adc645096ed5e370cb77" +uuid = "aea7be01-6a6a-4083-8856-8a6e6704d82a" +version = "1.3.3" + +[[deps.Preferences]] +deps = ["TOML"] +git-tree-sha1 = "8b770b60760d4451834fe79dd483e318eee709c4" +uuid = "21216c6a-2e73-6563-6e65-726566657250" +version = "1.5.2" + +[[deps.Printf]] +deps = ["Unicode"] +uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7" +version = "1.11.0" + +[[deps.ProgressLogging]] +deps = ["Logging", "SHA", "UUIDs"] +git-tree-sha1 = "f0803bc1171e455a04124affa9c21bba5ac4db32" +uuid = "33c8b6b6-d38a-422a-b730-caa89a2f386c" +version = "0.1.6" + +[[deps.REPL]] +deps = ["InteractiveUtils", "JuliaSyntaxHighlighting", "Markdown", "Sockets", "StyledStrings", "Unicode"] +uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb" +version = "1.11.0" + +[[deps.Random]] +deps = ["SHA"] +uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c" +version = "1.11.0" + +[[deps.SHA]] +uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce" +version = "0.7.0" + +[[deps.Serialization]] +uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b" +version = "1.11.0" + +[[deps.SimpleBufferStream]] +git-tree-sha1 = "f305871d2f381d21527c770d4788c06c097c9bc1" +uuid = "777ac1f9-54b0-4bf8-805c-2214025038e7" +version = "1.2.0" + +[[deps.Sockets]] +uuid = "6462fe0b-24de-5631-8697-dd941f90decc" +version = "1.11.0" + +[[deps.Statistics]] +deps = ["LinearAlgebra"] +git-tree-sha1 = "ae3bb1eb3bba077cd276bc5cfc337cc65c3075c0" +uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2" +version = "1.11.1" + + [deps.Statistics.extensions] + SparseArraysExt = ["SparseArrays"] + + [deps.Statistics.weakdeps] + SparseArrays = "2f01184e-e22b-5df5-ae63-d93ebab69eaf" + +[[deps.StyledStrings]] +uuid = "f489334b-da3d-4c2e-b8f0-e476e12c162b" +version = "1.11.0" + +[[deps.TOML]] +deps = ["Dates"] +uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76" +version = "1.0.3" + +[[deps.Tar]] +deps = ["ArgTools", "SHA"] +uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e" +version = "1.10.0" + +[[deps.Test]] +deps = ["InteractiveUtils", "Logging", "Random", "Serialization"] +uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40" +version = "1.11.0" + +[[deps.TranscodingStreams]] +git-tree-sha1 = "0c45878dcfdcfa8480052b6ab162cdd138781742" +uuid = "3bb67fe8-82b1-5028-8e26-92a6c54297fa" +version = "0.11.3" + +[[deps.URIs]] +git-tree-sha1 = "bef26fb046d031353ef97a82e3fdb6afe7f21b1a" +uuid = "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4" +version = "1.6.1" + +[[deps.UUIDs]] +deps = ["Random", "SHA"] +uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4" +version = "1.11.0" + +[[deps.Unicode]] +uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5" +version = "1.11.0" + +[[deps.XML2_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Libiconv_jll", "Zlib_jll"] +git-tree-sha1 = "80d3930c6347cfce7ccf96bd3bafdf079d9c0390" +uuid = "02c8fc9c-b97f-50b9-bbe4-9be30ff0a78a" +version = "2.13.9+0" + +[[deps.XZ_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl"] +git-tree-sha1 = "9cce64c0fdd1960b597ba7ecda2950b5ed957438" +uuid = "ffd25f8a-64ca-5728-b0f7-c24cf3aae800" +version = "5.8.2+0" + +[[deps.Xorg_libpciaccess_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Zlib_jll"] +git-tree-sha1 = "4909eb8f1cbf6bd4b1c30dd18b2ead9019ef2fad" +uuid = "a65dc6b1-eb27-53a1-bb3e-dea574b5389e" +version = "0.18.1+0" + +[[deps.Zlib_jll]] +deps = ["Libdl"] +uuid = "83775a58-1f1d-513f-b197-d71354ab007a" +version = "1.3.1+2" + +[[deps.Zstd_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl"] +git-tree-sha1 = "446b23e73536f84e8037f5dce465e92275f6a308" +uuid = "3161d3a3-bdf6-5164-811a-617609db77b4" +version = "1.5.7+1" + +[[deps.gh_cli_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl"] +git-tree-sha1 = "894b4ad6ac54832386f8d01e84cab691b588f364" +uuid = "5d31d589-30fb-542f-b82d-10325e863e38" +version = "2.83.2+0" + +[[deps.libaec_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl"] +git-tree-sha1 = "13b760f97c6e753b47df30cb438d4dc3b50df282" +uuid = "477f73a3-ac25-53e9-8cc3-50b2fa2566f0" +version = "1.1.5+0" + +[[deps.libblastrampoline_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "8e850b90-86db-534c-a0d3-1478176c7d93" +version = "5.15.0+0" + +[[deps.libzip_jll]] +deps = ["Artifacts", "Bzip2_jll", "JLLWrappers", "Libdl", "OpenSSL_jll", "XZ_jll", "Zlib_jll", "Zstd_jll"] +git-tree-sha1 = "86addc139bca85fdf9e7741e10977c45785727b7" +uuid = "337d8026-41b4-5cde-a456-74a10e5b31d1" +version = "1.11.3+0" + +[[deps.nghttp2_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d" +version = "1.64.0+1" + +[[deps.p7zip_jll]] +deps = ["Artifacts", "CompilerSupportLibraries_jll", "Libdl"] +uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0" +version = "17.7.0+0" diff --git a/soilgrids_ocd/OutputArtifacts.toml b/soilgrids_ocd/OutputArtifacts.toml new file mode 100644 index 0000000..a1db3c9 --- /dev/null +++ b/soilgrids_ocd/OutputArtifacts.toml @@ -0,0 +1,8 @@ +[soilgrids_ocd] +git-tree-sha1 = "39b86daa8a335c7efa894b2a8c38419e98e9b228" +[soilgrids_ocd_lowres_interp] +git-tree-sha1 = "11d53b5e1ff9031a5e94cfe4b93322edcb1a6129" + + [[soilgrids_ocd_lowres_interp.download]] + sha256 = "dec8ac8dfe23c088f7acdf790ce6fa8f958c6bfeb1e1d742a5904661f7facaef" + url = "https://caltech.box.com/shared/static/nr0bd7l18ka5fmvcmtwyj1t444vuonx8.gz" diff --git a/soilgrids_ocd/Project.toml b/soilgrids_ocd/Project.toml new file mode 100644 index 0000000..47f893f --- /dev/null +++ b/soilgrids_ocd/Project.toml @@ -0,0 +1,3 @@ +[deps] +ClimaArtifactsHelper = "6ffa2572-8378-4377-82eb-ea11db28b255" +NCDatasets = "85f8d34a-cbdd-5861-8df4-14fed0d494ab" diff --git a/soilgrids_ocd/README.md b/soilgrids_ocd/README.md new file mode 100644 index 0000000..3a74e6c --- /dev/null +++ b/soilgrids_ocd/README.md @@ -0,0 +1,45 @@ +# Soil Organic Carbon Density (OCD) from SoilGrids + +This artifact provides soil organic carbon density (OCD) from SoilGrids 2.0 at 6 standard depth +layers and ~5 km ans 1x1 degree resolution, matching the grid of the existing `soilgrids` artifact. + +## Variable + +| Variable | Description | Output units | +|----------|-------------|--------------| +| `ocd` | Soil organic carbon density | kg/m³ | + +## Depth layers + +Data is provided at the 6 SoilGrids standard depth layers: +`0-5 cm`, `5-15 cm`, `15-30 cm`, `30-60 cm`, `60-100 cm`, `100-200 cm` + +The depth coordinate `z` stores the midpoint of each layer in metres: +`[-0.025, -0.1, -0.225, -0.45, -0.8, -1.5]` + +## Processing steps + +1. **Download**: Run `download_soilgrids_ocd.sh` to retrieve OCD GeoTIFF files from SoilGrids + (requires `gdal_translate` and `gdalwarp`). +2. **Convert to NetCDF**: Run `transform_geotiff_to_netcdf.sh` to convert each `.tif` to `.nc`. +3. **Create artifact**: Run `julia --project create_artifacts.jl `. + This converts raw integer values (hg/m³) to SI units (kg/m³) and writes two NetCDF files: + - `soilgridsOCS/soil_ocd_soilgrids.nc` — full resolution (~5 km) + - `soilgridsOCS_lowres/soil_ocd_soilgrids_lowres.nc` — downsampled by area weighted averaging (~1 degree) + +## Unit conversion + +SoilGrids stores OCD as integers in units of hg/m³ (hectograms per cubic metre). +Conversion: `ocd_kg_m3 = raw_integer * 0.1`. +see https://www.isric.org/explore/soilgrids/faq-soilgrids if needed. + +## License + +The raw data is provided by SoilGrids under a Creative Commons BY 4.0 License. +See https://soilgrids.org for details. + +## References + +Poggio, L., de Sousa, L. M., Batjes, N. H., Heuvelink, G. B. M., Kempen, B., Ribeiro, E., and +Rossiter, D.: SoilGrids 2.0: producing soil information for the globe with quantified spatial +uncertainty, SOIL, 7, 217–240, 2021. DOI: https://doi.org/10.5194/soil-7-217-2021 diff --git a/soilgrids_ocd/combine_and_transform_raw_data.jl b/soilgrids_ocd/combine_and_transform_raw_data.jl new file mode 100644 index 0000000..cdf2840 --- /dev/null +++ b/soilgrids_ocd/combine_and_transform_raw_data.jl @@ -0,0 +1,33 @@ +# Combine the per-layer OCD files into a single file and convert units to SI. +# +# SoilGrids stores OCD (organic carbon density) as integers in units of hg/m³ +# (hectograms per cubic metre). Conversion to kg/m³: multiply by 0.1. +# Verify against https://www.isric.org/explore/soilgrids/faq-soilgrids if in doubt. + +z = [-1.5, -0.8, -0.45, -0.225, -0.1, -0.025] # depth of soil layer midpoints (m) +nlayers = length(z) +level_names = ["100-200cm","60-100cm","30-60cm","15-30cm","5-15cm","0-5cm"] + +attrib_ocd = (; + vartitle = "Soil organic carbon density", + varunits = "kg/m^3", + varname = "ocd", +) +transform_ocd(x) = x * 0.1 # hg/m³ -> kg/m³ + +# Open one file to get lat/lon dimensionality and pre-allocate memory +file = joinpath(filedir, "ocd_0-5cm_mean_5000.nc") +nc_data = NCDatasets.NCDataset(file) +lat = nc_data["lat"][:] +lon = nc_data["lon"][:] +nlat = length(lat) +nlon = length(lon) + +data = Array{Union{Missing, Float32}}(missing, nlon, nlat, nlayers) + +function create_combined_data!(data, files, attrib, transform, outfilepath) + read_nc_data!(data, files, filedir) + data[typeof.(data) .== Missing] .= Float32(NaN) + data .= transform(data) + write_nc_out(data, lat, lon, z, attrib, outfilepath) +end diff --git a/soilgrids_ocd/create_artifacts.jl b/soilgrids_ocd/create_artifacts.jl new file mode 100644 index 0000000..65be4da --- /dev/null +++ b/soilgrids_ocd/create_artifacts.jl @@ -0,0 +1,149 @@ +""" +Create the soilgridsOCS artifact: soil organic carbon density (OCD) from SoilGrids 2.0, +at 6 standard depth layers, with low-resolution output generated by 1x1 degree +block-mean aggregation. + +Usage: + julia --project create_artifacts_interp.jl + +Steps before running this script: + 1. Run download_soilgrids_ocd.sh (requires gdal_translate + gdalwarp) + 2. Run transform_geotiff_to_netcdf.sh + 3. Run this script with the path to the nc directory as argument +""" + +using NCDatasets +using ClimaArtifactsHelper + +if length(ARGS) < 1 + @error("Please provide the local path to the raw nc data directory (soilgridsOCD_nc).") +else + filedir = ARGS[1] +end + +outputdir = "soilgridsOCS" +outputdir_lowres = "soilgridsOCS_lowres" +for dir in [outputdir, outputdir_lowres] + if isdir(dir) + @warn "$dir already exists. Content will end up in the artifact and may be overwritten." + @warn "Abort this calculation, unless you know what you are doing." + else + mkdir(dir) + end +end + +include("utils.jl") +include("combine_and_transform_raw_data.jl") +# After including combine_and_transform_raw_data.jl the following are defined: +# nlayers, level_names, attrib_ocd, transform_ocd, data, lat, lon, nlat, nlon, z + +# Combine per-layer files into a single NetCDF +files = ["ocd_$(ln)_mean_5000.nc" for ln in level_names] +outfilepath_tmp = joinpath(outputdir, "ocd_soilgrids_combined.nc") +create_combined_data!(data, files, attrib_ocd, transform_ocd, outfilepath_tmp) + +replace_nan_with_zero(x) = isnan(x) ? typeof(x)(0) : x + +function degree_bin_groups(coord) + bins = floor.(Int, coord) + unique_bins = sort(unique(bins)) + groups = [findall(==(b), bins) for b in unique_bins] + centers = Float32.(unique_bins .+ 0.5) + return groups, centers +end + +function aggregate_1deg_mean(data, lon, lat) + lon_groups, lon_centers = degree_bin_groups(lon) + lat_groups, lat_centers = degree_bin_groups(lat) + nz = size(data, 3) + lat_weights = Float32.(max.(cosd.(lat), 0)) + + out = fill(Float32(NaN), length(lon_groups), length(lat_groups), nz) + + for k in 1:nz + for i in eachindex(lon_groups) + for j in eachindex(lat_groups) + block = @view data[lon_groups[i], lat_groups[j], k] + lat_idx = lat_groups[j] + block_weights = @view lat_weights[lat_idx] + weighted_sum = 0.0f0 + weight_sum = 0.0f0 + for ii in axes(block, 1) + for jj in axes(block, 2) + v = block[ii, jj] + w = block_weights[jj] + if !isnan(v) + weighted_sum += Float32(v) * w + weight_sum += w + end + end + end + if weight_sum > 0 + out[i, j, k] = weighted_sum / weight_sum + end + end + end + end + + return out, lon_centers, lat_centers +end + +# --- Full resolution output --- +ocd_ds = NCDataset(outfilepath_tmp) +ocd = ocd_ds["ocd"][:, :, :] +close(ocd_ds) + +outfilepath = joinpath(outputdir, "soil_ocd_soilgrids.nc") +ds = NCDataset(outfilepath, "c") +defDim(ds, "lon", nlon) +defDim(ds, "lat", nlat) +defDim(ds, "z", nlayers) +la = defVar(ds, "lat", Float32, ("lat",)) +lo = defVar(ds, "lon", Float32, ("lon",)) +zv = defVar(ds, "z", Float32, ("z",)) +la.attrib["units"] = "degrees_north" +la.attrib["standard_name"] = "latitude" +lo.attrib["standard_name"] = "longitude" +lo.attrib["units"] = "degrees_east" +zv.attrib["standard_name"] = "depth" +zv.attrib["units"] = "m" +la[:] = lat +lo[:] = lon +zv[:] = z +var = defVar(ds, "ocd", Float32, ("lon", "lat", "z")) +var.attrib["units"] = "kg/m^3" +var.attrib["longname"] = "Soil organic carbon density" +var[:, :, :] = replace_nan_with_zero.(ocd) +close(ds) + +# --- Low resolution output (~1 degree) --- +# Aggregate all source values that fall into the same 1x1 degree lon/lat bin. +# Calculate a latitude-area-weighted mean using cos(lat), ignoring NaNs. +# The output lon/lat coordinates are the centers of the bins. +ocd_lowres, lon_lowres, lat_lowres = aggregate_1deg_mean(ocd, lon, lat) + +outfilepath_lowres = joinpath(outputdir_lowres, "soil_ocd_soilgrids_lowres.nc") +ds = NCDataset(outfilepath_lowres, "c") +defDim(ds, "lon", length(lon_lowres)) +defDim(ds, "lat", length(lat_lowres)) +defDim(ds, "z", nlayers) +la = defVar(ds, "lat", Float32, ("lat",)) +lo = defVar(ds, "lon", Float32, ("lon",)) +zv = defVar(ds, "z", Float32, ("z",)) +la.attrib["units"] = "degrees_north" +la.attrib["standard_name"] = "latitude" +lo.attrib["standard_name"] = "longitude" +lo.attrib["units"] = "degrees_east" +zv.attrib["standard_name"] = "depth" +zv.attrib["units"] = "m" +la[:] = lat_lowres +lo[:] = lon_lowres +zv[:] = z +var = defVar(ds, "ocd", Float32, ("lon", "lat", "z")) +var.attrib["units"] = "kg/m^3" +var.attrib["longname"] = "Soil organic carbon density" +var[:, :, :] = replace_nan_with_zero.(ocd_lowres) +close(ds) + +create_artifact_guided(outputdir; artifact_name = basename(@__DIR__)) +create_artifact_guided(outputdir_lowres; artifact_name = basename(@__DIR__) * "_lowres_interp", append = true) diff --git a/soilgrids_ocd/download_soilgrids_ocd.sh b/soilgrids_ocd/download_soilgrids_ocd.sh new file mode 100644 index 0000000..37a9d57 --- /dev/null +++ b/soilgrids_ocd/download_soilgrids_ocd.sh @@ -0,0 +1,18 @@ +#!/bin/bash + +GEOTIFF_DATA_DIR="soilgridsOCD_geotiff" +mkdir -p "$GEOTIFF_DATA_DIR" +IGH="+proj=igh +lat_0=0 +lon_0=0 +datum=WGS84 +units=m +no_defs" # proj string for Homolosine projection +SG_URL="/vsicurl?max_retry=3&retry_delay=1&list_dir=no&url=https://files.isric.org/soilgrids/latest/data_aggregated/5000m" + +for level in "0-5cm" "5-15cm" "15-30cm" "30-60cm" "60-100cm" "100-200cm" +do + gdal_translate \ + -projwin_srs "$IGH" -co "TILED=YES" -co "COMPRESS=DEFLATE" -co "PREDICTOR=2" -co "BIGTIFF=YES" \ + $SG_URL"/ocd/ocd_"$level"_mean_5000.tif" \ + $GEOTIFF_DATA_DIR"/tmp.tif" + gdalwarp -overwrite -t_srs EPSG:4326 \ + $GEOTIFF_DATA_DIR"/tmp.tif" \ + $GEOTIFF_DATA_DIR"/ocd_"$level"_mean_5000.tif" +done +rm $GEOTIFF_DATA_DIR"/tmp.tif" diff --git a/soilgrids_ocd/transform_geotiff_to_netcdf.sh b/soilgrids_ocd/transform_geotiff_to_netcdf.sh new file mode 100644 index 0000000..b803996 --- /dev/null +++ b/soilgrids_ocd/transform_geotiff_to_netcdf.sh @@ -0,0 +1,24 @@ +#!/bin/bash + +# Define the source and destination directories specific to your local machine +SRC_DIR="soilgridsOCD_geotiff" +DEST_DIR="soilgridsOCD_nc" + +# Create the destination directory if it doesn't exist +mkdir -p "$DEST_DIR" + +# Loop over all GeoTIFF files in the source directory +for file in "$SRC_DIR"/*.tif; do + # Extract the base name of the file (without extension) + base_name=$(basename "$file" .tif) + + # Define the output NetCDF file name + output_file="$DEST_DIR/${base_name}.nc" + + # Convert the GeoTIFF file to NetCDF format + gdal_translate -ot Int16 -of netCDF "$file" "$output_file" + + echo "Converted $file to $output_file" +done + +echo "All files have been converted." diff --git a/soilgrids_ocd/utils.jl b/soilgrids_ocd/utils.jl new file mode 100644 index 0000000..24d95fb --- /dev/null +++ b/soilgrids_ocd/utils.jl @@ -0,0 +1,35 @@ +function read_nc_data!(data, files, filedir) + for i in 1:nlayers + @show(i) + filepath = joinpath(filedir, files[i]) + nc_data = NCDatasets.NCDataset(filepath) + data[:, :, i] .= nc_data["Band1"][:,:]; + end +end + +function write_nc_out(outdata, outlat, outlon, outz, attrib, outfilepath) + (vartitle, varunits, varname) = attrib + ds = NCDataset(outfilepath, "c") + defDim(ds, "lon", size(outdata)[1]) + defDim(ds, "lat", size(outdata)[2]) + defDim(ds, "z", nlayers) + ds.attrib["title"] = vartitle + + la = defVar(ds, "lat", Float32, ("lat",)) + lo = defVar(ds, "lon", Float32, ("lon",)) + zv = defVar(ds, "z", Float32, ("z",)) + var = defVar(ds, varname, Float32, ("lon", "lat", "z")) + var.attrib["units"] = varunits + la.attrib["units"] = "degrees_north" + la.attrib["standard_name"] = "latitude" + lo.attrib["standard_name"] = "longitude" + lo.attrib["units"] = "degrees_east" + zv.attrib["standard_name"] = "depth" + zv.attrib["units"] = "m" + + la[:] = outlat + lo[:] = outlon + zv[:] = z + var[:, :, :] = outdata + close(ds) +end