Home
A total of 165Gb of environmental data has been compiled for use in environmental assessment of Cockburn Sound.
The goal of the CSIEM environmental data management repository, as presented herein, is to allow compatibility, inter-operability and comparison of these critical data assets. Version control and data governance is required to ensure a comprehensive and integrated modelling platform.
This data repository is based around the following structure:
csiem-data/
171M ./code
18M ./data-governance
31M ./data-mapping
165M ./summary-images
98G ./data-lake ! Raw data not included in this GitHub repository : see access for further information.
65G ./data-warehouse. ! Ingested (standardised) data not included in this repository : see access for further information.
TOTAL = 165G
The repository is built around a framework that brings together three separate steps in the data "federation" process:
- Data Collation
- Data Governance & Reporting
- Data Integration
The relationship between the various iniatives, the CSIEM environmental data management framework, and downstream model applications are outlined schematically in the below image.
The aim of the data collation step is to bring data together in a co-ordinated way. Data that is sourced and collated from various government agencies, researchers and industry groups is stored in a “data lake” in their raw format. Each data provider is assigned a unique agency identifier, and datasets are also grouped based on the main programs or iniatives the collection was associated with. Raw data is stored in a rigid folder structure based on these two identifiers :
Agency/Program/ < ... data-sets ...>
For more information, see the The Cockburn Sound Integrated Ecosystem Model Manual
Aquatic EcoDynamics
