Skip to content

Confluence is a cloud-based workflow that produces river discharge estimates and suspended sediment concentration data using SWOT observations and HLS. This is the top-level repo that includes Terraform that deploys shared AWS infrastructure for the Confluence workflow.

License

Notifications You must be signed in to change notification settings

podaac/confluence

Repository files navigation

confluence

components

Confluence Component Data Flow Diagram

aws infrastructure

workflow-infrastructure

The Confluence workflow includes the following AWS service in the workflow-infrastructure directory

  • AWS Batch compute environment, job queue definitions.
  • AWS CloudWatch Logs.
  • AWS EFS.
  • AWS IAM Roles & Policies.
  • AWS S3 SoS Bucket.
  • AWS Security Groups and VPC definition.
  • AWS System Manager Parameter Store and Key Management Service.

stepfunction-infrastructure

The Confluence workflow includes the following AWS service in the stepfunction-infrastructure directory

  • AWS EventBridge rule.
  • AWS Step Function state machine.
  • AWS IAM Roles & Policies.

terraform

Deploys AWS infrastructure and stores state in an S3 backend. This top-level terraform repo contains AWS infrastructure that applies to all components. Each component may have additional terraform files for deploying AWS resources, see each components README.md for details.

To deploy:

  1. Initialize terraform: terraform init -reconfigure -backend-config="bucket=confluence-<env>-tf-state" -backend-config="key=confluence.tfstate" -backend-config="region=us-west-2" -backend-config="profile=<named_profile>"
  2. Plan terraform modifications: terraform plan -var-file="conf.tfvars" -out="tfplan"
  3. Apply terraform modifications: terraform apply tfplan

<env> can be dev1, dev2, dev3 or ops

<named_profile> is the name of the profile used to authenticate to AWS

deployment script

Script to deploy Terraform AWS infrastructure

REQUIRES:

Command line arguments:

[1] s3_state_bucket: Name of the S3 bucket to store Terraform state in (no need for s3:// prefix)

[2] terraform_state_key: Name of Terraform state key (e.g. confluence.tfstate or confluence-sfn.tfstate)

[3] terraform_directory: Which directory to deploy (e.g. workflow-infrastructure)

Infrastructure example usage: deploy/deploy.sh confluence-bucket-tf-state confluence.tfstate workflow-infrastructure

Step Function example usage: deploy/deploy.sh confluence-bucket-tf-state confluence-sfn.tfstate workflow-step-function

About

Confluence is a cloud-based workflow that produces river discharge estimates and suspended sediment concentration data using SWOT observations and HLS. This is the top-level repo that includes Terraform that deploys shared AWS infrastructure for the Confluence workflow.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published