-
Notifications
You must be signed in to change notification settings - Fork 0
Add tags or labels to all resources created with shared modules #9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 2 commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -221,6 +221,7 @@ module "data_processor" { | |
| - `main_file` - Python file name (e.g., "main.py"), relative to `source_dir`. | ||
| - `schedule` - Cron expression (e.g., "0 9 * * 1-5") | ||
| - `description` - Function/job description | ||
| - `owner` - The owner/team responsible for this scheduled job | ||
|
|
||
| ### Optional (with defaults) | ||
| - `execution_type` - "function" or "job" ("function") | ||
|
|
@@ -230,6 +231,7 @@ module "data_processor" { | |
| - `timeout_seconds` - Timeout for functions (60) | ||
| - `environment_variables` - Environment vars ({}) | ||
| - `secrets` - Secret Manager secrets ([]) | ||
| - `tags` - A map of tags to assign to all resources ({}) | ||
jwbron marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ### Cloud Run Job specific (when `execution_type = "job"`) | ||
| - `job_cpu` - CPU allocation (e.g., "1000m", "2") ("1000m") | ||
|
|
@@ -389,6 +391,42 @@ Or use Cloud Build directly: | |
| gcloud builds submit --tag gcr.io/YOUR_PROJECT_ID/YOUR_JOB_NAME:latest ./jobs/your-job | ||
| ``` | ||
|
|
||
| ## Resource Tagging | ||
|
|
||
| All resources created by this module are automatically tagged with common metadata: | ||
|
|
||
| ### Automatic Tags | ||
| - `terraform_module` - Set to "scheduled-job" | ||
| - `scheduled_job_name` - The name of your function/job | ||
| - `owner` - The owner/team responsible for this scheduled job | ||
|
|
||
| ### Custom Tags | ||
| You can add custom tags using the `tags` variable: | ||
|
|
||
| ```hcl | ||
| module "my_function" { | ||
| source = "git::https://github.com/Khan/terraform-modules.git//terraform/modules/scheduled-job?ref=v1.0.0" | ||
|
|
||
| job_name = "my-function" | ||
| owner = "data-team" | ||
| # ... other configuration | ||
|
|
||
| tags = { | ||
| "environment" = "production" | ||
| "team" = "data-engineering" | ||
| "cost-center" = "infrastructure" | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### Supported Resources | ||
| The following resources support tagging/labeling: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What happens if you try to add tags somewhere else? It seems like it's just silently ignored?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If you add a field not supported in a terraform resource (in this case |
||
| - **Storage Buckets** - Labels applied | ||
| - **Storage Objects** - Metadata applied | ||
| - **PubSub Topics** - Labels applied | ||
| - **Cloud Functions** - Labels applied | ||
| - **Cloud Run Jobs** - Labels applied | ||
|
|
||
| ## Common Cron Patterns | ||
|
|
||
| | Schedule | Description | | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -17,6 +17,15 @@ terraform { | |
| } | ||
| } | ||
|
|
||
| # Common tags for all resources | ||
| locals { | ||
| common_tags = merge(var.tags, { | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This seems misnamed -- I'd expect Also, who wins in case of a merge conflict? I'm guessing that the ones below take precedence over the ones in
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this means "common for all resources generated by this usage of this module". Does "all_tags" sound better to you? Good call on the merge conflicts, I'll make sure your guess is correct and document it.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| "terraform_module" = "scheduled-job" | ||
| "scheduled_job_name" = var.job_name | ||
| "owner" = var.owner | ||
| }) | ||
| } | ||
|
|
||
| # Service account for the Cloud Function/Job | ||
| resource "google_service_account" "function_sa" { | ||
| project = var.project_id | ||
|
|
@@ -34,6 +43,8 @@ resource "google_storage_bucket" "function_bucket" { | |
| location = var.region | ||
| uniform_bucket_level_access = true | ||
| force_destroy = true | ||
|
|
||
| labels = local.common_tags | ||
| } | ||
|
|
||
| # Create function source archive (only for Cloud Functions) | ||
|
|
@@ -57,6 +68,8 @@ resource "google_storage_bucket_object" "function_archive" { | |
| name = "${var.job_name}-function-${data.archive_file.function_archive[0].output_sha}.zip" | ||
| bucket = google_storage_bucket.function_bucket[0].name | ||
| source = data.archive_file.function_archive[0].output_path | ||
|
|
||
| metadata = local.common_tags | ||
| } | ||
|
|
||
| # PubSub topic for triggering the Cloud Function (only created when execution_type is "function") | ||
|
|
@@ -65,6 +78,8 @@ resource "google_pubsub_topic" "function_topic" { | |
|
|
||
| project = var.project_id | ||
| name = "${var.job_name}-topic" | ||
|
|
||
| labels = local.common_tags | ||
| } | ||
|
|
||
| # Cloud Scheduler job for Cloud Function (only created when execution_type is "function") | ||
|
|
@@ -104,6 +119,8 @@ resource "google_cloudfunctions2_function" "function" { | |
| description = var.description | ||
| location = var.region | ||
|
|
||
| labels = local.common_tags | ||
|
|
||
| build_config { | ||
| runtime = var.runtime | ||
| entry_point = var.entry_point | ||
|
|
@@ -159,6 +176,7 @@ resource "google_cloud_run_v2_job" "job" { | |
| project = var.project_id | ||
| name = var.job_name | ||
| location = var.region | ||
| labels = local.common_tags | ||
|
|
||
| lifecycle { | ||
| precondition { | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice! Do we want to use some consistent way of naming the team? Like should it be a github tag (
@owneror#team), or the list of teams in ownership_data.json, or something else?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I should probably add a predefined list of teams here.