Skip to content

Commit 9e3e9bf

Browse files
vladmldvladmld
authored
Migrate job data source from sdkv2 to fw sign 30 (#380)
* Add data source for single job and required model and schema modifications. Also moved the test files. * Make model changes for the case where we only 1 single job is requested * Generate docs for testing incremental changes on PR * Read implementation * Cleaning things up, running tests. * Code cleanup * Add old fields to not introduce breaking changes. * Add doc changes * Changes to make data_source job identical to the sdkv2 version. * Add deprectated message attribute * Changes to deprectation to be more visible and remove redundant fields.
1 parent de22f31 commit 9e3e9bf

11 files changed

Lines changed: 523 additions & 418 deletions

File tree

‎docs/data-sources/job.md‎

Lines changed: 76 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,12 @@
33
page_title: "dbtcloud_job Data Source - dbtcloud"
44
subcategory: ""
55
description: |-
6-
6+
Get detailed information for a specific dbt Cloud job.
77
---
88

99
# dbtcloud_job (Data Source)
1010

11-
11+
Get detailed information for a specific dbt Cloud job.
1212

1313

1414

@@ -17,29 +17,89 @@ description: |-
1717

1818
### Required
1919

20-
- `job_id` (Number) ID of the job
21-
- `project_id` (Number) ID of the project the job is in
20+
- `job_id` (Number) The ID of the job
21+
22+
### Optional
23+
24+
- `job_completion_trigger_condition` (Attributes List) Which other job should trigger this job when it finishes, and on which conditions. Format for the property will change in the next release to match the one from the one from dbtcloud_jobs. (see [below for nested schema](#nestedatt--job_completion_trigger_condition))
2225

2326
### Read-Only
2427

25-
- `deferring_environment_id` (Number) ID of the environment this job defers to
26-
- `deferring_job_id` (Number) ID of the job this job defers to
27-
- `description` (String) Long description for the job
28-
- `environment_id` (Number) ID of the environment the job is in
29-
- `id` (String) The ID of this resource.
30-
- `job_completion_trigger_condition` (Set of Object) Which other job should trigger this job when it finishes, and on which conditions. (see [below for nested schema](#nestedatt--job_completion_trigger_condition))
31-
- `name` (String) Given name for the job
32-
- `run_compare_changes` (Boolean) Whether the CI job should compare data changes introduced by the code change in the PR.
28+
- `dbt_version` (String) The version of dbt used for the job. If not set, the environment version will be used.
29+
- `deferring_environment_id` (Number) The ID of the environment this job defers to
30+
- `deferring_job_id` (Number, Deprecated) [Deprectated - Deferral is now set at the environment level] The ID of the job definition this job defers to
31+
- `description` (String) The description of the job
32+
- `environment` (Attributes) Details of the environment the job is running in (see [below for nested schema](#nestedatt--environment))
33+
- `environment_id` (Number) The ID of environment
34+
- `execute_steps` (List of String) The list of steps to run in the job
35+
- `execution` (Attributes) (see [below for nested schema](#nestedatt--execution))
36+
- `generate_docs` (Boolean) Whether the job generate docs
37+
- `id` (Number) The ID of the job
38+
- `job_type` (String) The type of job (e.g. CI, scheduled)
39+
- `name` (String) The name of the job
40+
- `project_id` (Number) The ID of the project
41+
- `run_compare_changes` (Boolean) Whether the job should compare data changes introduced by the code change in the PR
42+
- `run_generate_sources` (Boolean) Whether the job test source freshness
43+
- `schedule` (Attributes) (see [below for nested schema](#nestedatt--schedule))
3344
- `self_deferring` (Boolean) Whether this job defers on a previous run of itself (overrides value in deferring_job_id)
34-
- `timeout_seconds` (Number) Number of seconds before the job times out
35-
- `triggers` (Map of Boolean) Flags for which types of triggers to use, keys of github_webhook, git_provider_webhook, schedule, on_merge
45+
- `settings` (Attributes) (see [below for nested schema](#nestedatt--settings))
46+
- `timeout_seconds` (Number, Deprecated) [Deprectated - Moved to execution.timeout_seconds] Number of seconds before the job times out
47+
- `triggers` (Attributes) (see [below for nested schema](#nestedatt--triggers))
3648
- `triggers_on_draft_pr` (Boolean) Whether the CI job should be automatically triggered on draft PRs
3749

3850
<a id="nestedatt--job_completion_trigger_condition"></a>
3951
### Nested Schema for `job_completion_trigger_condition`
4052

4153
Read-Only:
4254

43-
- `job_id` (Number)
55+
- `job_id` (Number) The ID of the job that would trigger this job after completion.
56+
- `project_id` (Number) The ID of the project where the trigger job is running in.
57+
- `statuses` (Set of String) List of statuses to trigger the job on.
58+
59+
60+
<a id="nestedatt--environment"></a>
61+
### Nested Schema for `environment`
62+
63+
Read-Only:
64+
65+
- `deployment_type` (String) Type of deployment environment: staging, production
66+
- `id` (Number) ID of the environment
67+
- `name` (String) Name of the environment
4468
- `project_id` (Number)
45-
- `statuses` (Set of String)
69+
- `type` (String) Environment type: development or deployment
70+
71+
72+
<a id="nestedatt--execution"></a>
73+
### Nested Schema for `execution`
74+
75+
Read-Only:
76+
77+
- `timeout_seconds` (Number) The number of seconds before the job times out
78+
79+
80+
<a id="nestedatt--schedule"></a>
81+
### Nested Schema for `schedule`
82+
83+
Read-Only:
84+
85+
- `cron` (String) The cron schedule for the job. Only used if triggers.schedule is true
86+
87+
88+
<a id="nestedatt--settings"></a>
89+
### Nested Schema for `settings`
90+
91+
Read-Only:
92+
93+
- `target_name` (String) Value for `target.name` in the Jinja context
94+
- `threads` (Number) Number of threads to run dbt with
95+
96+
97+
<a id="nestedatt--triggers"></a>
98+
### Nested Schema for `triggers`
99+
100+
Read-Only:
101+
102+
- `git_provider_webhook` (Boolean) Whether the job runs automatically on PR creation
103+
- `github_webhook` (Boolean) Whether the job runs automatically on PR creation
104+
- `on_merge` (Boolean) Whether the job runs automatically once a PR is merged
105+
- `schedule` (Boolean) Whether the job runs on a schedule

‎docs/data-sources/jobs.md‎

Lines changed: 25 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -45,29 +45,52 @@ locals {
4545
<a id="nestedatt--jobs"></a>
4646
### Nested Schema for `jobs`
4747

48+
Optional:
49+
50+
- `job_completion_trigger_condition` (Attributes) Whether the job is triggered by the completion of another job (see [below for nested schema](#nestedatt--jobs--job_completion_trigger_condition))
51+
4852
Read-Only:
4953

5054
- `dbt_version` (String) The version of dbt used for the job. If not set, the environment version will be used.
5155
- `deferring_environment_id` (Number) The ID of the environment this job defers to
52-
- `deferring_job_definition_id` (Number) [Deprecated - deferral is now set at the environment level] The ID of the job definition this job defers to
56+
- `deferring_job_definition_id` (Number, Deprecated) [Deprectated - Deferral is now set at the environment level] The ID of the job definition this job defers to
5357
- `description` (String) The description of the job
5458
- `environment` (Attributes) Details of the environment the job is running in (see [below for nested schema](#nestedatt--jobs--environment))
5559
- `environment_id` (Number) The ID of environment
5660
- `execute_steps` (List of String) The list of steps to run in the job
5761
- `execution` (Attributes) (see [below for nested schema](#nestedatt--jobs--execution))
5862
- `generate_docs` (Boolean) Whether the job generate docs
5963
- `id` (Number) The ID of the job
60-
- `job_completion_trigger_condition` (Attributes) Whether the job is triggered by the completion of another job (see [below for nested schema](#nestedatt--jobs--job_completion_trigger_condition))
64+
- `job_id` (Number) The ID of the job
6165
- `job_type` (String) The type of job (e.g. CI, scheduled)
6266
- `name` (String) The name of the job
6367
- `project_id` (Number) The ID of the project
6468
- `run_compare_changes` (Boolean) Whether the job should compare data changes introduced by the code change in the PR
6569
- `run_generate_sources` (Boolean) Whether the job test source freshness
6670
- `schedule` (Attributes) (see [below for nested schema](#nestedatt--jobs--schedule))
6771
- `settings` (Attributes) (see [below for nested schema](#nestedatt--jobs--settings))
72+
- `timeout_seconds` (Number, Deprecated) [Deprectated - Moved to execution.timeout_seconds] Number of seconds before the job times out
6873
- `triggers` (Attributes) (see [below for nested schema](#nestedatt--jobs--triggers))
6974
- `triggers_on_draft_pr` (Boolean) Whether the CI job should be automatically triggered on draft PRs
7075

76+
<a id="nestedatt--jobs--job_completion_trigger_condition"></a>
77+
### Nested Schema for `jobs.job_completion_trigger_condition`
78+
79+
Read-Only:
80+
81+
- `condition` (Attributes) (see [below for nested schema](#nestedatt--jobs--job_completion_trigger_condition--condition))
82+
83+
<a id="nestedatt--jobs--job_completion_trigger_condition--condition"></a>
84+
### Nested Schema for `jobs.job_completion_trigger_condition.condition`
85+
86+
Read-Only:
87+
88+
- `job_id` (Number)
89+
- `project_id` (Number)
90+
- `statuses` (Set of String)
91+
92+
93+
7194
<a id="nestedatt--jobs--environment"></a>
7295
### Nested Schema for `jobs.environment`
7396

@@ -88,24 +111,6 @@ Read-Only:
88111
- `timeout_seconds` (Number) The number of seconds before the job times out
89112

90113

91-
<a id="nestedatt--jobs--job_completion_trigger_condition"></a>
92-
### Nested Schema for `jobs.job_completion_trigger_condition`
93-
94-
Read-Only:
95-
96-
- `condition` (Attributes) (see [below for nested schema](#nestedatt--jobs--job_completion_trigger_condition--condition))
97-
98-
<a id="nestedatt--jobs--job_completion_trigger_condition--condition"></a>
99-
### Nested Schema for `jobs.job_completion_trigger_condition.condition`
100-
101-
Read-Only:
102-
103-
- `job_id` (Number)
104-
- `project_id` (Number)
105-
- `statuses` (Set of String)
106-
107-
108-
109114
<a id="nestedatt--jobs--schedule"></a>
110115
### Nested Schema for `jobs.schedule`
111116

‎pkg/framework/objects/job/data_source.go‎

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
package job
2+
3+
import (
4+
"context"
5+
"fmt"
6+
"strconv"
7+
8+
"github.com/dbt-labs/terraform-provider-dbtcloud/pkg/dbt_cloud"
9+
"github.com/dbt-labs/terraform-provider-dbtcloud/pkg/helper"
10+
"github.com/dbt-labs/terraform-provider-dbtcloud/pkg/utils"
11+
"github.com/hashicorp/terraform-plugin-framework/datasource"
12+
"github.com/hashicorp/terraform-plugin-framework/path"
13+
"github.com/hashicorp/terraform-plugin-framework/types"
14+
"github.com/hashicorp/terraform-plugin-log/tflog"
15+
"github.com/samber/lo"
16+
)
17+
18+
var (
19+
_ datasource.DataSource = &jobDataSource{}
20+
_ datasource.DataSourceWithConfigure = &jobDataSource{}
21+
_ datasource.DataSourceWithValidateConfig = &jobDataSource{}
22+
)
23+
24+
func JobDataSource() datasource.DataSource {
25+
return &jobDataSource{}
26+
}
27+
28+
type jobDataSource struct {
29+
client *dbt_cloud.Client
30+
}
31+
32+
func (j *jobDataSource) Configure(_ context.Context, req datasource.ConfigureRequest, resp *datasource.ConfigureResponse) {
33+
tflog.Debug(context.Background(), "Configuring dbt Cloud job data source")
34+
35+
if req.ProviderData == nil {
36+
return
37+
}
38+
39+
j.client = req.ProviderData.(*dbt_cloud.Client)
40+
tflog.Debug(context.Background(), "Configured dbt Cloud job data source")
41+
}
42+
43+
func (j *jobDataSource) Metadata(_ context.Context, req datasource.MetadataRequest, resp *datasource.MetadataResponse) {
44+
resp.TypeName = req.ProviderTypeName + "_job"
45+
}
46+
47+
func (j *jobDataSource) ValidateConfig(
48+
ctx context.Context,
49+
req datasource.ValidateConfigRequest,
50+
resp *datasource.ValidateConfigResponse,
51+
) {
52+
var data SingleJobDataSourceModel
53+
54+
resp.Diagnostics.Append(req.Config.Get(ctx, &data)...)
55+
56+
if resp.Diagnostics.HasError() {
57+
return
58+
}
59+
60+
if data.JobId.IsNull() {
61+
resp.Diagnostics.AddAttributeError(
62+
path.Root("job_id"),
63+
"Missing Required Attribute",
64+
"job_id must be configured.",
65+
)
66+
}
67+
}
68+
69+
func (j *jobDataSource) Read(ctx context.Context, req datasource.ReadRequest, resp *datasource.ReadResponse) {
70+
71+
var state SingleJobDataSourceModel
72+
73+
resp.Diagnostics.Append(req.Config.Get(ctx, &state)...)
74+
75+
jobIdValue := state.JobId.ValueInt64()
76+
77+
jobId := strconv.FormatInt(jobIdValue, 10)
78+
79+
job, err := j.client.GetJob(jobId)
80+
81+
if err != nil {
82+
tflog.Error(ctx, fmt.Sprintf("Error getting job: %s", err.Error()))
83+
resp.Diagnostics.AddError("Error getting the job", err.Error())
84+
return
85+
}
86+
87+
state.Execution = &JobExecution{
88+
TimeoutSeconds: types.Int64Value(int64(job.Execution.TimeoutSeconds)),
89+
}
90+
state.TimeoutSeconds = types.Int64Value(int64(job.Execution.TimeoutSeconds))
91+
state.GenerateDocs = types.BoolValue(job.GenerateDocs)
92+
state.RunGenerateSources = types.BoolValue(job.RunGenerateSources)
93+
state.ID = types.Int64PointerValue(helper.IntPointerToInt64Pointer(job.ID))
94+
state.ProjectID = types.Int64Value(int64(job.ProjectId))
95+
state.EnvironmentID = types.Int64Value(int64(job.EnvironmentId))
96+
state.Name = types.StringValue(job.Name)
97+
state.Description = types.StringValue(job.Description)
98+
state.DbtVersion = types.StringPointerValue(job.DbtVersion)
99+
state.ExecuteSteps = helper.SliceStringToSliceTypesString(job.ExecuteSteps)
100+
state.DeferringJobId = types.Int64PointerValue(helper.IntPointerToInt64Pointer(job.DeferringJobId))
101+
state.SelfDeferring = types.BoolValue(job.DeferringJobId != nil && *job.DeferringJobId == *job.ID)
102+
state.DeferringEnvironmentID = types.Int64PointerValue(helper.IntPointerToInt64Pointer(job.DeferringEnvironmentId))
103+
state.Triggers = &JobTriggers{
104+
GithubWebhook: types.BoolValue(job.Triggers.GithubWebhook),
105+
GitProviderWebhook: types.BoolValue(job.Triggers.GitProviderWebhook),
106+
Schedule: types.BoolValue(job.Triggers.Schedule),
107+
OnMerge: types.BoolValue(job.Triggers.OnMerge),
108+
}
109+
state.Settings = &JobSettings{
110+
Threads: types.Int64Value(int64(job.Settings.Threads)),
111+
TargetName: types.StringValue(job.Settings.TargetName),
112+
}
113+
state.Schedule = &JobSchedule{
114+
Cron: types.StringValue(job.Schedule.Cron),
115+
}
116+
state.JobType = types.StringValue(job.JobType)
117+
state.TriggersOnDraftPr = types.BoolValue(job.TriggersOnDraftPR)
118+
state.RunCompareChanges = types.BoolValue(job.RunCompareChanges)
119+
120+
if job.JobCompletionTrigger != nil {
121+
state.JobCompletionTriggerCondition = []*JobCompletionTriggerCondition{
122+
{
123+
JobID: types.Int64Value(int64(job.JobCompletionTrigger.Condition.JobID)),
124+
ProjectID: types.Int64Value(int64(job.JobCompletionTrigger.Condition.ProjectID)),
125+
Statuses: lo.Map(
126+
job.JobCompletionTrigger.Condition.Statuses,
127+
func(status int, _ int) types.String {
128+
return types.StringValue(
129+
utils.JobCompletionTriggerConditionsMappingCodeHuman[status].(string),
130+
)
131+
},
132+
),
133+
},
134+
}
135+
}
136+
137+
diags := resp.State.Set(ctx, &state)
138+
resp.Diagnostics.Append(diags...)
139+
if resp.Diagnostics.HasError() {
140+
return
141+
}
142+
}

pkg/sdkv2/data_sources/job_acceptance_test.go renamed to pkg/framework/objects/job/data_source_acceptance_test.go

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
1-
package data_sources_test
1+
package job_test
22

33
import (
44
"fmt"
55
"testing"
66

7+
"github.com/dbt-labs/terraform-provider-dbtcloud/pkg/framework/acctest_config"
78
"github.com/dbt-labs/terraform-provider-dbtcloud/pkg/framework/acctest_helper"
89
"github.com/hashicorp/terraform-plugin-testing/helper/acctest"
910
"github.com/hashicorp/terraform-plugin-testing/helper/resource"
@@ -13,13 +14,14 @@ func TestDbtCloudJobDataSource(t *testing.T) {
1314

1415
randomJobName := acctest.RandStringFromCharSet(5, acctest.CharSetAlphaNum)
1516

16-
config := jobs(randomJobName)
17+
config := jobResourceConfig(randomJobName)
1718

1819
check := resource.ComposeAggregateTestCheckFunc(
1920
resource.TestCheckResourceAttrSet("data.dbtcloud_job.test", "job_id"),
2021
resource.TestCheckResourceAttrSet("data.dbtcloud_job.test", "project_id"),
2122
resource.TestCheckResourceAttrSet("data.dbtcloud_job.test", "environment_id"),
2223
resource.TestCheckResourceAttr("data.dbtcloud_job.test", "name", randomJobName),
24+
resource.TestCheckResourceAttr("data.dbtcloud_job.test", "execution.timeout_seconds", "180"),
2325
resource.TestCheckResourceAttr("data.dbtcloud_job.test", "timeout_seconds", "180"),
2426
resource.TestCheckResourceAttr("data.dbtcloud_job.test", "triggers_on_draft_pr", "false"),
2527
resource.TestCheckResourceAttr(
@@ -40,7 +42,7 @@ func TestDbtCloudJobDataSource(t *testing.T) {
4042
})
4143
}
4244

43-
func jobs(jobName string) string {
45+
func jobResourceConfig(jobName string) string {
4446
return fmt.Sprintf(`
4547
resource "dbtcloud_project" "test_project" {
4648
name = "jobs_test_project"
@@ -70,7 +72,6 @@ func jobs(jobName string) string {
7072
7173
data "dbtcloud_job" "test" {
7274
job_id = dbtcloud_job.test_job.id
73-
project_id = dbtcloud_project.test_project.id
7475
}
75-
`, DBT_CLOUD_VERSION, jobName)
76+
`, acctest_config.DBT_CLOUD_VERSION, jobName)
7677
}

0 commit comments

Comments
 (0)