Skip to content

Commit b5aa419

Browse files
fivetran-data-model-botfivetran-catfritzfivetran-joemarkiewiczgithub-actions[bot]fivetran-jamie
authored
Consolidate source + remove tests (#145)
* Consolidate dbt source into transform * Update issue.csv (#141) * Q2 FY26 Automatic Package Updates (#142) * Q2 FY26: Apply automated update. * Q2 FY26: Update auto-release workflow only. --------- Co-authored-by: fivetran-catfritz <111930712+fivetran-catfritz@users.noreply.github.com> * fixes * Generate dbt docs via GitHub Actions * Update README.md * Update README.md * Update CHANGELOG.md Co-authored-by: Jamie Rodriguez <65564846+fivetran-jamie@users.noreply.github.com> --------- Co-authored-by: fivetran-catfritz <111930712+fivetran-catfritz@users.noreply.github.com> Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jamie Rodriguez <65564846+fivetran-jamie@users.noreply.github.com>
1 parent 301e706 commit b5aa419

80 files changed

Lines changed: 1784 additions & 101 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/PULL_REQUEST_TEMPLATE/maintainer_pull_request_template.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,13 +12,15 @@ Before marking this PR as "ready for review":
1212

1313
## PR Overview
1414
**Package version introduced in this PR:**
15-
15+
-
16+
1617
**This PR addresses the following Issue/Feature(s):**
1718
<!-- Add Issue # or internal ticket reference -->
19+
-
1820

1921
**Summary of changes:**
2022
<!-- 1-2 sentences describing PR changes. -->
21-
23+
-
2224

2325
### Submission Checklist
2426
- [ ] Alignment meeting with the reviewer (if needed)
@@ -27,9 +29,10 @@ Before marking this PR as "ready for review":
2729
- [ ] **Validation Steps:** Check for unintentional effects (e.g., add/run consistency & integrity tests)
2830
- [ ] **Testing Instructions:** Confirm the change addresses the issue(s)
2931
- [ ] **Focus Areas:** Complex logic or queries that need extra attention
32+
- [ ] Merge any relevant open PRs into this PR
3033

3134
### Changelog
3235
<!-- Recommend drafting changelog notes, then refining via ChatGPT using:
3336
"Draft a changelog entry based on the following notes." -->
3437
- [ ] Draft changelog for PR
35-
- [ ] Final changelog for release review
38+
- [ ] Final changelog for release review

.github/workflows/auto-release.yml

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,12 @@ on:
33
pull_request:
44
types:
55
- closed
6-
branches:
7-
- main
6+
- labeled
87

98
jobs:
10-
call-workflow-passing-data:
11-
if: github.event.pull_request.merged
9+
release:
10+
if: |
11+
(github.event.pull_request.merged == true && github.event.pull_request.base.ref == 'main') ||
12+
github.event.label.name == 'pre-release'
1213
uses: fivetran/dbt_package_automations/.github/workflows/auto-release.yml@main
13-
secrets: inherit
14+
secrets: inherit
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
name: 'generate dbt docs'
2+
on:
3+
pull_request:
4+
types:
5+
- labeled
6+
7+
jobs:
8+
generate-docs:
9+
if: github.event.label.name == 'docs:ready'
10+
uses: fivetran/dbt_package_automations/.github/workflows/generate-docs.yml@main
11+
secrets: inherit
12+
with:
13+
schema_var_name: jira_schema

.gitignore

Lines changed: 69 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,73 @@
1-
target/
1+
# dbt
2+
**/package-lock.yml
3+
package-lock.yml
4+
.dbt/
25
dbt_modules/
6+
dbt_packages/
37
logs/
8+
profiles.yml
9+
target/
10+
*.log
11+
12+
# IDE files
13+
.idea/
14+
.vscode/
15+
*~
16+
*.swp
17+
*.swo
18+
19+
# Jupyter Notebook
20+
.ipynb_checkpoints
21+
22+
# OS generated files
23+
**/.DS_Store
424
.DS_Store
25+
.Spotlight-V100
26+
.Trashes
27+
._*
28+
Thumbs.db
29+
ehthumbs.db
30+
31+
# Python
32+
*.egg
33+
*.egg-info/
34+
*.py[cod]
35+
*.so
36+
*$py.class
37+
.Python
38+
__pycache__/
39+
build/
40+
develop-eggs/
41+
dist/
42+
downloads/
43+
eggs/
44+
.env
45+
.installed.cfg
46+
lib/
47+
lib64/
48+
MANIFEST
49+
parts/
50+
sdist/
51+
var/
52+
wheels/
53+
54+
# Secrets and credentials
55+
.env.*
56+
.secrets
57+
credentials.json
58+
service-account.json
59+
60+
# Temporary files
61+
.cache/
62+
*.temp
63+
*.tmp
64+
65+
# Virtual environments
66+
.conda/
67+
.env
68+
.venv
69+
ENV/
570
env/
6-
dbt_packages/
7-
package-lock.yml
71+
env.bak/
72+
venv/
73+
venv.bak/

CHANGELOG.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,27 @@
1+
# dbt_jira v1.0.0
2+
3+
[PR #145](https://github.com/fivetran/dbt_jira/pull/145) includes the following updates:
4+
5+
## Breaking Changes
6+
7+
### Source Package Consolidation
8+
- Removed the dependency on the `fivetran/jira_source` package.
9+
- All functionality from the source package has been merged into this transformation package for improved maintainability and clarity.
10+
- If you reference `fivetran/jira_source` in your `packages.yml`, you must remove this dependency to avoid conflicts.
11+
- Any source overrides referencing the `fivetran/jira_source` package will also need to be removed or updated to reference this package.
12+
- Update any jira_source-scoped variables to be scoped to only under this package. See the [README](https://github.com/fivetran/dbt_jira/blob/main/README.md) for how to configure the build schema of staging models.
13+
- As part of the consolidation, vars are no longer used to reference staging models, and only sources are represented by vars. Staging models are now referenced directly with `ref()` in downstream models.
14+
15+
### dbt Fusion Compatibility Updates
16+
- Updated package to maintain compatibility with dbt-core versions both before and after v1.10.6, which introduced a breaking change to multi-argument test syntax (e.g., `unique_combination_of_columns`).
17+
- Temporarily removed unsupported tests to avoid errors and ensure smoother upgrades across different dbt-core versions. These tests will be reintroduced once a safe migration path is available.
18+
- Removed all `dbt_utils.unique_combination_of_columns` tests.
19+
- Moved `loaded_at_field: _fivetran_synced` under the `config:` block in `src_jira.yml`.
20+
21+
## Under the Hood
22+
- Updated conditions in `.github/workflows/auto-release.yml`.
23+
- Added `.github/workflows/generate-docs.yml`.
24+
125
# dbt_jira v0.21.0
226

327
[PR #140](https://github.com/fivetran/dbt_jira/pull/140) includes the following updates:

README.md

Lines changed: 12 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Jira Transformation dbt Package ([Docs](https://fivetran.github.io/dbt_jira/))
1+
# Jira dbt Package ([Docs](https://fivetran.github.io/dbt_jira/))
22

33
<p align="left">
44
<a alt="License"
@@ -16,7 +16,7 @@
1616
</p>
1717

1818
## What does this dbt package do?
19-
- Produces modeled tables that leverage Jira data from [Fivetran's connector](https://fivetran.com/docs/applications/jira) in the format described by [this ERD](https://fivetran.com/docs/applications/jira#schemainformation) and builds off the output of our [Jira source package](https://github.com/fivetran/dbt_jira_source).
19+
- Produces modeled tables that leverage Jira data from [Fivetran's connector](https://fivetran.com/docs/applications/jira) in the format described by [this ERD](https://fivetran.com/docs/applications/jira#schemainformation).
2020
- Enables you to better understand the workload, performance, and velocity of your team's work using Jira issues. It performs the following actions:
2121
- Creates a daily issue history table so you can quickly create agile reports, such as burndown charts, along any issue field.
2222
- Enriches the core issue table with relevant data regarding its workflow and current state.
@@ -72,9 +72,11 @@ Include the following jira package version in your `packages.yml` file:
7272
```yaml
7373
packages:
7474
- package: fivetran/jira
75-
version: [">=0.21.0", "<0.22.0"]
76-
75+
version: [">=1.0.0", "<1.1.0"]
7776
```
77+
78+
> All required sources and staging models are now bundled into this transformation package. Do not include `fivetran/jira_source` in your `packages.yml` since this package has been deprecated.
79+
7880
### Step 3: Define database and schema variables
7981
By default, this package runs using your destination and the `jira` schema. If this is not where your Jira data is (for example, if your Jira schema is named `jira_fivetran`), add the following configuration to your root `dbt_project.yml` file:
8082

@@ -147,12 +149,12 @@ By default, this package builds the Jira staging models within a schema titled (
147149

148150
```yml
149151
models:
150-
jira_source:
151-
+schema: my_new_schema_name # leave blank for just the target_schema
152152
jira:
153-
+schema: my_new_schema_name # leave blank for just the target_schema
153+
+schema: my_new_schema_name # Leave +schema: blank to use the default target_schema.
154+
staging:
155+
+schema: my_new_schema_name # Leave +schema: blank to use the default target_schema.
154156
```
155-
157+
156158
#### Change the source table references
157159
If an individual source table has a different name than the package expects, add the table name as it appears in your destination to the respective variable:
158160

@@ -177,19 +179,16 @@ vars:
177179
### (Optional) Step 6: Orchestrate your models with Fivetran Transformations for dbt Core™
178180
<details><summary>Expand for details</summary>
179181
<br>
180-
182+
181183
Fivetran offers the ability for you to orchestrate your dbt project through [Fivetran Transformations for dbt Core™](https://fivetran.com/docs/transformations/dbt). Learn how to set up your project for orchestration through Fivetran in our [Transformations for dbt Core setup guides](https://fivetran.com/docs/transformations/dbt#setupguide).
182184
</details>
183185

184186
## Does this package have dependencies?
185187
This dbt package is dependent on the following dbt packages. These dependencies are installed by default within this package. For more information on the following packages, refer to the [dbt hub](https://hub.getdbt.com/) site.
186188
> IMPORTANT: If you have any of these dependent packages in your own `packages.yml` file, we highly recommend that you remove them from your root `packages.yml` to avoid package version conflicts.
187-
189+
188190
```yml
189191
packages:
190-
- package: fivetran/jira_source
191-
version: [">=0.9.0", "<0.10.0"]
192-
193192
- package: fivetran/fivetran_utils
194193
version: [">=0.4.0", "<0.5.0"]
195194

dbt_project.yml

Lines changed: 23 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,32 +1,36 @@
11
name: 'jira'
2-
version: '0.21.0'
2+
version: '1.0.0'
33
config-version: 2
44
require-dbt-version: [">=1.3.0", "<2.0.0"]
5-
vars:
65

6+
vars:
77
jira:
8-
issue: "{{ ref('stg_jira__issue') }}"
9-
project: "{{ ref('stg_jira__project') }}"
10-
user: "{{ ref('stg_jira__user') }}"
11-
issue_type: "{{ ref('stg_jira__issue_type') }}"
12-
status: "{{ ref('stg_jira__status') }}"
13-
status_category: "{{ ref('stg_jira__status_category') }}"
14-
resolution: "{{ ref('stg_jira__resolution') }}"
15-
priority: "{{ ref('stg_jira__priority') }}"
16-
issue_multiselect_history: "{{ ref('stg_jira__issue_multiselect_history') }}"
17-
issue_field_history: "{{ ref('stg_jira__issue_field_history') }}"
18-
comment: "{{ ref('stg_jira__comment') }}"
19-
issue_link: "{{ ref('stg_jira__issue_link') }}"
20-
component: "{{ ref('stg_jira__component') }}"
21-
field: "{{ ref('stg_jira__field') }}"
22-
field_option: "{{ ref('stg_jira__field_option') }}"
23-
sprint: "{{ ref('stg_jira__sprint') }}"
24-
version: "{{ ref('stg_jira__version') }}"
8+
issue: "{{ source('jira', 'issue') }}"
9+
project: "{{ source('jira', 'project') }}"
10+
user: "{{ source('jira', 'user') }}"
11+
issue_type: "{{ source('jira', 'issue_type') }}"
12+
status: "{{ source('jira', 'status') }}"
13+
status_category: "{{ source('jira', 'status_category') }}"
14+
resolution: "{{ source('jira', 'resolution') }}"
15+
priority: "{{ source('jira', 'priority') }}"
16+
issue_multiselect_history: "{{ source('jira', 'issue_multiselect_history') }}"
17+
issue_field_history: "{{ source('jira', 'issue_field_history') }}"
18+
comment: "{{ source('jira', 'comment') }}"
19+
issue_link: "{{ source('jira', 'issue_link') }}"
20+
component: "{{ source('jira', 'component') }}"
21+
field: "{{ source('jira', 'field') }}"
22+
field_option: "{{ source('jira', 'field_option') }}"
23+
sprint: "{{ source('jira', 'sprint') }}"
24+
version: "{{ source('jira', 'version') }}"
2525
jira_issue_history_buffer: 1
26+
2627
models:
2728
jira:
2829
+materialized: table
2930
+schema: jira
3031
intermediate:
3132
+materialized: ephemeral
3233
+schema: int_jira
34+
staging:
35+
+materialized: view
36+
+schema: jira_source

docs/catalog.json

Lines changed: 1 addition & 1 deletion
Large diffs are not rendered by default.

docs/manifest.json

Lines changed: 1 addition & 1 deletion
Large diffs are not rendered by default.

integration_tests/dbt_project.yml

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
11
name: 'jira_integration_tests'
2-
version: '0.21.0'
2+
version: '1.0.0'
33
config-version: 2
44
profile: 'integration_tests'
55

66
vars:
77
# Comment out the below when generating docs
8-
# issue_field_history_columns: ['summary', 'components']
9-
jira_source:
8+
issue_field_history_columns: ['summary', 'components'] # @docs-ignore
9+
jira:
1010
jira_schema: jira_integrations_tests_42
1111
jira_comment_identifier: "comment"
1212
jira_component_identifier: "component"
@@ -36,6 +36,8 @@ models:
3636
# +schema: "jira_{{ var('directed_schema','dev') }}"
3737

3838
seeds:
39+
+docs:
40+
show: False
3941
jira_integration_tests:
4042
+column_types:
4143
_fivetran_synced: timestamp
@@ -89,3 +91,4 @@ dispatch:
8991

9092
flags:
9193
send_anonymous_usage_stats: False
94+
use_colors: True

0 commit comments

Comments
 (0)