Skip to content

Commit d4a4f01

Browse files
fivetran-data-model-botfivetran-catfritzfivetran-avinashfivetran-joemarkiewiczgithub-actions[bot]
authored
Consolidate source + remove tests (#18)
* Consolidate dbt source into transform * Q2 FY26 Automatic Package Updates (#16) * Q2 FY26: Apply automated update. * Q2 FY26: Update auto-release workflow only. --------- Co-authored-by: fivetran-catfritz <111930712+fivetran-catfritz@users.noreply.github.com> Co-authored-by: Avinash Kunnath <108772760+fivetran-avinash@users.noreply.github.com> * Change var to ref * update staging schema * config loaded_at_field * Apply suggestions from code review Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com> * Generate dbt docs via GitHub Actions --------- Co-authored-by: fivetran-catfritz <111930712+fivetran-catfritz@users.noreply.github.com> Co-authored-by: Avinash Kunnath <108772760+fivetran-avinash@users.noreply.github.com> Co-authored-by: Avinash Kunnath <avinash.kunnath@fivetran.com> Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
1 parent 0046108 commit d4a4f01

35 files changed

+1885
-154
lines changed

.buildkite/pipeline.yml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ steps:
33
key: "run-dbt-postgres"
44
plugins:
55
- docker#v3.13.0:
6-
image: "python:3.8"
6+
image: "python:3.10.13"
77
shell: [ "/bin/bash", "-e", "-c" ]
88
environment:
99
- "BASH_ENV=/tmp/.bashrc"
@@ -18,7 +18,7 @@ steps:
1818
key: "run_dbt_snowflake"
1919
plugins:
2020
- docker#v3.13.0:
21-
image: "python:3.8"
21+
image: "python:3.10.13"
2222
shell: [ "/bin/bash", "-e", "-c" ]
2323
environment:
2424
- "BASH_ENV=/tmp/.bashrc"
@@ -35,7 +35,7 @@ steps:
3535
key: "run_dbt_bigquery"
3636
plugins:
3737
- docker#v3.13.0:
38-
image: "python:3.8"
38+
image: "python:3.10.13"
3939
shell: [ "/bin/bash", "-e", "-c" ]
4040
environment:
4141
- "BASH_ENV=/tmp/.bashrc"
@@ -47,7 +47,7 @@ steps:
4747
key: "run_dbt_redshift"
4848
plugins:
4949
- docker#v3.13.0:
50-
image: "python:3.8"
50+
image: "python:3.10.13"
5151
shell: [ "/bin/bash", "-e", "-c" ]
5252
environment:
5353
- "BASH_ENV=/tmp/.bashrc"
@@ -62,7 +62,7 @@ steps:
6262
key: "run_dbt_databricks"
6363
plugins:
6464
- docker#v3.13.0:
65-
image: "python:3.8"
65+
image: "python:3.10.13"
6666
shell: [ "/bin/bash", "-e", "-c" ]
6767
environment:
6868
- "BASH_ENV=/tmp/.bashrc"
Lines changed: 32 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,38 @@
1-
## PR Overview
2-
**This PR will address the following Issue/Feature:**
1+
<!--
2+
Pre-Submission Reminders
3+
Before marking this PR as "ready for review":
34
4-
**This PR will result in the following new package version:**
5-
<!--- Please add details around your decision for breaking vs non-breaking version upgrade. If this is a breaking change, were backwards-compatible options explored? -->
5+
- `dbt run --full-refresh && dbt test`
6+
- `dbt run` && `dbt test` (if incremental models are present)
7+
- The related issue is linked, tagged, and appropriately assigned
8+
- Documentation and version updates are included, if applicable
9+
- `docs` have been regenerated (unless there are no code or YAML changes)
10+
- BuildKite integration tests are passing
11+
-->
612

7-
**Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:**
8-
<!--- Copy/paste the CHANGELOG for this version below. -->
13+
## PR Overview
14+
**Package version introduced in this PR:**
15+
-
916

10-
## PR Checklist
11-
### Basic Validation
12-
Please acknowledge that you have successfully performed the following commands locally:
13-
- [ ] dbt run –full-refresh && dbt test
14-
- [ ] dbt run (if incremental models are present) && dbt test
17+
**This PR addresses the following Issue/Feature(s):**
18+
<!-- Add Issue # or internal ticket reference -->
19+
-
1520

16-
Before marking this PR as "ready for review" the following have been applied:
17-
- [ ] The appropriate issue has been linked, tagged, and properly assigned
18-
- [ ] All necessary documentation and version upgrades have been applied
19-
- [ ] docs were regenerated (unless this PR does not include any code or yml updates)
20-
- [ ] BuildKite integration tests are passing
21-
- [ ] Detailed validation steps have been provided below
21+
**Summary of changes:**
22+
<!-- 1-2 sentences describing PR changes. -->
23+
-
2224

23-
### Detailed Validation
24-
Please share any and all of your validation steps:
25-
<!--- Provide the steps you took to validate your changes below. -->
25+
### Submission Checklist
26+
- [ ] Alignment meeting with the reviewer (if needed)
27+
- [ ] Timeline and validation requirements discussed
28+
- [ ] Provide validation details:
29+
- [ ] **Validation Steps:** Check for unintentional effects (e.g., add/run consistency & integrity tests)
30+
- [ ] **Testing Instructions:** Confirm the change addresses the issue(s)
31+
- [ ] **Focus Areas:** Complex logic or queries that need extra attention
32+
- [ ] Merge any relevant open PRs into this PR
2633

27-
### If you had to summarize this PR in an emoji, which would it be?
28-
<!--- For a complete list of markdown compatible emojis check our this git repo (https://gist.github.com/rxaviers/7360908) -->
29-
:dancer:
34+
### Changelog
35+
<!-- Recommend drafting changelog notes, then refining via ChatGPT using:
36+
"Draft a changelog entry based on the following notes." -->
37+
- [ ] Draft changelog for PR
38+
- [ ] Final changelog for release review

.github/workflows/auto-release.yml

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,12 @@ on:
33
pull_request:
44
types:
55
- closed
6-
branches:
7-
- main
6+
- labeled
87

98
jobs:
10-
call-workflow-passing-data:
11-
if: github.event.pull_request.merged
9+
release:
10+
if: |
11+
(github.event.pull_request.merged == true && github.event.pull_request.base.ref == 'main') ||
12+
github.event.label.name == 'pre-release'
1213
uses: fivetran/dbt_package_automations/.github/workflows/auto-release.yml@main
13-
secrets: inherit
14+
secrets: inherit
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
name: 'generate dbt docs'
2+
on:
3+
pull_request:
4+
types:
5+
- labeled
6+
7+
jobs:
8+
generate-docs:
9+
if: github.event.label.name == 'docs:ready'
10+
uses: fivetran/dbt_package_automations/.github/workflows/generate-docs.yml@main
11+
secrets: inherit
12+
with:
13+
schema_var_name: facebook_pages_schema

.gitignore

Lines changed: 70 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,73 @@
1-
2-
target/
1+
# dbt
2+
**/package-lock.yml
3+
package-lock.yml
4+
.dbt/
35
dbt_modules/
6+
dbt_packages/
47
logs/
8+
profiles.yml
9+
target/
10+
*.log
11+
12+
# IDE files
13+
.idea/
14+
.vscode/
15+
*~
16+
*.swp
17+
*.swo
18+
19+
# Jupyter Notebook
20+
.ipynb_checkpoints
21+
22+
# OS generated files
23+
**/.DS_Store
524
.DS_Store
6-
dbt_packages/
25+
.Spotlight-V100
26+
.Trashes
27+
._*
28+
Thumbs.db
29+
ehthumbs.db
30+
31+
# Python
32+
*.egg
33+
*.egg-info/
34+
*.py[cod]
35+
*.so
36+
*$py.class
37+
.Python
38+
__pycache__/
39+
build/
40+
develop-eggs/
41+
dist/
42+
downloads/
43+
eggs/
44+
.env
45+
.installed.cfg
46+
lib/
47+
lib64/
48+
MANIFEST
49+
parts/
50+
sdist/
51+
var/
52+
wheels/
53+
54+
# Secrets and credentials
55+
.env.*
56+
.secrets
57+
credentials.json
58+
service-account.json
59+
60+
# Temporary files
61+
.cache/
62+
*.temp
63+
*.tmp
64+
65+
# Virtual environments
66+
.conda/
67+
.env
68+
.venv
69+
ENV/
70+
env/
71+
env.bak/
72+
venv/
73+
venv.bak/

CHANGELOG.md

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,25 @@
1-
# dbt_facebook_pages version.version
1+
# dbt_facebook_pages v1.0.0
2+
[PR #18](https://github.com/fivetran/dbt_facebook_pages/pull/18) includes the following updates:
3+
4+
## Breaking Changes
5+
6+
### Source Package Consolidation
7+
- Removed the dependency on the `fivetran/facebook_pages_source` package.
8+
- All functionality from the source package has been merged into this transformation package for improved maintainability and clarity.
9+
- If you reference `fivetran/facebook_pages_source` in your `packages.yml`, you must remove this dependency to avoid conflicts.
10+
- Any source overrides referencing the `fivetran/facebook_pages_source` package will also need to be removed or updated to reference this package.
11+
- Update any facebook_pages_source-scoped variables to be scoped to only under this package. See the [README](https://github.com/fivetran/dbt_facebook_pages?tab=readme-ov-file#change-the-build-schema) for how to configure the build schema of staging models.
12+
- As part of the consolidation, vars are no longer used to reference staging models, and only sources are represented by vars. Staging models are now referenced directly with `ref()` in downstream models.
13+
14+
### dbt Fusion Compatibility Updates
15+
- Updated package to maintain compatibility with dbt-core versions both before and after v1.10.6, which introduced a breaking change to multi-argument test syntax (e.g., `unique_combination_of_columns`).
16+
- Temporarily removed unsupported tests to avoid errors and ensure smoother upgrades across different dbt-core versions. These tests will be reintroduced once a safe migration path is available.
17+
- Removed all `dbt_utils.unique_combination_of_columns` tests.
18+
- Moved `loaded_at_field: _fivetran_synced` under the `config:` block in `src_facebook_pages.yml`.
19+
20+
### Under the Hood
21+
- Updated conditions in `.github/workflows/auto-release.yml`.
22+
- Added `.github/workflows/generate-docs.yml`.
223

324
## Documentation
425
- Added Quickstart model counts to README. ([#15](https://github.com/fivetran/dbt_facebook_pages/pull/15))

LICENSE

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -186,7 +186,7 @@
186186
same "printed page" as the copyright notice for easier
187187
identification within third-party archives.
188188

189-
Copyright [yyyy] [name of copyright owner]
189+
Copyright © 2025 Fivetran Inc.
190190

191191
Licensed under the Apache License, Version 2.0 (the "License");
192192
you may not use this file except in compliance with the License.

README.md

Lines changed: 18 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
1-
<p align="center">
1+
2+
# Facebook Pages dbt Package ([Docs](https://fivetran.github.io/dbt_facebook_pages/))
3+
4+
<p align="left">
25
<a alt="License"
36
href="https://github.com/fivetran/dbt_facebook_pages/blob/main/LICENSE">
47
<img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" /></a>
@@ -8,13 +11,14 @@
811
<img src="https://img.shields.io/badge/Maintained%3F-yes-green.svg" /></a>
912
<a alt="PRs">
1013
<img src="https://img.shields.io/badge/Contributions-welcome-blueviolet" /></a>
14+
<a alt="Fivetran Quickstart Compatible"
15+
href="https://fivetran.com/docs/transformations/dbt/quickstart">
16+
<img src="https://img.shields.io/badge/Fivetran_Quickstart_Compatible%3F-yes-green.svg" /></a>
1117
</p>
1218

13-
# Facebook Pages Modeling dbt Package ([Docs](https://fivetran.github.io/dbt_facebook_pages/))
14-
1519
## What does this dbt package do?
1620

17-
- Produces modeled tables that leverage Facebook Pages from [Fivetran's connector](https://fivetran.com/docs/applications/facebook-pages) in the format described by [this ERD](https://fivetran.com/docs/applications/facebook_pages#schemainformation) and builds off the output of our [Facebook Pages source package](https://github.com/fivetran/dbt_facebook_pages_source).
21+
- Produces modeled tables that leverage Facebook Pages from [Fivetran's connector](https://fivetran.com/docs/applications/facebook-pages) in the format described by [this ERD](https://fivetran.com/docs/applications/facebook_pages#schemainformation).
1822

1923
The main focus of the package is to transform the core social media object tables into analytics-ready models that can be easily unioned in to other social media platform packages to get a single view. This is aided by our [Social Media Reporting package](https://github.com/fivetran/dbt_social_media_reporting).
2024

@@ -53,10 +57,10 @@ Include the following Facebook Pages package version in your `packages.yml`
5357
```yaml
5458
packages:
5559
- package: fivetran/facebook_pages
56-
version: [">=0.3.0", "<0.4.0"] # we recommend using ranges to capture non-breaking changes automatically
60+
version: [">=1.0.0", "<1.1.0"] # we recommend using ranges to capture non-breaking changes automatically
5761
```
5862

59-
Do NOT include the `facebook_pages_source` package in this file. The transformation package itself has a dependency on it and will install the source package as well.
63+
> All required sources and staging models are now bundled into this transformation package. Do not include `fivetran/facebook_pages_source` in your `packages.yml` since this package has been deprecated.
6064

6165
### Step 3: Configure Your Variables
6266
#### Database and Schema Variables
@@ -78,15 +82,15 @@ By default, this package will build the Facebook Pages staging models within a s
7882
```yml
7983
models:
8084
facebook_pages:
81-
+schema: my_new_schema_name # leave blank for just the target_schema
82-
facebook_pages_source:
83-
+schema: my_new_schema_name # leave blank for just the target_schema
85+
+schema: my_new_schema_name # Leave +schema: blank to use the default target_schema.
86+
staging:
87+
+schema: my_new_schema_name # Leave +schema: blank to use the default target_schema.
8488
```
8589

8690
#### Change the source table references
8791
If an individual source table has a different name than the package expects, add the table name as it appears in your destination to the respective variable:
88-
> IMPORTANT: See this project's [`dbt_project.yml`](https://github.com/fivetran/dbt_facebook_pages_source/blob/main/dbt_project.yml) variable declarations to see the expected names.
89-
92+
> IMPORTANT: See this project's [`dbt_project.yml`](https://github.com/fivetran/dbt_facebook_pages/blob/main/dbt_project.yml) variable declarations to see the expected names.
93+
9094
```yml
9195
vars:
9296
facebook_pages_<default_source_table_name>_identifier: your_table_name
@@ -111,19 +115,16 @@ vars:
111115
### (Optional) Step 5: Orchestrate your models with Fivetran Transformations for dbt Core™
112116
<details><summary>Expand for configurations</summary>
113117
<br>
114-
118+
115119
Fivetran offers the ability for you to orchestrate your dbt project through the [Fivetran Transformations for dbt Core™](https://fivetran.com/docs/transformations/dbt) product. Refer to the linked docs for more information on how to setup your project for orchestration through Fivetran.
116120
</details>
117121

118122
## Does this package have dependencies?
119123
This dbt package is dependent on the following dbt packages. These dependencies are installed by default within this package. For more information on the following packages, refer to the [dbt hub](https://hub.getdbt.com/) site.
120124
> IMPORTANT: If you have any of these dependent packages in your own `packages.yml` file, we highly recommend that you remove them from your root `packages.yml` to avoid package version conflicts.
121-
125+
122126
```yml
123127
packages:
124-
- package: fivetran/facebook_pages_source
125-
version: [">=0.3.0", "<0.4.0"]
126-
127128
- package: fivetran/fivetran_utils
128129
version: [">=0.4.0", "<0.5.0"]
129130
@@ -144,5 +145,5 @@ These dbt packages are developed by a small team of analytics engineers at Fivet
144145
We highly encourage and welcome contributions to this package. Check out [this post](https://discourse.getdbt.com/t/contributing-to-a-dbt-package/657) on the best workflow for contributing to a package.
145146

146147
## Are there any resources available?
147-
- If you encounter any questions or want to reach out for help, see the [GitHub Issue](https://github.com/fivetran/dbt_facebook_pages/issues/new/choose section to find the right avenue of support for you.
148+
- If you encounter any questions or want to reach out for help, see the [GitHub Issue](https://github.com/fivetran/dbt_facebook_pages/issues/new/choose) section to find the right avenue of support for you.
148149
- If you would like to provide feedback to the dbt package team at Fivetran, or would like to request a future dbt package to be developed, then feel free to fill out our [Feedback Form](https://www.surveymonkey.com/r/DQ7K7WW).

dbt_project.yml

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,18 @@
11
name: 'facebook_pages'
2-
version: '0.3.0'
2+
version: '1.0.0'
33
config-version: 2
44
require-dbt-version: [">=1.3.0", "<2.0.0"]
55
vars:
6-
posts: "{{ ref('stg_facebook_pages__post_history') }}"
7-
pages: "{{ ref('stg_facebook_pages__page') }}"
8-
post_metrics: "{{ ref('stg_facebook_pages__lifetime_post_metrics_total') }}"
9-
page_metrics: "{{ ref('stg_facebook_pages__daily_page_metrics_total') }}"
6+
posts: "{{ source('facebook_pages', 'post_history') }}"
7+
pages: "{{ source('facebook_pages', 'page') }}"
8+
post_metrics: "{{ source('facebook_pages', 'lifetime_post_metrics_total') }}"
9+
page_metrics: "{{ source('facebook_pages', 'daily_page_metrics_total') }}"
1010
models:
1111
facebook_pages:
1212
+schema: facebook_pages
1313
+materialized: table
14+
staging:
15+
+schema: stg_facebook_pages
16+
+materialized: table
17+
tmp:
18+
+materialized: view

docs/catalog.json

Lines changed: 1 addition & 1 deletion
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)