Skip to content

feature/country-report #24

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Apr 30, 2025
2 changes: 2 additions & 0 deletions .buildkite/scripts/run_models.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,6 @@ dbt run --target "$db" --full-refresh
dbt test --target "$db"
dbt run --vars '{ad_reporting__url_report__using_null_filter: false}' --target "$db" --full-refresh
dbt test --vars '{ad_reporting__url_report__using_null_filter: false}' --target "$db"
dbt run --vars '{tiktok_ads__using_ad_country_report: false, tiktok_ads__using_campaign_country_report: false}' --target "$db" --full-refresh
dbt test --target "$db"
dbt run-operation fivetran_utils.drop_schemas_automation --target "$db"
7 changes: 6 additions & 1 deletion .quickstart/quickstart.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@ schema_key: tiktok_ads_schema

dbt_versions: ">=1.3.0 <2.0.0"

table_variables:
tiktok_ads__using_campaign_country_report:
- campaign_country_report

destination_configurations:
databricks:
dispatch:
Expand All @@ -14,5 +18,6 @@ public_models: [
"tiktok_ads__url_report",
"tiktok_ads__campaign_report",
"tiktok_ads__ad_report",
"tiktok_ads__advertiser_report"
"tiktok_ads__advertiser_report",
"tiktok_ads__campaign_country_report"
]
21 changes: 20 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,28 @@
# dbt_tiktok_ads version.version
# dbt_tiktok_ads v0.8.0

[PR #24](https://github.com/fivetran/dbt_tiktok_ads/pull/24) includes the following updates:
## Schema Changes
**4 total changes • 0 possible breaking changes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I presume we want this bolded, so:

Suggested change
**4 total changes • 0 possible breaking changes
**4 total changes • 0 possible breaking changes**

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update applied.

| Table/Column | Change Type | Old Name | New Name | Notes |
|---------------------------------------------------|-------------|----------|-------------------------------------------|-------------------------------------------------------------------|
| tiktok_ads__campaign_country_report | New Model | | | New table that represents the daily performance of a campaign at the country/geographic region level. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we should we add links to these models in the DAG?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed - added

| stg_tiktok_ads__campaign_country_report_tmp | New Model | | | Temp model added for `campaign_country_report`. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we differentiate between Staging and Transform Models here? "New Staging Model" vs. "New Transformation Model" for example.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed - added

| stg_tiktok_ads__campaign_country_report | New Model | | | Staging model added for `campaign_country_report`. |
| objective_type, status, budget, budget_mode, created_at, is_new_structure | New Columns | | | New columns added to the stg_tiktok_ads__campaign_history model. |

## Feature Updates
- Added the `tiktok_ads__campaign_country_report` end model and upstream staging models. See above for schema change details and new models added.
- For dbt Core users: If you do not sync this table or would like disable these new models you can disable the models by setting the `tiktok_ads__using_campaign_country_report` variable to `false` in your `dbt_project.yml` file (`true` by default).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a link to the README?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed - added

- Included the `tiktok_ads__campaign_country_report_passthrough_metrics` passthrough variable in the above mentioned new staging models. Refer to the [README](https://github.com/fivetran/dbt_tiktok_ads/tree/main?tab=readme-ov-file#passing-through-additional-metrics) for more details.
- Introduced the above mentioned new columns to the `stg_tiktok_ads__campaign_history` model.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this repetitive with line 11 of the Schema Changes table?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed - added


## Documentation
- Added Quickstart model counts to README. ([#23](https://github.com/fivetran/dbt_tiktok_ads/pull/23))
- Corrected references to connectors and connections in the README. ([#23](https://github.com/fivetran/dbt_tiktok_ads/pull/23))

## Under the Hood
- Added vertical integrity test to ensure data accuracy of the new tiktok_ads__campaign_country_report.

# dbt_tiktok_ads v0.7.0
[PR #22](https://github.com/fivetran/dbt_tiktok_ads/pull/22) includes the following updates:

Expand Down
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright [yyyy] [name of copyright owner]
Copyright © 2025 Fivetran Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
17 changes: 14 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,13 @@ The following table provides a detailed list of all tables materialized within t
| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| [tiktok_ads__ad_group_report](https://github.com/fivetran/dbt_tiktok_ads/blob/main/models/tiktok_ads__ad_group_report.sql) | Each record represents the daily performance for each ad group. This also includes additional data on the demographics you are targeting. |
| [tiktok_ads__campaign_report](https://github.com/fivetran/dbt_tiktok_ads/blob/main/models/tiktok_ads__campaign_report.sql) | Each record represents the daily performance for each campaign. |
| [tiktok_ads__campaign_country_report](https://github.com/fivetran/dbt_tiktok_ads/blob/main/models/tiktok_ads__campaign_country_report.sql) | Each record in this table represents the daily performance of a campaign at the country/geographic region level. |
| [tiktok_ads__advertiser_report](https://github.com/fivetran/dbt_tiktok_ads/blob/main/models/tiktok_ads__advertiser_report.sql) | Each record represents the daily performance for each account. |
| [tiktok_ads__ad_report](https://github.com/fivetran/dbt_tiktok_ads/blob/main/models/tiktok_ads__ad_report.sql) | Each record represents the daily performance for each ad. This also includes additional data on the demographics you are targeting. |
| [tiktok_ads__url_report](https://github.com/fivetran/dbt_tiktok_ads/blob/main/models/tiktok_ads__url_report.sql) | Each record in this table represents the daily performance of URLs at the ad level. This also includes additional data on the demographics you are targeting.

### Materialized Models
Each Quickstart transformation job run materializes 19 models if all components of this data model are enabled. This count includes all staging, intermediate, and final models materialized as `view`, `table`, or `incremental`.
Each Quickstart transformation job run materializes 22 models if all components of this data model are enabled. This count includes all staging, intermediate, and final models materialized as `view`, `table`, or `incremental`.
<!--section-end-->

## How do I use the dbt package?
Expand All @@ -57,7 +58,7 @@ Include the following tiktok_ads package version in your `packages.yml` file _if
```yaml
packages:
- package: fivetran/tiktok_ads
version: [">=0.7.0", "<0.8.0"]
version: [">=0.8.0", "<0.9.0"]

```
Do **NOT** include the `tiktok_ads_source` package in this file. The transformation package itself has a dependency on it and will install the source package as well.
Expand Down Expand Up @@ -89,6 +90,13 @@ vars:

To connect your multiple schema/database sources to the package models, follow the steps outlined in the [Union Data Defined Sources Configuration](https://github.com/fivetran/dbt_fivetran_utils/tree/releases/v0.4.latest#union_data-source) section of the Fivetran Utils documentation for the union_data macro. This will ensure a proper configuration and correct visualization of connections in the DAG.

#### Disable Country Reports
This package leverages the `campaign_country_report` table to help report on ad and campaign performance by country. However, if you are not actively syncing this report from your TikTok Ads connection, you may disable the transformations for the `campaign_country_report` by adding the following variable configuration to your root `dbt_project.yml` file:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This package leverages the `campaign_country_report` table to help report on ad and campaign performance by country. However, if you are not actively syncing this report from your TikTok Ads connection, you may disable the transformations for the `campaign_country_report` by adding the following variable configuration to your root `dbt_project.yml` file:
This package leverages the `campaign_country_report` table to help report on ad and campaign performance by country and region. However, if you are not actively syncing this report from your TikTok Ads connection, you may disable the transformations for the `campaign_country_report` by adding the following variable configuration to your root `dbt_project.yml` file:

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See same comment from the source. It's not entirely accurate to claim and region here. Instead we'll clarify country/geographic region level.

```yml
vars:
tiktok_ads__using_campaign_country_report: False # True by default
```

#### Passing Through Additional Metrics
By default, this package will select `clicks`, `impressions`, `spend` , `conversion`, `real_time_conversion`, `total_purchase_value`, and `total_sales_lead_value` from the source reporting tables to store into the staging models. If you would like to pass through additional metrics to the staging models, add the below configurations to your `dbt_project.yml` file. These variables allow for the pass-through fields to be aliased (`alias`) if desired, but not required. Use the below format for declaring the respective pass-through variables:

Expand All @@ -102,6 +110,9 @@ vars:
- name: "my_other_field"
tiktok_ads__ad_hourly_passthrough_metrics:
- name: "this_field"
tiktok_ads__campaign_country_report_passthrough_metrics:
- name: "unique_string_field"
alias: "field_id"
tiktok_ads__campaign_hourly_passthrough_metrics:
- name: "unique_string_field"
alias: "field_id"
Expand Down Expand Up @@ -146,7 +157,7 @@ This dbt package is dependent on the following dbt packages. These dependencies
```yml
packages:
- package: fivetran/tiktok_ads_source
version: [">=0.7.0", "<0.8.0"]
version: [">=0.8.0", "<0.9.0"]

- package: fivetran/fivetran_utils
version: [">=0.4.0", "<0.5.0"]
Expand Down
6 changes: 5 additions & 1 deletion dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 'tiktok_ads'
version: '0.7.0'
version: '0.8.0'
config-version: 2
require-dbt-version: [">=1.3.0", "<2.0.0"]
models:
Expand All @@ -15,6 +15,10 @@ vars:
ad_report_hourly: "{{ ref('stg_tiktok_ads__ad_report_hourly') }}"
ad_history: "{{ ref('stg_tiktok_ads__ad_history') }}"
advertiser: "{{ ref('stg_tiktok_ads__advertiser') }}"
ad_country_report: "{{ ref('stg_tiktok_ads__ad_country_report') }}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to no longer being built, so it can be removed.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, this was initially included and then removed. Removing the artifact.

campaign_country_report: "{{ ref('stg_tiktok_ads__campaign_country_report') }}"
tiktok_ads__ad_group_hourly_passthrough_metrics: []
tiktok_ads__ad_hourly_passthrough_metrics: []
tiktok_ads__campaign_hourly_passthrough_metrics: []
tiktok_ads__campaign_country_report_passthrough_metrics: []
tiktok_ads__ad_country_report_passthrough_metrics: []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This macro seems to no longer being built, so it can be removed.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, this was initially included and then removed. Removing the artifact.

2 changes: 1 addition & 1 deletion docs/catalog.json

Large diffs are not rendered by default.

253 changes: 214 additions & 39 deletions docs/index.html

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/manifest.json

Large diffs are not rendered by default.

7 changes: 5 additions & 2 deletions integration_tests/dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 'tiktok_ads_integration_tests'
version: '0.7.0'
version: '0.8.0'
profile: 'integration_tests'
config-version: 2

Expand All @@ -16,14 +16,17 @@ vars:
tiktok_ads_ad_report_hourly_identifier: "tiktok_ad_report_hourly_data"
tiktok_ads_adgroup_report_hourly_identifier: "tiktok_adgroup_report_hourly_data"
tiktok_ads_campaign_report_hourly_identifier: "tiktok_campaign_report_hourly_data"
tiktok_ads_campaign_country_report_identifier: "tiktok_campaign_country_report_data"

tiktok_ads__ad_group_hourly_passthrough_metrics:
- name: total_purchase_value
- name: total_sales_lead_value
alias: total_sales_lead_value_aliased

seeds:
+quote_columns: "{{ true if target.type == 'redshift' else false }}"
+docs:
show: false

dispatch:
- macro_namespace: dbt_utils
Expand Down
8 changes: 4 additions & 4 deletions integration_tests/seeds/tiktok_ad_history_data.csv
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
AD_ID,UPDATED_AT,ADVERTISER_ID,ADGROUP_ID,CAMPAIGN_ID,CREATE_TIME,AD_NAME,CALL_TO_ACTION,SECONDARY_STATUS,OPERATION_STATUS,AD_TEXT,VIDEO_ID,APP_NAME,DEEPLINK,LANDING_PAGE_URL,DISPLAY_NAME,PROFILE_IMAGE_URL,IMPRESSION_TRACKING_URL,CLICK_TRACKING_URL,PLAYABLE_URL,IS_ACO,CREATIVE_AUTHORIZED,IS_NEW_STRUCTURE,IMAGE_IDS,_FIVETRAN_SYNCED
1,2021-09-15 04:26:28.000,1,1,1,2021-08-27 14:13:57.000,ad name one,SIGN_UP,AD_STATUS_DELIVERY_OK,DISABLE,subscribe,v10033g50000c4kf54jc77ucvou6fv0g,,,https://www.stuffthatworks.health/,health,,,,,TRUE,FALSE,TRUE,[ 124938917 ],2021-09-15 15:53:16.797
2,2021-09-15 03:19:25.000,2,2,2,2021-09-13 18:54:11.000,ad name two,SIGN_UP,AD_STATUS_DELIVERY_OK,ENABLE,join us,v10033g50000c4vnvmbc77u58bhlklvg,,,https://swaggystocks.com/dashboard/home,stocks,,,,,TRUE,FALSE,TRUE,[ 124938917 ],2021-09-15 15:53:16.641
3,2021-09-15 03:54:51.000,3,3,3,2021-07-26 16:28:39.000,ad name three,INTERESTED,AD_STATUS_DELIVERY_OK,ENABLE,like and share,v10033g50000c3qbo6jc77uaacd9hvv0,,,https://finviz.com/screener.ashx,more stocks,,,,,TRUE,FALSE,TRUE,[ 124938917 ],2021-09-15 15:53:18.918
4,2021-09-15 04:26:26.000,4,4,4,2021-08-05 21:02:33.000,ad name four,SUBSCRIBE,AD_STATUS_DELIVERY_OK,DISABLE,get stuck in,v10033g50000c3qbo6jc77ucp113o3qg,,,linkedin.com,linkedin,,,,,TRUE,FALSE,TRUE,[ 124938917 ],2021-09-15 15:53:18.897
1,2021-09-15 04:26:28.000,1,1,1,2021-08-27 14:13:57.000,ad name one,SIGN_UP,AD_STATUS_DELIVERY_OK,DISABLE,subscribe,v10033g50000c4kf54jc77ucvou6fv0g,,,landing_page,health,,,,,TRUE,FALSE,TRUE,[ 124938917 ],2021-09-15 15:53:16.797
2,2021-09-15 03:19:25.000,2,2,2,2021-09-13 18:54:11.000,ad name two,SIGN_UP,AD_STATUS_DELIVERY_OK,ENABLE,join us,v10033g50000c4vnvmbc77u58bhlklvg,,,landing_page,stocks,,,,,TRUE,FALSE,TRUE,[ 124938917 ],2021-09-15 15:53:16.641
3,2021-09-15 03:54:51.000,3,3,3,2021-07-26 16:28:39.000,ad name three,INTERESTED,AD_STATUS_DELIVERY_OK,ENABLE,like and share,v10033g50000c3qbo6jc77uaacd9hvv0,,,landing_page,more stocks,,,,,TRUE,FALSE,TRUE,[ 124938917 ],2021-09-15 15:53:18.918
4,2021-09-15 04:26:26.000,4,4,4,2021-08-05 21:02:33.000,ad name four,SUBSCRIBE,AD_STATUS_DELIVERY_OK,DISABLE,get stuck in,v10033g50000c3qbo6jc77ucp113o3qg,,,landing_page,linkedin,,,,,TRUE,FALSE,TRUE,[ 124938917 ],2021-09-15 15:53:18.897
4 changes: 2 additions & 2 deletions integration_tests/seeds/tiktok_advertiser_data.csv
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
ID,NAME,ADDRESS,COMPANY,CONTACTER,COUNTRY,CURRENCY,DESCRIPTION,EMAIL,INDUSTRY,LICENSE_NO,LICENSE_URL,PROMOTION_AREA,REJECTED_REASON,ROLE,STATUS,TELEPHONE_NUMBER,TIMEZONE,BALANCE,CREATE_TIME,LANGUAGE,CELLPHONE_NUMBER,_FIVETRAN_SYNCED
1,US,1600 pennsylvania avenue,White House,,US,USD,hello,[email protected],123,,,111,,ROLE_ADVERTISER,STATUS_ENABLE,111-222-3333,Etc/GMT+5,32.12,2021-08-27 18:13:41.000,en,001-002-0003,2021-09-15 15:53:18.938
2,Canada,21 w wacker st,Blanco House,,CAN,CAN,world,[email protected],321,,,0,,ROLE_ADVERTISER,STATUS_ENABLE,222-333-4444,Etc/GMT+5,321.22,2021-03-09 20:55:37.000,en,001-003-0004,2021-09-15 15:53:07.621
1,US,222 address,Comany X,,US,USD,hello,fake-email,123,,,111,,ROLE_ADVERTISER,STATUS_ENABLE,111-222-3333,Etc/GMT+5,32.12,2021-08-27 18:13:41.000,en,001-002-0003,2021-09-15 15:53:18.938
2,Canada,111 address,Company Y,,CAN,CAN,world,fake-email,321,,,0,,ROLE_ADVERTISER,STATUS_ENABLE,222-333-4444,Etc/GMT+5,321.22,2021-03-09 20:55:37.000,en,001-003-0004,2021-09-15 15:53:07.621
11 changes: 11 additions & 0 deletions integration_tests/seeds/tiktok_campaign_country_report_data.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
_fivetran_synced,campaign_id,clicks,conversion,conversion_rate,cost_per_conversion,country_code,cpc,cpm,ctr,impressions,real_time_conversion,spend,stat_time_day
2025-04-21 10:00:00,1,220,70,0.318,2.571,US,0.730,9.800,0.022,10000,65,154.60,2025-04-15 00:00:00
2025-04-21 10:00:00,2,210,65,0.310,2.615,CA,0.720,10.000,0.021,9800,60,150.00,2025-04-16 00:00:00
2025-04-21 10:00:00,3,180,58,0.322,2.759,GB,0.700,9.700,0.019,9500,55,138.20,2025-04-17 00:00:00
2025-04-21 10:00:00,4,200,66,0.330,2.727,AU,0.740,10.300,0.020,9900,62,145.90,2025-04-18 00:00:00
2025-04-21 10:00:00,1,230,72,0.313,2.639,US,0.760,10.400,0.023,10000,70,165.60,2025-04-19 00:00:00
2025-04-21 10:00:00,2,195,61,0.313,2.623,CA,0.710,9.600,0.020,9700,58,140.00,2025-04-20 00:00:00
2025-04-21 10:00:00,3,185,59,0.319,2.712,GB,0.690,9.800,0.019,9600,56,135.00,2025-04-10 00:00:00
2025-04-21 10:00:00,4,205,68,0.332,2.647,AU,0.750,10.100,0.021,9800,64,145.90,2025-04-11 00:00:00
2025-04-21 10:00:00,1,215,69,0.321,2.623,US,0.740,10.200,0.022,9900,66,150.90,2025-04-12 00:00:00
2025-04-21 10:00:00,2,225,71,0.316,2.676,CA,0.770,10.500,0.023,10000,68,160.00,2025-04-13 00:00:00
10 changes: 5 additions & 5 deletions integration_tests/seeds/tiktok_campaign_history_data.csv
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
AD_ID,UPDATED_AT,ADVERTISER_ID,ADGROUP_ID,CAMPAIGN_ID,CREATE_TIME,AD_NAME,CALL_TO_ACTION,SECONDARY_STATUS,OPERATION_STATUS,AD_TEXT,VIDEO_ID,APP_NAME,DEEPLINK,LANDING_PAGE_URL,DISPLAY_NAME,PROFILE_IMAGE_URL,IMPRESSION_TRACKING_URL,CLICK_TRACKING_URL,PLAYABLE_URL,IS_ACO,CREATIVE_AUTHORIZED,IS_NEW_STRUCTURE,IMAGE_IDS,_FIVETRAN_SYNCED
1,2021-09-15 04:26:28.000,1,1,1,2021-08-27 14:13:57.000,ad name one,SIGN_UP,AD_STATUS_DELIVERY_OK,DISABLE,subscribe,v10033g50000c4kf54jc77ucvou6fv0g,,,https://www.stuffthatworks.health/,health,,,,,TRUE,FALSE,TRUE,[ 124938917 ],2021-09-15 15:53:16.797
2,2021-09-15 03:19:25.000,2,2,2,2021-09-13 18:54:11.000,ad name two,SIGN_UP,AD_STATUS_DELIVERY_OK,ENABLE,join us,v10033g50000c4vnvmbc77u58bhlklvg,,,https://swaggystocks.com/dashboard/home,stocks,,,,,TRUE,FALSE,TRUE,[ 124938917 ],2021-09-15 15:53:16.641
3,2021-09-15 03:54:51.000,3,3,3,2021-07-26 16:28:39.000,ad name three,INTERESTED,AD_STATUS_DELIVERY_OK,ENABLE,like and share,v10033g50000c3qbo6jc77uaacd9hvv0,,,https://finviz.com/screener.ashx,more stocks,,,,,TRUE,FALSE,TRUE,[ 124938917 ],2021-09-15 15:53:18.918
4,2021-09-15 04:26:26.000,4,4,4,2021-08-05 21:02:33.000,ad name four,SUBSCRIBE,AD_STATUS_DELIVERY_OK,DISABLE,get stuck in,v10033g50000c3qbo6jc77ucp113o3qg,,,linkedin.com,linkedin,,,,,TRUE,FALSE,TRUE,[ 124938917 ],2021-09-15 15:53:18.897
CAMPAIGN_ID,UPDATED_AT,ADVERTISER_ID,CAMPAIGN_NAME,CAMPAIGN_TYPE,BUDGET,BUDGET_MODE,OPERATION_STATUS,OBJECTIVE_TYPE,IS_NEW_STRUCTURE,SPLIT_TEST_VARIABLE,CREATE_TIME,_FIVETRAN_SYNCED
1,2021-09-15 00:52:08.000,1,campaign name one,REGULAR_CAMPAIGN,450,BUDGET_MODE_DAY,ENABLE,CONVERSIONS,TRUE,,2021-08-16 20:40:15.000,2021-09-15 02:47:54.737
2,2021-09-15 01:33:37.000,2,campaign name two,REGULAR_CAMPAIGN,300,BUDGET_MODE_DAY,ENABLE,CONVERSIONS,TRUE,,2021-08-16 20:29:06.000,2021-09-15 02:47:54.737
3,2021-09-14 02:53:48.000,3,campaign name three,REGULAR_CAMPAIGN,450,BUDGET_MODE_DAY,ENABLE,CONVERSIONS,TRUE,,2021-08-16 20:40:15.000,2021-09-14 03:01:03.005
4,2021-09-14 02:59:01.000,4,campaign name four,REGULAR_CAMPAIGN,300,BUDGET_MODE_DAY,ENABLE,CONVERSIONS,TRUE,,2021-08-16 20:29:06.000,2021-09-14 03:01:03.006
59 changes: 59 additions & 0 deletions integration_tests/tests/integrity/vertical_sum_country_report.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
{{ config(
tags="fivetran_validations",
enabled=var('fivetran_validation_tests_enabled', false) and var('tiktok_ads__using_campaign_country_report', true)
) }}

with staging as (
select
campaign_id,
count(campaign_id) as count_campaign,
count(distinct stat_time_day) as count_unique_days,
sum(impressions) as total_impressions,
sum(clicks) as total_clicks,
sum(spend) as total_spend,
sum(conversion) as total_conversion,
from {{ ref('stg_tiktok_ads__campaign_country_report') }}
group by 1
),

end_model as (
select
campaign_id,
count(campaign_id) as count_campaign,
count(distinct date_day) as count_unique_days,
sum(impressions) as total_impressions,
sum(clicks) as total_clicks,
sum(spend) as total_spend,
sum(conversion) as total_conversion,
from {{ ref('tiktok_ads__campaign_country_report') }}
group by 1
),

combined as (
select
end_model.campaign_id,
end_model.count_campaign as end_count_campaign,
staging.count_campaign as staging_count_campaign,
end_model.count_unique_days as end_count_unique_days,
staging.count_unique_days as staging_count_unique_days,
end_model.total_impressions as end_total_impressions,
staging.total_impressions as staging_total_impressions,
end_model.total_clicks as end_total_clicks,
staging.total_clicks as staging_total_clicks,
end_model.total_spend as end_total_spend,
staging.total_spend as staging_total_spend,
end_model.total_conversion as end_total_conversion,
staging.total_conversion as staging_total_conversion
from end_model
full outer join staging
on end_model.campaign_id = staging.campaign_id
)

select *
from combined
where end_count_campaign != staging_count_campaign or
end_count_unique_days != staging_count_unique_days or
end_total_impressions != staging_total_impressions or
end_total_clicks != staging_total_clicks or
end_total_spend != staging_total_spend or
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With monetary floats, I've always noticed that the numbers can deviate by an insigifcant amount to throw errors that would not be logged in the financial reports. Perhaps this would be better:

Suggested change
end_total_spend != staging_total_spend or
abs(end_total_spend - staging_total_spend) > 0.01 or

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good approach and helps to future proof any false positives. Added.

end_total_conversion != staging_total_conversion
Loading