Skip to content

Commit 4e8dad7

Browse files
eric-laurellodesmond-hui
eric-laurello
and
desmond-hui
authored
An 4127/add standard fields (#230)
* append new cols setting * wip * add standard fields for labels * add standard fields for ntr * add standard fields for observability * add standard fields for tags * add standard field descriptions * add standard fields for core * add standard fields for cosmos * add standard fields for defi * add standard fields for price and ens * update fields based on upstream new cols * fix syntax/ambiguous col issues * fix typos * fix/verify uniqueness core * fix/verify uniqueness cosmos * fix/verify uniqueness cosmos * wip * pk cleanup --------- Co-authored-by: Desmond Hui <[email protected]>
1 parent 51f7657 commit 4e8dad7

File tree

145 files changed

+4909
-3246
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

145 files changed

+4909
-3246
lines changed

‎dbt_project.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ models:
3939
+persist_docs:
4040
relation: true
4141
columns: true
42+
+on_schema_change: "append_new_columns"
4243

4344
# In this example config, we tell dbt to build all models in the example/ directory
4445
# as tables. These settings can be overridden in the individual model files

‎models/descriptions/inserted_timestamp.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{% docs inserted_timestamp %}
2+
3+
The utc timestamp at which the row was inserted into the table.
4+
5+
{% enddocs %}

‎models/descriptions/modified_timestamp.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{% docs modified_timestamp %}
2+
3+
The utc timestamp at which the row was last modified.
4+
5+
{% enddocs %}

‎models/descriptions/pk.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{% docs pk %}
2+
3+
The unique identifier for each row in the table.
4+
5+
{% enddocs %}

‎models/gold/core/core__dim_contracts.sql

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,10 @@ SELECT
1313
created_block_timestamp,
1414
created_tx_hash,
1515
creator_address,
16-
'ethereum' AS blockchain
16+
blockchain,
17+
inserted_timestamp,
18+
modified_timestamp,
19+
dim_contracts_id
1720
FROM
1821
{{ ref(
1922
'silver__contracts'

‎models/gold/core/core__dim_contracts.yml

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,4 +15,12 @@ models:
1515
- name: DECIMALS
1616
description: The number of decimal places this contract needs adjusted where token values exist.
1717
- name: BLOCKCHAIN
18-
description: The corresponding blockchain for contract details.
18+
description: The corresponding blockchain for contract details.
19+
- name: DIM_CONTRACTS_ID
20+
description: '{{ doc("pk") }}'
21+
tests:
22+
- unique
23+
- name: INSERTED_TIMESTAMP
24+
description: '{{ doc("inserted_timestamp") }}'
25+
- name: MODIFIED_TIMESTAMP
26+
description: '{{ doc("modified_timestamp") }}'

‎models/gold/core/core__dim_evm_event_abis.sql

Lines changed: 14 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -14,21 +14,22 @@
1414
%}
1515
SELECT *
1616
FROM (
17-
{% for models in models %}
17+
{% for models in models %}
1818
SELECT
19-
'{{ models[0] }}' AS blockchain,
20-
parent_contract_address,
21-
event_name,
22-
abi,
23-
simple_event_name,
24-
event_signature,
25-
start_block,
26-
end_block
19+
'{{ models[0] }}' AS blockchain,
20+
parent_contract_address,
21+
event_name,
22+
abi,
23+
simple_event_name,
24+
event_signature,
25+
start_block,
26+
end_block,
27+
COALESCE(inserted_timestamp,'2000-01-01') as inserted_timestamp,
28+
COALESCE(modified_timestamp,'2000-01-01') as modified_timestamp,
29+
{{ dbt_utils.generate_surrogate_key(['parent_contract_address','event_signature','start_block','blockchain']) }} AS dim_evm_event_abis_id
2730
FROM {{ models[1] }}
2831
{% if not loop.last %}
29-
{% if is_incremental() %}
30-
{% endif %}
3132
UNION ALL
3233
{% endif %}
33-
{% endfor %}
34-
)
34+
{% endfor %}
35+
)

‎models/gold/core/core__dim_evm_event_abis.yml

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,4 +19,12 @@ models:
1919
- name: START_BLOCK
2020
description: The block number of the creation of the contract containing the event. This could be from a proxy or an implementation.
2121
- name: END_BLOCK
22-
description: The block number before a similar event was created in the contract. This could be from a proxy or an implementation.
22+
description: The block number before a similar event was created in the contract. This could be from a proxy or an implementation.
23+
- name: DIM_EVM_EVENT_ABIS_ID
24+
description: '{{ doc("pk") }}'
25+
tests:
26+
- unique
27+
- name: INSERTED_TIMESTAMP
28+
description: '{{ doc("inserted_timestamp") }}'
29+
- name: MODIFIED_TIMESTAMP
30+
description: '{{ doc("modified_timestamp") }}'

‎models/gold/core/core__dim_labels.sql

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,10 @@ SELECT
1212
case when label_type = 'layer2' then 'bridge' else label_type end as label_type,
1313
label_subtype,
1414
address_name,
15-
project_name
15+
project_name,
16+
COALESCE(inserted_timestamp,'2000-01-01') as inserted_timestamp,
17+
COALESCE(modified_timestamp,'2000-01-01') as modified_timestamp,
18+
COALESCE(address_labels_id,{{ dbt_utils.generate_surrogate_key(['blockchain','creator','address']) }}) AS dim_labels_id
1619
FROM
1720
{{ ref('silver__address_labels') }}
1821
where delete_flag is null
@@ -26,7 +29,10 @@ SELECT
2629
case when label_type = 'layer2' then 'bridge' else label_type end as label_type,
2730
label_subtype,
2831
address_name,
29-
project_name
32+
project_name,
33+
COALESCE(inserted_timestamp,'2000-01-01') as inserted_timestamp,
34+
COALESCE(modified_timestamp,'2000-01-01') as modified_timestamp,
35+
COALESCE(deposit_wallets_id,{{ dbt_utils.generate_surrogate_key(['blockchain','creator','address']) }}) AS dim_labels_id
3036
FROM
3137
{{ ref('silver__deposit_wallets') }}
3238
UNION ALL
@@ -39,7 +45,10 @@ SELECT
3945
case when label_type = 'layer2' then 'bridge' else label_type end as label_type,
4046
label_subtype,
4147
address_name,
42-
project_name
48+
project_name,
49+
COALESCE(inserted_timestamp,'2000-01-01') as inserted_timestamp,
50+
COALESCE(modified_timestamp,'2000-01-01') as modified_timestamp,
51+
COALESCE(contract_autolabels_id,{{ dbt_utils.generate_surrogate_key(['blockchain','creator','address']) }}) AS dim_labels_id
4352
FROM
4453
{{ ref('silver__contract_autolabels') }}
4554
UNION ALL
@@ -52,7 +61,10 @@ SELECT
5261
case when label_type = 'layer2' then 'bridge' else label_type end as label_type,
5362
label_subtype,
5463
address_name,
55-
project_name
64+
project_name,
65+
COALESCE(inserted_timestamp,'2000-01-01') as inserted_timestamp,
66+
COALESCE(modified_timestamp,'2000-01-01') as modified_timestamp,
67+
COALESCE(labels_eth_contracts_table_id,{{ dbt_utils.generate_surrogate_key(['address']) }}) AS dim_labels_id
5668
FROM
5769
{{ ref('silver__labels_eth_contracts_table') }}
5870

‎models/gold/core/core__dim_labels.yml

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,4 +48,12 @@ models:
4848
enabled: False
4949
- dbt_expectations.expect_column_value_lengths_to_equal:
5050
value: 58
51-
where: BLOCKCHAIN = 'algorand'
51+
where: BLOCKCHAIN = 'algorand'
52+
- name: DIM_LABELS_ID
53+
description: '{{ doc("pk") }}'
54+
tests:
55+
- unique
56+
- name: INSERTED_TIMESTAMP
57+
description: '{{ doc("inserted_timestamp") }}'
58+
- name: MODIFIED_TIMESTAMP
59+
description: '{{ doc("modified_timestamp") }}'

0 commit comments

Comments
 (0)