Skip to content

Commit 50fbb26

Browse files
Wizard1209claudeWizard1209
authored
evm.sqd_portal: add SQD Portal datasource (EVM) (#1319)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Wizard1209 <wizard1209@gmail.com>
1 parent 914630c commit 50fbb26

37 files changed

Lines changed: 736 additions & 86 deletions

‎CHANGELOG.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ Releases prior to 7.0 has been removed from this file to declutter search result
1010

1111
### Added
1212

13+
- context: Added `get_evm_portal_datasource` method to access `evm.sqd_portal` datasources from handlers and hooks with a precise type.
14+
- evm.sqd_portal: Added a new datasource backed by the SQD Portal (`portal.sqd.dev`), a streaming alternative to the v2.archive `evm.subsquid` gateway.
1315
- evm.subsquid, starknet.subsquid, substrate.subsquid: Added `api_key` option to authenticate with `v2.archive.subsquid.io` gateways, which require a key since 2026-05-19.
1416

1517
## [8.5.2] - 2026-04-08

‎docs/1.getting-started/7.datasources.md‎

Lines changed: 18 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -9,23 +9,24 @@ Datasources are DipDup connectors to various APIs. They are defined in config an
99

1010
Index datasources, ones that can be attached to a specific index, are prefixed with blockchain name, e.g. `tezos.tzkt` or `evm.subsquid`.
1111

12-
| kind | blockchain | description |
13-
| --------------------------------------------------------------- | ---------------- | ----------------------------------------------- |
14-
| [evm.subsquid](../3.datasources/1.evm_subsquid.md) | ⟠ EVM-compatible | Subsquid Network API |
15-
| [evm.node](../3.datasources/2.evm_node.md) | ⟠ EVM-compatible | Ethereum node |
16-
| [evm.etherscan](../3.datasources/3.evm_etherscan.md) | ⟠ EVM-compatible | Provides ABIs for EVM contracts |
17-
| [evm.blockvision](../3.datasources/4.evm_blockvision.md) | ⟠ EVM-compatible | Provides ABIs for EVM contracts |
18-
| [evm.sourcify](../3.datasources/5.evm_sourcify.md) | ⟠ EVM-compatible | Provides ABIs for EVM contracts |
19-
| [starknet.subsquid](../3.datasources/6.starknet_subsquid.md) | 🐺 Starknet | Subsquid Network API |
20-
| [starknet.node](../3.datasources/7.starknet_node.md) | 🐺 Starknet | Starknet node |
21-
| [substrate.node](../3.datasources/8.substrate_node.md) | 🔮 Substrate | Substrate node |
22-
| [substrate.subscan](../3.datasources/9.substrate_subscan.md) | 🔮 Substrate | Provides pallet metadata for Substrate networks |
23-
| [substrate.subsquid](../3.datasources/10.substrate_subsquid.md) | 🔮 Substrate | Subsquid Network API |
24-
| [tezos.tzkt](../3.datasources/11.tezos_tzkt.md) | ꜩ Tezos | TzKT API |
25-
| [tzip_metadata](../3.datasources/12.tzip_metadata.md) | ꜩ Tezos | TZIP-16 metadata |
26-
| [coinbase](../3.datasources/13.coinbase.md) | any | Coinbase price feed |
27-
| [ipfs](../3.datasources/14.ipfs.md) | any | IPFS gateway |
28-
| [http](../3.datasources/15.http.md) | any | Generic HTTP API |
12+
| kind | blockchain | description |
13+
| ----------------------------------------------------------------------- | ---------------- | ----------------------------------------------- |
14+
| [evm.sqd_portal](../3.datasources/1.evm_subsquid.md#sqd-portal) | ⟠ EVM-compatible | SQD Network Portal (recommended) |
15+
| [evm.subsquid](../3.datasources/1.evm_subsquid.md#v2-archive-gateway) | ⟠ EVM-compatible | SQD Network v2.archive gateway (legacy) |
16+
| [evm.node](../3.datasources/2.evm_node.md) | ⟠ EVM-compatible | Ethereum node |
17+
| [evm.etherscan](../3.datasources/3.evm_etherscan.md) | ⟠ EVM-compatible | Provides ABIs for EVM contracts |
18+
| [evm.blockvision](../3.datasources/4.evm_blockvision.md) | ⟠ EVM-compatible | Provides ABIs for EVM contracts |
19+
| [evm.sourcify](../3.datasources/5.evm_sourcify.md) | ⟠ EVM-compatible | Provides ABIs for EVM contracts |
20+
| [starknet.subsquid](../3.datasources/6.starknet_subsquid.md) | 🐺 Starknet | Subsquid Network API |
21+
| [starknet.node](../3.datasources/7.starknet_node.md) | 🐺 Starknet | Starknet node |
22+
| [substrate.node](../3.datasources/8.substrate_node.md) | 🔮 Substrate | Substrate node |
23+
| [substrate.subscan](../3.datasources/9.substrate_subscan.md) | 🔮 Substrate | Provides pallet metadata for Substrate networks |
24+
| [substrate.subsquid](../3.datasources/10.substrate_subsquid.md) | 🔮 Substrate | Subsquid Network API |
25+
| [tezos.tzkt](../3.datasources/11.tezos_tzkt.md) | ꜩ Tezos | TzKT API |
26+
| [tzip_metadata](../3.datasources/12.tzip_metadata.md) | ꜩ Tezos | TZIP-16 metadata |
27+
| [coinbase](../3.datasources/13.coinbase.md) | any | Coinbase price feed |
28+
| [ipfs](../3.datasources/14.ipfs.md) | any | IPFS gateway |
29+
| [http](../3.datasources/15.http.md) | any | Generic HTTP API |
2930

3031
## Connection settings
3132

‎docs/3.datasources/1.evm_subsquid.md‎

Lines changed: 38 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,57 @@
11
---
2-
title: "EVM: Subsquid"
3-
description: "DipDup uses Subsquid Network as a source of historical data for EVM-compatible blockchains."
2+
title: "EVM: SQD Network"
3+
description: "DipDup uses SQD Network (formerly Subsquid) as a source of historical data for EVM-compatible blockchains, served by the SQD Portal."
44
network: "ethereum"
55
---
66

7-
# Subsquid Network
7+
# SQD Network
88

99
{{ #include 3.datasources/_evm_banner.md }}
1010

1111
{{ #include 3.datasources/_subsquid_banner.md }}
1212

13-
DipDup uses [Subsquid Network](https://docs.subsquid.io/subsquid-network/reference/evm-api/) as a source of historical data for EVM-compatible blockchains.
13+
DipDup uses [SQD Network](https://docs.sqd.dev) as a source of historical data for EVM-compatible blockchains. The recommended way to access it is the **SQD Portal** (`evm.sqd_portal`); the older `v2.archive` gateway (`evm.subsquid`) is still supported but [legacy](#v2-archive-gateway). DipDup uses the configured gateway when possible and falls back to EVM nodes for the latest data and realtime updates.
1414

15-
Configure the datasource in your project config:
15+
## SQD Portal
16+
17+
[SQD Portal](https://portal.sqd.dev) is the current way to access SQD Network: a single streaming endpoint (NDJSON), keyless for public datasets. Configure it with the `evm.sqd_portal` kind and the per-dataset Portal URL:
1618

1719
```yaml [dipdup.yaml]
18-
{{ #include ../src/demo_evm_events/dipdup.yaml:3:9 }}
20+
datasources:
21+
sqd_portal:
22+
kind: evm.sqd_portal
23+
url: https://portal.sqd.dev/datasets/ethereum-mainnet
24+
# api_key: ${SQD_PORTAL_API_KEY:-} # optional
1925
```
2026

21-
`v2.archive.subsquid.io` gateways [require an API key](https://docs.sqd.dev/v2-keys) since 2026-05-19. Provide it via the `api_key` field; SQD Cloud users are unaffected.
27+
## Adding to an index
2228

23-
Then, add it to EVM index definitions:
29+
Reference the datasource by name in your EVM index definitions. Other datasources — Etherscan for ABIs, an EVM node for realtime data — can be listed alongside it:
2430

2531
```yaml [dipdup.yaml]
26-
{{ #include ../src/demo_evm_events/dipdup.yaml:23: }}
32+
indexes:
33+
eth_usdt_events:
34+
kind: evm.events
35+
datasources:
36+
- sqd_portal
37+
- etherscan
38+
- evm_node
39+
handlers:
40+
- callback: on_transfer
41+
contract: eth_usdt
42+
name: Transfer
2743
```
2844
29-
DipDup will use Subsquid Network when possible and fallback to EVM nodes for the latest data and realtime updates.
45+
## v2 archive gateway
46+
47+
::banner{type="warning"}
48+
The `v2.archive` gateway (`evm.subsquid`) is legacy and scheduled for sunset in 2026. Prefer the SQD Portal above for new projects.
49+
::
50+
51+
The original worker-based `v2.archive` gateway uses the `evm.subsquid` kind:
52+
53+
```yaml [dipdup.yaml]
54+
{{ #include ../src/demo_evm_events/dipdup.yaml:3:9 }}
55+
```
56+
57+
`v2.archive.subsquid.io` gateways [require an API key](https://docs.sqd.dev/v2-keys) since 2026-05-19. Provide it via the `api_key` field; SQD Cloud users are unaffected.

‎docs/3.datasources/10.substrate_subsquid.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@ network: "substrate"
88

99
{{ #include 3.datasources/_subsquid_banner.md }}
1010

11+
{{ #include 3.datasources/_v2archive_sunset_banner.md }}
12+
1113
DipDup uses [Subsquid Network](https://docs.subsquid.io/subsquid-network/reference/evm-api/) as a source of historical data for Substrate blockchains.
1214

1315
Configure the datasource in your project config:

‎docs/3.datasources/6.starknet_subsquid.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@ network: "starknet"
88

99
{{ #include 3.datasources/_subsquid_banner.md }}
1010

11+
{{ #include 3.datasources/_v2archive_sunset_banner.md }}
12+
1113
DipDup uses [Subsquid Network](https://docs.subsquid.io/subsquid-network/reference/starknet-api/) as a source of historical data for Starknet blockchain.
1214

1315
Configure the datasource in your project config:

‎docs/3.datasources/_subsquid_banner.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
<!-- markdownlint-disable first-line-h1 -->
22
::banner{type="note"}
3-
Subsquid Network [has been rebranded](https://blog.sqd.dev/subsquid-becomes-sqd-meet-our-new-self/) to SQD Network. We keep using old name in documentation and config specification for compatibility reasons.
3+
Subsquid Network has been [rebranded](https://blog.sqd.dev/subsquid-becomes-sqd-meet-our-new-self/) to **SQD Network**. The `subsquid` datasource kinds (e.g. `evm.subsquid`) keep the original name for backward compatibility.
44
::

‎docs/3.datasources/_v2archive_sunset_banner.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
<!-- markdownlint-disable first-line-h1 -->
2+
::banner{type="warning"}
3+
The `v2.archive.subsquid.io` gateways are legacy and scheduled for sunset in 2026.
4+
::

0 commit comments

Comments
 (0)