8.6.0: SQD Network datasource updates#1318
Open
Wizard1209 wants to merge 12 commits into
Open
Conversation
Self-hosted `v2.archive.subsquid.io` gateways require an API key since 2026-05-19 (`/<level>/worker` returns 403 without one, while `/height` still 200), breaking self-hosted EVM/Starknet/Substrate subsquid indexers at the data-fetch step.
Add an `api_key` option to evm/starknet/substrate.subsquid datasource configs and send it as `Authorization: Bearer <key>` + `Token: <key>` (matching squid-sdk's ArchiveClient) on the gateway (`/height`, `/<level>/worker`) and worker POST requests. Demos, templates and test configs gain `api_key: ${SUBSQUID_API_KEY:-}` (empty default => no header). Includes docs, regenerated JSON schema, CHANGELOG and a test skip when SUBSQUID_API_KEY is unset.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Un-ignore CLAUDE.md (CLAUDE.local.md stays ignored) and document the script-driven workflow: generated artifacts (demos, JSON schema, references), the docs include-macro exclusive-end footgun, and conventions via @import of docs/14.contributing.md instead of duplicating README/docs/Makefile. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduce a shared `SubsquidDatasourceConfig` base holding `api_key` and the common `merge_subscriptions`/`rollback_depth` properties; bind the abstract datasource to it so both worker and datasource read `self._config.api_key` directly (no `getattr`, no `Any`). Make the `api_key` docstring consistent with other datasources and drop the inaccurate "self-hosted" framing (v2.archive is SQD's public gateway). Regenerate schema and config reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Wizard1209 <wizard1209@gmail.com>
Ghostnet was decommissioned; api.ghostnet.tzkt.io no longer resolves, so the demo_tezos_events run/init tests and test_tzkt.test_no_content all failed with DNS errors (not an infra flake). Shadownet is the current Tezos testnet. - tezos_tzkt: swap the ghostnet entry in TZKT_API_URLS for shadownet - demo_tezos_events: repoint the events demo at a shadownet contract with equivalent properties — KT1E3EQJmoajoTj9YXJWgenuYTHnDUoWSiFt (WETH bridge) emits two frequent typed events (Wrap/Unwrap, like move/roll) plus a rare Create caught by the fallback handler. Regenerated handlers and types via `dipdup init`; move/roll -> wrap/unwrap. - tests: point demo_tezos_events config and test_no_content at shadownet - docs: update tezos.events include to on_wrap_event.py - regenerate merged changelog (was stale, missing the Portal/api_key Added entries) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Release prep generated by `make before_release`: regenerated demos, docs references, demos table, JSON schema, and the v8.6 release-notes page. Also includes two fixes surfaced while making the release green: - sentry: crash on init with sentry-sdk 2.61+ (`DEFAULT_MAX_VALUE_LENGTH` is now `None`) - tests: mypy arg-type error in test_models_evm.py Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
droserasprout
approved these changes
Jun 2, 2026
Many gateways (e.g. nginx limit_req) shed load with 503 instead of 429, so apply ratelimit_sleep backoff for both statuses. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sentry's event_from_exception omits the stacktrace for exceptions that have no traceback (e.g. a chained __context__/__cause__ cause that was constructed but never raised), so indexing exception['stacktrace'] raised KeyError: 'stacktrace' inside the crash reporter, masking the real error. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
run_postgres_container/run_hasura_container now publish a random host port and the host connects via 127.0.0.1, instead of the container bridge IP which isn't routable from a WSL2 distro under Docker Desktop (the connection hung forever). Container-to-container traffic (Hasura->Postgres) still uses the bridge IP. Adds a host-side readiness wait with a connect timeout so a networking problem fails fast. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Subsquid decommissioned the starknet-mainnet v2.archive dataset (now 404 on every path) and the SQD Portal has no starknet dataset either, so the demo can no longer reach an archive. Drop the subsquid datasource and index from the starknet node only. Also drop starknet from the SUBSQUID_API_KEY skip-guard in test_demos.py since it no longer needs an archive key. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Minor release 8.6.0 — SQD Network datasource updates + subsquid
api_keysupport. This branch is the full scope of 8.6.0.Part 1 —
api_keyfor v2.archive subsquidLegacy
v2.archive.subsquid.iogateways require an API key since 2026-05-19api_keyoption onevm.subsquid/starknet.subsquid/substrate.subsquid.Authorization: Bearer <key>andToken: <key>(matching squid-sdk'sArchiveClient).api_key: ${SUBSQUID_API_KEY:-}(empty default → no header).Part 2 —
evm.sqd_portaldatasourceNew
evm.sqd_portalkind backed by the SQD Portal (portal.sqd.dev), a streaming (/finalized-stream, NDJSON) alternative to the v2.archiveevm.subsquidgateway, reusing the existingiter_events/iter_transactions. Addscontext.get_evm_portal_datasource.🤖 Generated with Claude Code