Skip to content

feat(data): add link and connection pooling composites for v12 CLI - #184

Merged
eablack merged 5 commits into
mainfrom
eb/W-23387136/link-pooling-extensions
Nov 7, 2026
Merged

eablack merged 5 commits into
mainfrom
eb/W-23387136/link-pooling-extensions

Conversation

@eablack

@eablack eablack commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds SDK composites so the v12 CLI can move pg:links, pg:links:create, pg:links:destroy, and pg:connection-pooling:attach to @heroku/sdk without resolving databases itself. The CLI passes raw app / database / remote input. These methods resolve internally with the private resolvePgDatabase, the same way data.database.describe(app, database?) does today.

New methods (the names don't reuse the generated link.{list,create,delete,info} or connectionPooling.{info,create} route names, so those routes stay callable):

Method Behavior
data.postgresDatabase.listLegacyByApp(app) Lists the app's attached legacy (non-Advanced) Postgres databases. Each database appears once, with all its attachmentNames, in API order. Includes Essential-tier databases and databases owned by other apps. Mirrors DatabaseResolver.getAllLegacyDatabases.
data.link.listByApp(app, database?) With database, resolves that one database. Without it, uses listLegacyByApp. Then fetches each database's links in parallel. Returns [] when the app has no databases.
data.link.createForDatabase(app, database, remote, {as}, {beforeCreate}) Resolves the database and the remote in parallel. The remote resolves scoped to the app first; it falls back to a global lookup only when the add-on itself is missing. A 404 for the app is rethrown. The remote's service must be exactly the Postgres service name, since Heroku Data Links connects Postgres to Postgres only. Any other remote, including Heroku Key-Value Store, throws InvalidLinkRemoteError. Essential tier is rejected on either side. Then runs beforeCreate({addon, target}), followed by link.create(addon.id, {as, target}).
data.link.deleteFromDatabase(app, database, link, {beforeDelete}) Resolves the database and rejects Essential tier. Runs beforeDelete({addon, link}), where the CLI prompts for confirmation; a throw or abort there prevents the delete. Then calls link.delete.
data.connectionPooling.attach(app, database?, {name, credential}, {onResolved, onAttached}) Resolves the database (defaults to DATABASE_URL) and rejects Essential tier. Runs onResolved(addon), then connectionPooling.create(addon.name, {app, name, credential}) (credential defaults to default), then onAttached(attachment). Then reads the newest release using a Range header, so no pagination. This lookup is best-effort: if it fails after the pool is attached, the call returns releaseVersion: undefined instead of rejecting. An abort still rejects.

Hook naming: before<Verb> hooks take a preview object and act as a gate, where a throw prevents the mutation. onResolved / on<PastTense> hooks are notifications.

Errors and types exported from @heroku/sdk/extensions/data:

  • New: EssentialDatabaseNotSupportedError (carries the add-on; modeled on NotAdvancedDatabaseError) and InvalidLinkRemoteError (carries the remote add-on). Both have a body getter.
  • Now exported: PgDatabaseNotFoundError and ResolvedPgDatabase, so callers can match not-found errors without importing internals.

Neither message mentions CLI commands. The CLI maps them to its existing strings.

createAttachment fix: when the app-scoped resolve matched only non-Postgres add-ons, resolveAddonInternal threw AddonNotFoundError instead of NotFoundError, so the global retry never ran. The scoped-then-global logic now lives in a single private helper, resolveAddonScopedThenGlobal, which createAttachment and link.createForDatabase both use. The helper retries only on AddonNotFoundError or a 404 labeled resource: 'add_on'. Any other 404, whether for the app or without a label, is rethrown so a mistyped app doesn't slip through. It resolves an app::addon identity in one request.

Attachment resolution fix: when an attachment lookup matches several attachments, resolveAddonAttachment now keeps only the default (un-namespaced) ones, as heroku-cli-util did. Previously, resolving a database by add-on name failed as ambiguous when the database also had a credential (credential:<name>) or connection-pooling attachment on the same app. If none of the matches is a default attachment, the error is still ambiguous. This applies to every command that resolves databases through the SDK, not only the new composites.

Open question: should these composites also reject Advanced databases? Right now they rely on the API to do that.

resolvePgDatabase stays private: @heroku/sdk/resources/data/internal/* still resolves to ERR_PACKAGE_PATH_NOT_EXPORTED.

Type of Change

Breaking Changes (major semver update)

  • Add a ! after your change type to denote a change that breaks current behavior

Feature Additions (minor semver update)

  • feat: Introduces a new feature to the codebase

Patch Updates (patch semver update)

  • fix: Bug fix
  • deps: Dependency upgrade
  • revert: Revert a previous commit
  • chore: Change that does not affect production code
  • refactor: Refactoring existing code without changing behavior
  • test: Add/update/remove tests

Testing

Notes:
You need an app with at least two Postgres databases, at least one non-Essential, plus a Heroku Key-Value Store add-on for the rejection check. You also need a second app that owns a non-Essential Postgres add-on. Build the branch (npm run build), then run the snippets below with HEROKU_API_KEY set.

import {HerokuSDK} from '@heroku/sdk'
import {connectionPoolingExtensions, linkExtensions, postgresDatabaseExtensions} from '@heroku/sdk/extensions/data'
const sdk = new HerokuSDK({extensions: [linkExtensions, connectionPoolingExtensions, postgresDatabaseExtensions]})

Steps:

  1. Run await sdk.data.postgresDatabase.listLegacyByApp('<app>'). Each database should appear once, with all its attachment names. Advanced databases shouldn't appear.
  2. Run await sdk.data.link.listByApp('<app>') and await sdk.data.link.listByApp('<app>', 'DATABASE_URL'). The results should match heroku pg:links -a <app> (v11).
  3. Run await sdk.data.link.createForDatabase('<app>', 'DATABASE_URL', '<other-postgres-addon-name>', {as: 'sdk_test'}). Confirm with heroku pg:links -a <app> that the link exists.
  4. Repeat step 3 with a remote add-on owned by the second app. It should resolve through the global retry.
  5. Repeat step 3 with a remote that isn't Postgres: a Heroku Key-Value Store add-on, then a Papertrail add-on. Both should throw InvalidLinkRemoteError before any link request is sent.
  6. Repeat step 3 with a mistyped app ('<app>-typo') and an other-app::DATABASE database. It should reject with a 404 for the app and must not create a link on the remote.
  7. Repeat step 3 with an Essential-tier database or remote. It should throw EssentialDatabaseNotSupportedError and no link should be created.
  8. Run await sdk.data.link.deleteFromDatabase('<app>', 'DATABASE_URL', 'sdk_test', {beforeDelete: () => { throw new Error('abort') }}). It should throw abort and leave the link in place. Then run it again without beforeDelete; the link should be gone.
  9. Run await sdk.data.connectionPooling.attach('<app>', undefined, {name: 'SDK_POOL'}). The attachment should be named SDK_POOL, and releaseVersion should equal the version shown by heroku releases -a <app> -n 1. Detach the pool before re-running: a rejection after onAttached leaves the attachment in place.
  10. While SDK_POOL is still attached, run await sdk.data.link.listByApp('<app>', '<database-addon-name>') using the pooled database's add-on name. It should resolve to the default attachment instead of throwing AddonAttachmentAmbiguousError. Then detach SDK_POOL.
  11. Check that sdk.data.link.list and sdk.data.connectionPooling.info still work, which confirms the generated routes weren't replaced.

Screenshots (if applicable)

Related Issues

GitHub issue: N/A
GUS work item: W-23387136

@eablack
eablack requested a review from a team as a code owner November 6, 2026 17:36

@jdodson jdodson left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI found a thing in testing (not sure how I'd test what you did manually)

Redis link support appears inconsistent with Heroku’s Data Links API. This PR accepts heroku-redis as a valid remote, but I tested link creation with Mini Redis both on the target app and on a second app. In both cases, SDK resolution succeeded, then Heroku’s link-create API returned 404: “Remote service … cannot be found or is not supported.”

For comparison, createForDatabase successfully created a link to a Standard-0 Postgres on a second app; listing and deleting the link also passed. That confirms the cross-app fallback works for Postgres.

The current Heroku Data Links docs (https://devcenter.heroku.com/articles/heroku-data-links) describe Postgres-to-Postgres links. Can you confirm whether any Key-Value Store plans are supported as link remotes? If not, the heroku-redis acceptance in link.ts should be removed and the PR/GUS expectations updated.

One blocker, some other AI suggestions.

  • Blocker: src/resources/data/internal/resolve-addon-scoped.ts:42 retries globally when a 404 lacks a resource label. If that response means the requested app is missing, link creation can proceed using a database from other-app::DATABASE. Fail closed on unclassified 404s, or prove the API always labels app-not-found responses.

  • Consider: connectionPooling.attach can reject after creating the pool if onAttached throws or the signal aborts. The attachment remains; callers should avoid blindly retrying and potentially creating a duplicate.

  • Consider: This PR provides the SDK prerequisite, not the full GUS story. Keep the work item open until the downstream CLI migration, characterization tests, and V12-CHANGES.md updates are completed.

@eablack

eablack commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, all four were useful. Pushed 0290b3d.

Blocker: unlabeled 404 retry. Fixed: resolveAddonScopedThenGlobal now retries globally only on an empty match or a 404 labeled resource: 'add_on'. Any other 404 is rethrown. For the record, I checked the live API before changing it. POST /actions/addons/resolve labels a missing app resource: "app" (whether or not the add-on name exists), and a missing add-on resource: "add_on", both scoped and global. Also, createForDatabase resolves the local database against the same app in parallel, so a missing app already failed before link.create. So this is defense in depth, not a behavior change. The new case is in resolve-addon-scoped.test.ts.

Redis remotes. Agreed and removed. The current Data Links article describes Postgres-to-Postgres links only, and your Mini test matches. createForDatabase now accepts only Postgres remotes and throws InvalidLinkRemoteError for Key-Value Store before calling the API. I updated the error message, docs, tests, and the PR description. The CLI branch makes the same change, including the pg:links:create error and examples, with an entry in V12-CHANGES.md. If anyone knows of a Key-Value Store plan that still works as a link remote, I'm happy to revisit.

connectionPooling.attach rejecting after create. I added a note to the doc comment: once onAttached runs, the attachment exists, so a rejection from an onAttached throw or an abort can still leave the pool attached, and callers shouldn't blindly retry. I kept the abort-rejects behavior to match the signal contract everywhere else. The CLI doesn't retry.

GUS work item. Agreed; W-23387136 stays open until the CLI PR lands. The CLI branch already has the migration, characterization tests, and V12-CHANGES.md entries; it's waiting on this release.

Add data.link.listByApp, createForDatabase, and deleteFromDatabase,
data.connectionPooling.attach, and data.postgresDatabase.listLegacyByApp
so the v12 CLI can migrate pg:links:* and pg:connection-pooling:attach
without resolving databases itself. Extension names avoid the generated
link and connectionPooling route names.

Adds EssentialDatabaseNotSupportedError and InvalidLinkRemoteError.

W-23387136
resolveAddonInternal throws AddonNotFoundError (not NotFoundError) when the
app-scoped resolve only matches non-Postgres add-ons, so the global retry
never ran. Catch both.

W-23387136
…posites

- Retry add-on resolution globally only on an add-on miss; a 404 for the
  app is rethrown. Shared by createAttachment and createForDatabase, and
  app::addon identities resolve once.
- Require an exact heroku-redis or Postgres service name for link remotes
  and throw InvalidLinkRemoteError otherwise.
- Rename hooks to beforeCreate/beforeDelete; onResolved takes the add-on.
- attach takes {name, credential}; the release lookup is best-effort.
- Export PgDatabaseNotFoundError, ResolvedPgDatabase, and
  EssentialDatabaseNotSupportedError from the data barrel.
- Remove the duplicated createAttachment test block; tests now use derived
  mock clients and check extensions don't shadow generated routes.
- resolveAddonScopedThenGlobal retries globally only on an empty match or a
  404 labeled resource: 'add_on'. Unlabeled 404s now rethrow (fail closed).
  The live API labels both app and add-on misses, so this changes no
  observed behavior.
- link.createForDatabase accepts only Heroku Postgres remotes. Heroku Data
  Links connects Postgres to Postgres; the API rejects Key-Value Store remotes
  with a 404, so the SDK now fails fast with InvalidLinkRemoteError instead.
- Document that connectionPooling.attach can reject after the attachment
  exists (onAttached throw or abort), so callers shouldn't blindly retry.
…eral

An add-on can have namespaced attachments on the same app alongside its
default one (credential:<name>, connection-pooling:default). heroku-cli-util
kept only un-namespaced matches when a resolve returned several; the SDK
reported them as ambiguous, so resolving such a database by add-on name
failed (e.g. pg:connection-pooling:attach <addon> after a pool exists).

singularize now narrows multiple matches to the default attachments when
any exist, and stays ambiguous otherwise.
@eablack
eablack force-pushed the eb/W-23387136/link-pooling-extensions branch from ebf3326 to 8ff7eed Compare November 6, 2026 23:19

@jdodson jdodson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid work.

Approved.

@eablack
eablack merged commit d89018c into main Nov 7, 2026
10 checks passed
@eablack
eablack deleted the eb/W-23387136/link-pooling-extensions branch November 7, 2026 00:43
eablack added a commit to heroku/cli that referenced this pull request Nov 7, 2026
0.12.6 ships the link and connection pooling composites from
heroku/heroku-sdk#184, replacing the temporary git branch dependency.

W-23387136
eablack added a commit to heroku/cli that referenced this pull request Nov 7, 2026
…ch (#4046)

* test(pg:links:create): add characterization cases

* test(pg:links:destroy): add characterization cases

* test(pg:links): add characterization cases

* test(pg:connection-pooling:attach): add characterization cases

* refactor: use @heroku/sdk for pg:links:create command

* test(pg:links:create): stub @heroku/sdk directly, drop nock

* refactor: remove unused src/lib/addons/resolve.ts

pg:links:create was its last consumer; add-on resolution now happens in the SDK.

* refactor(pg:links): share the SDK-to-CLI link error mapping

* refactor: use @heroku/sdk for pg:links:destroy command

* test(pg:links:destroy): stub @heroku/sdk directly, drop nock

* refactor: use @heroku/sdk for pg:links command

* test(pg:links): stub @heroku/sdk directly, drop nock

* refactor: use @heroku/sdk for pg:connection-pooling:attach command

* test(pg:connection-pooling:attach): stub @heroku/sdk directly, drop nock

* docs: note pg:links and pg:connection-pooling:attach behavior changes in V12-CHANGES

* fix(pg:links:create): accept only Heroku Postgres remotes

Heroku Data Links connects Postgres to Postgres; the API rejects Key-Value
Store remotes with a 404. The SDK now rejects them up front with
InvalidLinkRemoteError, so the CLI error and command examples drop the
heroku-redis remote, and V12-CHANGES records the change.

* fix(pg:links:create): show the SDK's invalid-remote message

Pass InvalidLinkRemoteError through instead of replacing it with a
CLI-written string. The SDK message names the remote and says Data Links
only connect Heroku Postgres databases. Also drop a stale comment in
pg:links about the pre-SDK response shape, and give the --as create test
a distinct remote so it catches a swapped spinner label.

* docs: clarify pg:links V12-CHANGES entries after review

* deps: point @heroku/sdk at the heroku-sdk#184 branch

Temporary until the link and connection pooling composites are released.
Main already published 0.12.5 without them, so they will ship in 0.12.6.

W-23387136

* deps: bump @heroku/sdk to ^0.12.6

0.12.6 ships the link and connection pooling composites from
heroku/heroku-sdk#184, replacing the temporary git branch dependency.

W-23387136
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants