Skip to content

Support multi-SCM provider auth and skill resolution (GitLab, Azure DevOps, BitBucket) #1421

Description

@ptone

Summary

Scion's skill resolution (gh://), template/harness-config import, git credential management, and web UI are all hardcoded to GitHub. Users with repositories on GitLab, Azure DevOps, or BitBucket cannot use skills or import resources from those providers, and there is no token convention (GL_, ADO_, BB_) for authenticating against them.

Related: #1419 (GH_ token convention for private repos — the GitHub-only precursor).

Current state

GitHub is embedded across six subsystems with no provider abstraction:

Subsystem GitHub-specific entry points
Skill URI scheme gh:// parsed in pkg/agent/github_uri.go:41, resolved via GitHub Contents API in github_skill_resolver.go:193
URL detection Hard-coded github.com host checks in 7+ locations across remote_templates.go, handlers_skills_discover.go, web UI
GitHub API Contents API, Commits API, raw downloads, tarball fetch — all in github_skill_resolver.go and remote_templates.go
Token convention GH_ prefix in github_skill_resolver.go:125-188 — works only for skills, not templates/HCs
GitHub App Full JWT/installation-token infrastructure in pkg/hub/githubapp/, webhook handlers, DB schema
Git auth x-access-token pattern hardcoded in remote_templates.go:256,448; credential helper uses username=oauth2
Web UI GitHub-specific validation, icons, discovery button across 10+ components

Already provider-agnostic (extensible today): RoutingSkillResolver dispatch table, SkillResolver interface, RemoteTemplateType enum, config.IsRemoteURI(), archive/rclone fetch, secret.SecretBackend.

Proposed conventions

Provider URI scheme Token prefix Example keys Git clone username
GitHub gh:// GH_ GH_ACME_CORP, GH_ACME__REPO x-access-token
GitLab gl:// GL_ GL_MYGROUP, GL_MYGROUP__REPO oauth2
Azure DevOps ado:// ADO_ ADO_MYORG, ADO_MYORG__PROJECT (any non-empty)
BitBucket bb:// BB_ BB_WORKSPACE, BB_WORKSPACE__REPO x-token-auth

Recommended phasing

Phase 1: Token convention + import auth (Small)

Extract shared normalization to pkg/scmauth/. Teach fetchRemoteForImport() to detect provider from URL host and look up GL_/ADO_/BB_ prefixed tokens. Extend DetectRemoteType() for new hosts. Add provider-specific tarball/archive fetch. ~6 files, ~200-300 lines.

Phase 2: Skill URI schemes + resolvers (Medium)

Add gl://, ado://, bb:// URI parsers and resolvers. Register in detectScheme(), buildSkillRouter(), NormalizeSkillURI(). ~15 files, ~1000-1500 lines.

Phase 3: Provider app integrations (Large)

GitLab/ADO/BB equivalents of GitHub App: OAuth/token minting, webhooks, health checks, UI setup pages, DB schema. ~25+ files, ~3000+ lines.

Phase 4: UI multi-provider support (Medium)

Extend web components that hardcode github.com. Multi-provider URL validation, icons, discovery. ~10 web components.

Each phase is independently valuable and shippable.

Key code references

  • Skills: pkg/agent/github_uri.go, pkg/agent/github_skill_resolver.go, pkg/agent/routing_skill_resolver.go
  • Templates/HC: pkg/config/remote_templates.go, pkg/hub/resource_import.go
  • Token convention: pkg/agent/github_skill_resolver.go:125-188
  • App integration: pkg/hub/githubapp/, pkg/hub/handlers_github_app*.go
  • Credential helper: cmd/sciontool/commands/credential_helper.go, pkg/agent/provision.go:1360-1380
  • Investigation doc: scratchpad/projects/roadmap/inv-multi-scm-provider-support.md

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:authArea: autharea:hubArea: hubtype:epicLarge multi-issue effort; parent of sub-issues

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions