feat(BA-2820, BA-2821): Implement API layer of model deployment by seedspirit · Pull Request #6394 · lablup/backend.ai

seedspirit · 2025-10-24T09:48:02Z

Resolve #6396 #6397 (BA-2820, BA-2821)
Following of #5754 and #5672
The work from #5752 and #5672 was lost during the merge with #5548 due to a force push, so I am creating and submitting a new PR

Checklist: (if applicable)

Milestone metadata specifying the target backport version
Mention to the original issue
Installer updates including:
- Fixtures for db schema changes
- New mandatory config options
Update of end-to-end CLI integration tests in ai.backend.test
API server-client counterparts (e.g., manager API -> client SDK)
Test case(s) to:
- Demonstrate the difference of before/after
- Demonstrate the flow of abstract/conceptual models with a concrete implementation
Documentation
- Contents in the docs directory
- docstrings in public interfaces and type annotations

📚 Documentation preview 📚: https://sorna--6394.org.readthedocs.build/en/6394/

📚 Documentation preview 📚: https://sorna-ko--6394.org.readthedocs.build/ko/6394/

…created in middleware

…nt connection

…oyment id

Copilot

Pull Request Overview

This PR implements the API layer for model deployment functionality, introducing GraphQL endpoints and service methods for managing model deployments, revisions, replicas, and auto-scaling rules. The implementation includes:

Service layer methods for deployment lifecycle operations (create, update, destroy, batch operations)
GraphQL API definitions for model deployments, revisions, replicas, and auto-scaling rules
Data loader registry for efficient batch loading of related entities
Exception classes for model deployment domain errors

Reviewed Changes

Copilot reviewed 61 out of 66 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`src/ai/backend/manager/services/deployment/service.py`	Implements service layer methods with mock data returns for deployment operations
`src/ai/backend/manager/services/deployment/processors.py`	Defines processors and protocol for handling deployment actions
`src/ai/backend/manager/api/gql/model_deployment/*.py`	GraphQL type definitions and resolvers for deployments, revisions, replicas
`src/ai/backend/manager/api/gql/base.py`	Helper utilities for pagination, global IDs, and JSON serialization
`src/ai/backend/manager/api/gql/data_loader/registry.py`	DataLoader registry for batch loading entities
`src/ai/backend/manager/data/deployment/*.py`	Data types, creators, and modifiers for deployment entities
`src/ai/backend/common/exception.py`	New exception classes for model deployment errors
`docs/manager/graphql-reference/*.graphql`	Updated GraphQL schema documentation

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-24T09:50:26Z

    """
    All available GraphQL queries.
-    Type name changed from 'Queries' to 'Query' in 25.13.0
+    Type name changed from 'Queries' to 'Query' in 25.15.0


The version number mentioned (25.15.0) appears inconsistent with other version numbers in the PR which reference 25.16.0. This should likely be 25.15.0 if it refers to a previous change, or be updated to match the current PR's version.

Suggested change

Type name changed from 'Queries' to 'Query' in 25.15.0

Type name changed from 'Queries' to 'Query' in 25.16.0

+    @classmethod
+    def from_dataclass(cls, data: ExtraVFolderMountData) -> "ExtraVFolderMount":
+        return cls(
+            # TODO: fix id generation logic


+    )
+    active_revision_id: OptionalState[UUID] = field(
+        default_factory=OptionalState[UUID].nop
+    )  # TODO: Check if TriState is more appropriate


+                ),
+                network_access=DeploymentNetworkSpec(
+                    open_to_public=True,
+                    url="http://example.com",


+                ),
+                network_access=DeploymentNetworkSpec(
+                    open_to_public=True,
+                    url="http://example.com",


+                    ),
+                    network_access=DeploymentNetworkSpec(
+                        open_to_public=True,
+                        url="http://example.com",


seedspirit added 30 commits October 24, 2025 18:39

feat: Implment basic api layers for model deployment gql types

1bf6615

refactor: replace resolve global id method to it's new version

8099123

fix: Remove uneccessary global id resolving in access token resolver

8c106a2

refactor: Offload empty token handling to the Dataloader

1f91a69

refactor: Change global id generator to support type safety

9c222b0

docs: Add news fragment

2eeaff0

feat: Add page info builder method

2a9ba21

feat: Implement pagination, ordering in model deployment

a92dec7

docs: Update gql schema reference

1bca174

refactor: Apply filtering, ordering in ReplicaState replicas resolver

30897cc

feat: Support pagination, filtering resolving deployments

2fd9c7d

feat: Inject user id in deployment creator with user data in context …

2930e4e

…created in middleware

feat: Replace mock page info to real implementation in model deployme…

4229559

…nt connection

misc: Remove uneccessary comment

af439ea

feat: Add pagination, filter support in revision history resolver

4a88edc

fix: Add missing deployment id field in add model revision action

bec09e0

feat: Implement create model revision handler

05b51a0

refactor: Use json util method defined in common

ea034c1

refactor: Remove unused batch load function

956f6d9

chore: Remove unused mock instances

b1a8c4f

fix: Make revision dataloader resolve N+1 problem

c120c3b

chore: Remove Error postfix

5e36cae

docs: Change model deployment type introduction version to 25.15.0

4dd798c

fix: Make access token dataloader resolve N+1 problem

6299732

fix: Make model deployment dataloader resolve N+1 problem

6f0a830

fix: Make replica batch load by deployment ids func resolve N+1 problem

81bfda8

fix: Make replica batch load by revision ids func resolve N+1 problem

9066df4

fix: make dataloader return correct data

d85fb6f

fix: Remove routing node type as it is not used

b231004

refactor: Resolve access tokens, scaling rules with their id not depl…

b386991

…oyment id

seedspirit added 8 commits October 24, 2025 18:40

fix: Remove unused replica fetch action

9c85726

refactor: Introduce dataloader registry to reuse dataloader instance

2e98367

fix: resolve replicas belongs to each model deployment

2305d59

feat: Implement pagination, ordering in access token resolver

a8f5d85

fix: Fix type hint of dataloader registry

71907c4

fix: Fix conflicts

20ff1bf

docs: Change deployment gql object introductory version

f56230d

fix: Revert wrong introduction version of session_pending_queue

7570756

seedspirit self-assigned this Oct 24, 2025

seedspirit added the skip:changelog Make the action workflow to skip towncrier check label Oct 24, 2025

github-actions Bot added size:XL 500~ LoC area:docs Documentations comp:manager Related to Manager component comp:common Related to Common component labels Oct 24, 2025

seedspirit requested a review from HyeockJinKim October 24, 2025 09:49

seedspirit marked this pull request as ready for review October 24, 2025 09:49

Copilot AI review requested due to automatic review settings October 24, 2025 09:49

Copilot AI reviewed Oct 24, 2025

View reviewed changes

github-advanced-security AI found potential problems Oct 24, 2025

View reviewed changes

fix: Revert version change comment

593d37f

HyeockJinKim removed the skip:changelog Make the action workflow to skip towncrier check label Oct 24, 2025

seedspirit changed the title ~~feat: Implement API layer of model deployment~~ feat(BA-2820, BA-2821): Implement API layer of model deployment Oct 24, 2025

seedspirit added 2 commits October 24, 2025 20:02

docs: Add news fragment

a3c60ae

docs: Remove past news fragment

64d9ba4

HyeockJinKim added this pull request to the merge queue Oct 26, 2025

Merged via the queue into main with commit 3a51bca Oct 26, 2025
29 of 30 checks passed

HyeockJinKim deleted the feat/Implement-deployment-api-layer branch October 26, 2025 03:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(BA-2820, BA-2821): Implement API layer of model deployment#6394

feat(BA-2820, BA-2821): Implement API layer of model deployment#6394
HyeockJinKim merged 41 commits into
mainfrom
feat/Implement-deployment-api-layer

seedspirit commented Oct 24, 2025 •

edited by github-actions Bot

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Oct 24, 2025

Uh oh!

Check notice

Check notice

Check warning

Check warning

Check warning

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	Type name changed from 'Queries' to 'Query' in 25.15.0
	Type name changed from 'Queries' to 'Query' in 25.16.0

Conversation

seedspirit commented Oct 24, 2025 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

Check notice

Check notice

Check warning

Check warning

Check warning

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

seedspirit commented Oct 24, 2025 •

edited by github-actions Bot

Loading