Skip to content

feat(scheduler): abstract model/server storage #7070

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
domsolutions wants to merge 52 commits into
base: v2
Choose a base branch
from
Open

feat(scheduler): abstract model/server storage #7070

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
6b53a17
abstract underlying memory store
domsolutions Dec 18, 2025
570a87d
pb for storage
domsolutions Dec 29, 2025
e282f42
wip
domsolutions Dec 29, 2025
663ba91
abstracted storage, now uses proto to store model/servers
domsolutions Dec 29, 2025
2ab0cd0
abstracted storage, now uses proto to store model/servers
domsolutions Dec 29, 2025
01a7d8c
use generics
domsolutions Dec 30, 2025
0953f03
update to use new proto definitions for models/servers
domsolutions Jan 2, 2026
ae2692e
wip: adding DB save points
domsolutions Jan 2, 2026
032fe1a
added save points
domsolutions Jan 2, 2026
79e4122
fix nil pointer panics and align enum state values so operator can de…
domsolutions Jan 5, 2026
3d8b923
fix new model version numbers not being set
domsolutions Jan 5, 2026
3d80eb8
add context for future DB calls
domsolutions Jan 5, 2026
64b9cf0
fix states
domsolutions Jan 5, 2026
efa9a25
wip: fix auto-scaling where expected replicas being cast to -1 during…
domsolutions Jan 6, 2026
d80c203
fix auto scaling
domsolutions Jan 6, 2026
a57aac4
server locks to prevent overwriting with stale data
domsolutions Jan 6, 2026
2c9e98b
lock should be before getting server to avoid stale data
domsolutions Jan 6, 2026
551f88c
use SetReplicas which checks for nil map and will initialise if empty
domsolutions Jan 7, 2026
76bedab
fix adding a model which was previously deleted not loading onto repl…
domsolutions Jan 8, 2026
14fc906
remove DB dump proto for separate PR
domsolutions Jan 8, 2026
8242d4c
fix circular import, wip fixing tests
domsolutions Jan 8, 2026
d9e75be
wip: fixing tests
domsolutions Jan 8, 2026
1b9bc95
nit
domsolutions Jan 9, 2026
f413f51
wip fix tests
domsolutions Jan 9, 2026
773d291
wip: fixing tests
domsolutions Jan 9, 2026
d90619c
more test fixes
domsolutions Jan 9, 2026
41d078f
wip fix test
domsolutions Jan 9, 2026
2ec851b
wip: fixing test
domsolutions Jan 9, 2026
7c54bb1
wip: fixng test. Additionally fixed bug where reserved memory not upd…
domsolutions Jan 12, 2026
3c6808b
wip: fixing test
domsolutions Jan 12, 2026
c6b597f
fix test and fix draining bug not picking up loaded models to re-sche…
domsolutions Jan 12, 2026
2bd3f62
fix expierment tests
domsolutions Jan 12, 2026
61517ef
fix envoy processor tests
domsolutions Jan 12, 2026
7723b36
fix xdscache tests
domsolutions Jan 12, 2026
16ed20b
add auto mockModelServerAPI instead of our custom mocks (#7086)
MiguelAAe Jan 12, 2026
317ea2a
agent test fixes
domsolutions Jan 12, 2026
79c036b
Merge branch 'dom/exp-ha' of github.com:SeldonIO/seldon-core into dom…
domsolutions Jan 12, 2026
535208a
wip: control plane tests
domsolutions Jan 12, 2026
9f69dfb
wip: fixing server tests
domsolutions Jan 12, 2026
f237969
handle model not found when adding pipelines/experiments. it is valid…
domsolutions Jan 13, 2026
a5f3a83
fixed server tests
domsolutions Jan 13, 2026
a683591
fix tests
domsolutions Jan 13, 2026
5021cbe
fix filter tests
domsolutions Jan 14, 2026
ba96b71
fix scheduler test
MiguelAAe Jan 14, 2026
de82c91
fix sorter tests
domsolutions Jan 14, 2026
6171089
Merge branch 'dom/exp-ha' of github.com:SeldonIO/seldon-core into dom…
domsolutions Jan 14, 2026
1b1a95f
wip
domsolutions Jan 14, 2026
e80e6fa
wip
domsolutions Jan 14, 2026
cf19ed2
fix Reschedule test
domsolutions Jan 14, 2026
94f7670
revert
domsolutions Jan 14, 2026
cc9334a
scheduler part two fixes
MiguelAAe Jan 14, 2026
6ebeac0
remove dead code and add tests for new storage and proto helpers
domsolutions Jan 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apis/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ build-go: \
./mlops/agent_debug/agent_debug.proto \
./mlops/proxy/proxy.proto \
./mlops/scheduler/scheduler.proto \
./mlops/scheduler/storage.proto \
./mlops/scheduler/db/db.proto \
./mlops/chainer/chainer.proto \
./mlops/v2_dataplane/v2_dataplane.proto \
./mlops/health/health.proto \
Expand Down
11 changes: 7 additions & 4 deletions apis/go/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,17 @@ go 1.23.0
toolchain go1.24.4

require (
github.com/onsi/gomega v1.39.0
google.golang.org/grpc v1.73.0
google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.5.1
google.golang.org/protobuf v1.36.6
google.golang.org/protobuf v1.36.7
)

require (
golang.org/x/net v0.41.0 // indirect
golang.org/x/sys v0.33.0 // indirect
golang.org/x/text v0.26.0 // indirect
github.com/google/go-cmp v0.7.0 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/net v0.43.0 // indirect
golang.org/x/sys v0.35.0 // indirect
golang.org/x/text v0.28.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250603155806-513f23925822 // indirect
)
38 changes: 28 additions & 10 deletions apis/go/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,23 @@
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/Masterminds/semver/v3 v3.4.0 h1:Zog+i5UMtVoCU8oKka5P7i9q9HgrJeGzI9SA1Xbatp0=
github.com/Masterminds/semver/v3 v3.4.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI=
github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/pprof v0.0.0-20250403155104-27863c87afa6 h1:BHT72Gu3keYf3ZEu2J0b1vyeLSOYI8bm5wbJM/8yDe8=
github.com/google/pprof v0.0.0-20250403155104-27863c87afa6/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/onsi/ginkgo/v2 v2.25.3 h1:Ty8+Yi/ayDAGtk4XxmmfUy4GabvM+MegeB4cDLRi6nw=
github.com/onsi/ginkgo/v2 v2.25.3/go.mod h1:43uiyQC4Ed2tkOzLsEYm7hnrb7UJTWHYNsuy3bG/snE=
github.com/onsi/gomega v1.39.0 h1:y2ROC3hKFmQZJNFeGAMeHZKkjBL65mIZcvrLQBF9k6Q=
github.com/onsi/gomega v1.39.0/go.mod h1:ZCU1pkQcXDO5Sl9/VVEGlDyp+zm0m1cmeG5TOzLgdh4=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/otel v1.35.0 h1:xKWKPxrxB6OtMCbmMY021CqC45J+3Onta9MqjhnusiQ=
Expand All @@ -20,17 +30,25 @@ go.opentelemetry.io/otel/sdk/metric v1.35.0 h1:1RriWBmCKgkeHEhM7a2uMjMUfP7MsOF5J
go.opentelemetry.io/otel/sdk/metric v1.35.0/go.mod h1:is6XYCUMpcKi+ZsOvfluY5YstFnhW0BidkR+gL+qN+w=
go.opentelemetry.io/otel/trace v1.35.0 h1:dPpEfJu1sDIqruz7BHFG3c7528f6ddfSWfFDVt/xgMs=
go.opentelemetry.io/otel/trace v1.35.0/go.mod h1:WUk7DtFp1Aw2MkvqGdwiXYDZZNvA/1J8o6xRXLrIkyc=
golang.org/x/net v0.41.0 h1:vBTly1HeNPEn3wtREYfy4GZ/NECgw2Cnl+nK6Nz3uvw=
golang.org/x/net v0.41.0/go.mod h1:B/K4NNqkfmg07DQYrbwvSluqCJOOXwUjeb/5lOisjbA=
golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/text v0.26.0 h1:P42AVeLghgTYr4+xUnTRKDMqpar+PtX7KWuNQL21L8M=
golang.org/x/text v0.26.0/go.mod h1:QK15LZJUUQVJxhz7wXgxSy/CJaTFjd0G+YLonydOVQA=
go.uber.org/automaxprocs v1.6.0 h1:O3y2/QNTOdbF+e/dpXNNW7Rx2hZ4sTIPyybbxyNqTUs=
go.uber.org/automaxprocs v1.6.0/go.mod h1:ifeIMSnPZuznNm6jmdzmU3/bfk01Fe2fotchwEFJ8r8=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/net v0.43.0 h1:lat02VYK2j4aLzMzecihNvTlJNQUq316m2Mr9rnM6YE=
golang.org/x/net v0.43.0/go.mod h1:vhO1fvI4dGsIjh73sWfUVjj3N7CA9WkKJNQm2svM6Jg=
golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI=
golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/text v0.28.0 h1:rhazDwis8INMIwQ4tpjLDzUhx6RlXqZNPEM0huQojng=
golang.org/x/text v0.28.0/go.mod h1:U8nCwOR8jO/marOQ0QbDiOngZVEBB7MAiitBuMjXiNU=
golang.org/x/tools v0.36.0 h1:kWS0uv/zsvHEle1LbV5LE8QujrxB3wfQyxHfhOk0Qkg=
golang.org/x/tools v0.36.0/go.mod h1:WBDiHKJK8YgLHlcQPYQzNCkUxUypCaa5ZegCVutKm+s=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250603155806-513f23925822 h1:fc6jSaCT0vBduLYZHYrBBNY4dsWuvgyff9noRNDdBeE=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250603155806-513f23925822/go.mod h1:qQ0YXyHHx3XkvlzUtpXDkS29lDSafHMZBAZDc03LQ3A=
google.golang.org/grpc v1.73.0 h1:VIWSmpI2MegBtTuFt5/JWy2oXxtjJ/e89Z70ImfD2ok=
google.golang.org/grpc v1.73.0/go.mod h1:50sbHOUqWoCQGI8V2HQLJM0B+LMlIUjNSZmow7EVBQc=
google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.5.1 h1:F29+wU6Ee6qgu9TddPgooOdaqsxTMunOoj8KA5yuS5A=
google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.5.1/go.mod h1:5KF+wpkbTSbGcR9zteSqZV6fqFOWBl4Yde8En8MryZA=
google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY=
google.golang.org/protobuf v1.36.6/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY=
google.golang.org/protobuf v1.36.7 h1:IgrO7UwFQGJdRNXH/sQux4R1Dj1WAKcLElzeeRaXV2A=
google.golang.org/protobuf v1.36.7/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
Loading
Loading