Skip to content

Add object store support to llm-d storage offloading#499

Open
effi-ofer wants to merge 7 commits intollm-d:mainfrom
effi-ofer:nixl
Open

Add object store support to llm-d storage offloading#499
effi-ofer wants to merge 7 commits intollm-d:mainfrom
effi-ofer:nixl

Conversation

@effi-ofer
Copy link
Copy Markdown

@effi-ofer effi-ofer commented Apr 6, 2026

This PR adds object store backend to the llm-d storage offloading. It does this by introducing a "backend" configuration parameter which can be set as follows, with POSIX as the default.

  "POSIX"           - POSIX engine, GDS disabled
  "POSIX_GDS_READ"  - GDS for reads only
  "POSIX_GDS_WRITE" - GDS for writes only
  "POSIX_GDS"       - GDS for reads and writes
  "POSIX_BB_READ"   - bounce-buffer GDS for reads only
  "POSIX_BB_WRITE"  - bounce-buffer GDS for writes only
  "POSIX_BB"        - bounce-buffer GDS for reads and writes
  "OBJ"             - NIXL S3 object store backend

The object store access is implemented using NIXL (for PUT/GET) and boto3 (for lookup).

Documentation:
The full configuration of the object store is described in docs/object_store.md.

Testing:

  • all the tests in the tests directories
  • added a new test_obj_backend test case

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 6, 2026

Unsigned commits detected! Please sign your commits.

For instructions on how to set up GPG/SSH signing and verify your commits, please see GitHub Documentation.

@github-actions github-actions bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Apr 6, 2026
effi-ofer added 7 commits April 6, 2026 19:48
@effi-ofer
add nixl support
c615a85 
Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>
@effi-ofer
add s3 lookup
eb23f9e 
Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debugging

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>
@effi-ofer
add dictionary lookup
396f6f1 
Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

debug

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>
@effi-ofer
add test_obj_backend
9b38358 
Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

add test_obj_backend

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

add certificate to boto3

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

add certificate to boto3

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

fix test_obj_backend.py

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

fix test_obj_backend.py

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

fix test_obj_backend.py

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

fix test_obj_backend.py

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

combine gds parameter with backend

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

combine gds parameter with backend

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

move nixl source to a new subdirectory

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

change dir nixl to llmd_nixl to avoid name collision with nixl library

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

change dir nixl to llmd_nixl to avoid name collision with nixl library

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

add nixl dependency

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

rm old nixl subdir

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

minor changes

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

fix wait logic

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>

clean staging buffer release

Signed-off-by: Effi Ofer <effi.ofer@gmail.com>
@effi-ofer
add doc
3cd3dd6 
Signed-off-by: Effi Ofer <effi.ofer@gmail.com>
@effi-ofer
change logger.info to logger.debug calls
084029b 
Signed-off-by: Effi Ofer <effi.ofer@gmail.com>
@effi-ofer
fix gds test case
b8d3524 
Signed-off-by: Effi Ofer <effi.ofer@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vMaroon vMaroon Awaiting requested review from vMaroon vMaroon is a code owner

@dannyharnik dannyharnik Awaiting requested review from dannyharnik dannyharnik is a code owner

@kfirtoledo kfirtoledo Awaiting requested review from kfirtoledo kfirtoledo is a code owner

@liu-cong liu-cong Awaiting requested review from liu-cong liu-cong is a code owner

@hyeongyun0916 hyeongyun0916 Awaiting requested review from hyeongyun0916

@sagearc sagearc Awaiting requested review from sagearc

@yankay yankay Awaiting requested review from yankay

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@effi-ofer