Skip to content

[ENH] Add feature to the rust log service to proxy the go log service. #4559

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 5 commits into
base: rescrv/wal3-bootstrap-impl
Choose a base branch
from
+283 −29
Draft

[ENH] Add feature to the rust log service to proxy the go log service. #4559

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
7a9ea53
[ENH] Add a lower bound to the scout logs response.
rescrv May 16, 2025
0a0b86e
[ENH] Add feature to the rust log service to proxy the go log service.
rescrv May 16, 2025
c9383bd
collapse code paths to not have two means by which logs end up on RLS
rescrv May 16, 2025
b883001
Start to effectuate log transfer (WIP)
rescrv May 17, 2025
a953a19
untested effectuate transfer
rescrv May 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions go/pkg/log/server/server.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,16 +54,18 @@ func (s *logServer) ScoutLogs(ctx context.Context, req *logservicepb.ScoutLogsRe
if err != nil {
return
}
var start int64
var limit int64
_, limit, err = s.lr.GetBoundsForCollection(ctx, collectionID.String())
start, limit, err = s.lr.GetBoundsForCollection(ctx, collectionID.String())
if err != nil {
return
}
// +1 to convert from the (] bound to a [) bound.
res = &logservicepb.ScoutLogsResponse{
FirstUncompactedRecordOffset: int64(start + 1),
FirstUninsertedRecordOffset: int64(limit + 1),
}
trace_log.Info("Scouted Logs", zap.Int64("limit", int64(limit + 1)), zap.String("collectionId", req.CollectionId))
trace_log.Info("Scouted Logs", zap.Int64("start", int64(start + 1)), zap.Int64("limit", int64(limit + 1)), zap.String("collectionId", req.CollectionId))
return
}

Expand Down
2 changes: 2 additions & 0 deletions idl/chromadb/proto/logservice.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ message ScoutLogsResponse {
reserved 1;
// The next record to insert will have this offset.
int64 first_uninserted_record_offset = 2;
// The oldest record on the log will have this offset.
int64 first_uncompacted_record_offset = 3;
}

message PullLogsRequest {
Expand Down
Loading
Loading