Skip to content

Remove offsets from Tokenizer interface #462

@sagearc

Description

@sagearc

Render and RenderChat in the Tokenizer interface return ([]uint32, []types.Offset, error), but since switching to the renderer service (#461) these methods always return nil offsets. The offset return value is now dead weight — all callers already discard it with _.

Should drop []types.Offset from both signatures in the interface and all implementations (UdsTokenizer, CachedTokenizer, CompositeTokenizer, mocks).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions