Skip to content

Create library-specific versions of logits processors #1445

Closed
@RobinPicard

Description

@RobinPicard

First suggested in #1444:

Not for this PR, but I think we'll have to break down the logits processor into library-specific ones. On top of this we'll soon have methods in outlines-core to mask on-device that see optimized for torch, MLX and NumPy. We can already open an issue for that.

The justification for this proposed change is that libraries providing local models (transformers, mlxlm, lamma.cpp...) use different object types and shapes to represent logits.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions