Skip to content

Commit 59b766e

Browse files
Support time-based LPs (#27)
* max-time + trigger with time Signed-off-by: alessiodevoto <devoto.alessio@gmail.com> * flake8 fix Signed-off-by: alessiodevoto <devoto.alessio@gmail.com> * update with suggestions Signed-off-by: alessiodevoto <devoto.alessio@gmail.com> * fix style Signed-off-by: alessiodevoto <devoto.alessio@gmail.com> * fix notebooks Signed-off-by: alessiodevoto <devoto.alessio@gmail.com> * fix notebooks Signed-off-by: alessiodevoto <devoto.alessio@gmail.com> --------- Signed-off-by: alessiodevoto <devoto.alessio@gmail.com>
1 parent dc165dd commit 59b766e

20 files changed

+1458
-99
lines changed

README.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -81,11 +81,14 @@ I am getting a lot of calls during the day. What is more important for me to con
8181
The goal is to make LLM generate "3" as an answer.
8282

8383
### TriggerPhraseLogitsProcessor
84-
A logits processor which triggers phrases when it encounters a given token.
84+
A logits processor which triggers phrases when it encounters a given token or after a specified time.
8585
One common use case is to force writing python code just after thinking:
8686
```python
8787
trigger_python = TriggerPhraseLogitsProcessor(phrase="\n```python", trigger_token_phrase="</think>",
8888
tokenizer=tokenizer, trigger_count=1, trigger_after=True)
8989
```
9090
### PreventHallucinationLogitsProcessor
91-
A logits processor that mitigates hallucinated model outputs by enforcing a predefined fallback phrase when token confidence falls below a specified threshold.
91+
A logits processor that mitigates hallucinated model outputs by enforcing a predefined fallback phrase when token confidence falls below a specified threshold.
92+
93+
### MaxTimeLogitsProcessor
94+
A logits processor that enforces the end-of-sentence (EOS) token after a specified maximum time passes, optionally waiting for a new line or a full stop. Useful for controlling generation time and ensuring responses complete within time constraints.

0 commit comments

Comments
 (0)