Skip to content

Commit 89856ef

Browse files
committed
Bugfix: only eval new tokens
1 parent 887f3b7 commit 89856ef

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

Diff for: llama_cpp/llama.py

+1
Original file line numberDiff line numberDiff line change
@@ -280,6 +280,7 @@ def generate(
280280
if self.verbose:
281281
print("generate cache hit", file=sys.stderr)
282282
reset = False
283+
tokens = tokens[len(self.tokens) :]
283284
###
284285
if reset:
285286
self.reset()

0 commit comments

Comments
 (0)