Skip to content

Commit a2c280e

Browse files
authored
bug-fix: Skip using compile_context when RBLN_COMPILE_MODEL=0 (V0) (#116)
1 parent 82a1bbc commit a2c280e

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

vllm_rbln/worker/worker.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -392,7 +392,7 @@ def _init_cache_engine(self):
392392

393393
bind_kv_cache(self.compilation_config.static_forward_context,
394394
self.cpu_cache)
395-
if not self.model_config.enforce_eager:
395+
if not self.model_config.enforce_eager and envs.RBLN_COMPILE_MODEL:
396396
for kv_cache in cpu_cache:
397397
self.model_runner.compile_context.mark_static_address(kv_cache)
398398

0 commit comments

Comments
 (0)