More intuitive error message #369
Description
The error message "Weakly-referenced object no longer exists." could be more descriptive / intuitive. It took me a while to notice what the error meant.
Not sure what to put exactly, but one suggestion might be appending "You are likely calling a model related object outside the tracing context. As the Intervention graph has already been executed at this point, the model related object no longer exists."
Snippet throwing the error
with torch.no_grad(), model.generate(
batch,
# {"input_ids": padded_input_ids_BL, "attention_mask": padded_attention_mask_BL},
max_new_tokens=None, # Generate until the model is done
do_sample=False,
temperature=0.0,
top_p=1.0,
pad_token_id=model.tokenizer.pad_token_id,
eos_token_id=model.tokenizer.eos_token_id,
remote=True, # Run the model remotely on NDIF
):
outputs = nnsight.list().save()
with model.lm_head.all(): # Error here, this block should be indented
outputs.append(model.lm_head.output[:, -1, :].argmax(dim=-1))Solution
with torch.no_grad(), model.generate(
batch,
# {"input_ids": padded_input_ids_BL, "attention_mask": padded_attention_mask_BL},
max_new_tokens=None, # Generate until the model is done
do_sample=False,
temperature=0.0,
top_p=1.0,
pad_token_id=model.tokenizer.pad_token_id,
eos_token_id=model.tokenizer.eos_token_id,
remote=True, # Run the model remotely on NDIF
):
outputs = nnsight.list().save()
with model.lm_head.all():
outputs.append(model.lm_head.output[:, -1, :].argmax(dim=-1))Metadata
Metadata
Assignees
Labels
No labels