Skip to content

Better error message for incorrect input in LanguageModel #593

@Butanium

Description

@Butanium

Currently this snippet:

from nnsight import LanguageModel

model  = LanguageModel("gpt2")
toks = model.tokenizer("Hello, world!", return_tensors="pt")
input_ids = toks["input_ids"]
attention_mask = toks["attention_mask"]
with model.trace(
            input_ids=input_ids, attention_mask=attention_mask
):
    x = model.transformer.h[0].output

Fails with this very uninformative message:

---------------------------------------------------------------------------
NNsightException                          Traceback (most recent call last)
Cell In[6], line 5
      2 from nnsight import LanguageModel
      4 model  = LanguageModel("gpt2")
----> 5 with model.trace(
      6             input_ids=input_ids, attention_mask=attention_mask
      7 ):
      8     x = model.transformer.h[0].output

File ~/projects/diffing-toolkit/.venv/lib/python3.12/site-packages/nnsight/intervention/tracing/base.py:665, in Tracer.__exit__(self, exc_type, exc_val, exc_tb)
    662 if self.asynchronous:
    663     return self.backend(self)
--> 665 self.backend(self)
    667 return True

File ~/projects/diffing-toolkit/.venv/lib/python3.12/site-packages/nnsight/intervention/backends/execution.py:24, in ExecutionBackend.__call__(self, tracer)
     21     return tracer.execute(fn)
     22 except Exception as e:
---> 24     raise wrap_exception(e, tracer.info) from None
     25 finally:
     26     Globals.exit()

NNsightException: 

Traceback (most recent call last):
  File "<ipython-input-6-77795bbc4ead>", line 8, in <module>
    x = model.transformer.h[0].output

ValueError: Cannot return output of Envoy that is not interleaving nor has a fake output set.

I guess this is a LanguageModel specific issue and not a general NNsight issue, but this can be confusing for people used to the hf interfact where this works:

from transformers import AutoModelForCausalLM
hf_model = AutoModelForCausalLM.from_pretrained("gpt2")
hf_model(input_ids=input_ids, attention_mask=attention_mask)

if there are some limitations that do not allow to support this synthax a nicer error message would be nice

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions