Skip to content

Conversation

@fede-kamel
Copy link

@fede-kamel fede-kamel commented Nov 27, 2025

Summary

Fixes #220 - TypeError when running Mamba models in interactive mode.

Root Cause: The interactive() function was passing images as a positional argument to both generate() and generate_mamba(), but their signatures differ:

  • generate(encoded_prompts, model, images, *, ...) - accepts images as 3rd positional arg
  • generate_mamba(encoded_prompts, model, *, ...) - does NOT accept images (Mamba models don't support vision)

The Fix: Use separate code paths for Transformer and Mamba models in interactive():

if isinstance(model, Transformer):
    generated_tokens, _ = generate([tokens], model, [images], ...)
else:
    # Mamba models don't support images
    generated_tokens, _ = generate_mamba([tokens], model, ...)

Changes

  • src/mistral_inference/main.py: Separate branches for Transformer vs Mamba in interactive()
  • tests/test_main_generate_dispatch.py: Regression tests verifying function signatures

Test Plan

  • Unit tests verify generate_mamba() doesn't accept images parameter
  • Unit tests verify generate() accepts images as positional parameter
  • Manual test with Mamba-Codestral-7B-v0.1 on NVIDIA GPU (in progress)

Before (crash)

TypeError: generate_mamba() takes 2 positional arguments but 3 were given

After (works)

Mamba models generate correctly in interactive mode without the images argument.

The `interactive()` function was calling `generate_mamba()` with `images`
as a 3rd positional argument, but `generate_mamba()` only accepts 2
positional arguments (encoded_prompts, model) followed by keyword-only args.

This caused a TypeError when using Mamba models (e.g., Codestral-Mamba):
  TypeError: generate_mamba() takes 2 positional arguments but 3
  positional arguments (and 3 keyword-only arguments) were given

The fix separates the function calls:
- Transformer models: call generate() with images argument
- Mamba models: call generate_mamba() without images (not supported)

Fixes mistralai#220
@fede-kamel
Copy link
Author

cc @patrickvonplaten @timlacroix @juliendenize for review - this fixes issue #220 which affects all Mamba models in interactive mode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG: TypeError: generate_mamba() takes 2 positional arguments but 3 positional arguments were given

1 participant