Skip to content

Commit ae89575

Browse files
Make from_pretrained work for all Mamba models and rewrite README
Patch from_pretrained to auto-inject mamba2 ssm_cfg defaults, disable fused_add_norm (no Triton on macOS), and load with strict=False for missing norm biases. README now mirrors diffusers: pip install + a Python quickstart that works without git clone or download scripts. Remove noisy print on import.
1 parent 30c0e48 commit ae89575

4 files changed

Lines changed: 57 additions & 74 deletions

File tree

LICENSE

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -187,6 +187,7 @@
187187
identification within third-party archives.
188188

189189
Copyright 2023 Tri Dao, Albert Gu
190+
Copyright 2026 Saurabh Purohit
190191

191192
Licensed under the Apache License, Version 2.0 (the "License");
192193
you may not use this file except in compliance with the License.

README.md

Lines changed: 42 additions & 67 deletions
Original file line numberDiff line numberDiff line change
@@ -8,65 +8,35 @@
88
[![PyPI](https://img.shields.io/pypi/v/mamba-ssm-macos)](https://pypi.org/project/mamba-ssm-macos/)
99
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
1010

11-
Training and inference of Mamba 1 & 2 on Apple Silicon with MPS acceleration. Works without CUDA/Triton. Supports CLI, Python API, and interactive demos.
11+
Training and inference of Mamba 1 & 2 on Apple Silicon with MPS acceleration. Works without CUDA/Triton.
1212

1313
## Installation
1414

1515
```bash
1616
pip install mamba-ssm-macos
1717
```
1818

19-
Or install from source:
19+
Prerequisites: macOS 12.3+ with Apple Silicon, Python 3.10+, 8GB+ RAM recommended.
2020

21-
```bash
22-
git clone https://github.com/purohit10saurabh/mamba-ssm-macos.git
23-
cd mamba-ssm-macos
24-
uv sync # or: pip install -r requirements.txt
25-
```
26-
27-
## Quick Start
28-
29-
```bash
30-
31-
python -m scripts.download_models mamba1 # Mamba 1 (493MB)
32-
python -m scripts.download_models mamba2 # Mamba 2 (493MB)
33-
34-
make run-mamba1 # Quick Mamba 1 demo
35-
make run-mamba2 # Quick Mamba 2 demo
36-
```
37-
38-
**Prerequisites:** macOS 12.3+ with Apple Silicon, Python 3.10+, 8GB+ RAM recommended.
21+
## Quickstart
3922

40-
## Usage
41-
42-
### Text Generation
43-
```bash
44-
python -m scripts.run_models mamba1 --prompt "The future of AI" --max-length 50
45-
python -m scripts.run_models mamba2 --prompt "The future of AI" --max-length 30
46-
python -m scripts.run_models mamba1 --prompt "Once upon a time" --temperature 0.8
47-
python -m examples.02_text_generation --interactive
48-
```
49-
50-
### Examples
51-
```bash
52-
python -m examples.01_core_modules # Core modules usage
53-
python -m examples.02_text_generation # Text generation demo
54-
python -m examples.03_training # Training example
23+
```python
24+
import torch
25+
from transformers import AutoTokenizer
26+
from mamba_ssm import MambaLMHeadModel, generate_text_with_model, get_device
27+
28+
device = get_device()
29+
model_name = "state-spaces/mamba-130m" # or "state-spaces/mamba2-130m"
30+
model = MambaLMHeadModel.from_pretrained(model_name, device=device)
31+
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neox-20b")
32+
text = generate_text_with_model(model, tokenizer, "Once upon a time", device, max_length=50, temperature=0.8)
33+
print(text)
5534
```
5635

57-
### Makefile Commands
58-
```bash
59-
make download-models # Download both models
60-
make run-mamba1 # Quick Mamba 1 demo
61-
make run-mamba2 # Quick Mamba 2 demo
62-
make test-quick # Fast integration test
63-
make test # Full test suite
64-
```
36+
The model is downloaded from Hugging Face Hub on first run and cached under `~/.cache/huggingface/`.
6537

6638
## Training
6739

68-
See `examples/03_training.py` for a full example. Snippet:
69-
7040
```python
7141
import torch
7242
from torch import nn
@@ -84,25 +54,37 @@ for input_ids, labels in dataloader:
8454
optimizer.step()
8555
```
8656

87-
## Repository Structure
57+
See `examples/03_training.py` for a complete training example.
8858

89-
```
90-
mamba-ssm-macos/
91-
├── mamba_ssm/ # Core library (models, modules, ops, utils)
92-
├── scripts/ # download_models.py, run_models.py
93-
├── tests/ # unit/, integration/, run_unit_tests.py
94-
├── examples/ # 01_core_modules, 02_text_generation, 03_training
95-
├── Makefile
96-
└── pyproject.toml
59+
## Examples
60+
61+
Clone the repo to run examples:
62+
63+
```bash
64+
git clone https://github.com/purohit10saurabh/mamba-ssm-macos.git
65+
cd mamba-ssm-macos
66+
uv sync
67+
python -m examples.01_core_modules # Core modules usage
68+
python -m examples.02_text_generation --interactive # Text generation
69+
python -m examples.03_training # Training
9770
```
9871

9972
## Troubleshooting
10073

101-
**"Model files not found"**Run `make download-models` or `python -m scripts.download_models mamba1|mamba2`.
74+
**"MPS not available"**Check with `python -c "import torch; print(torch.backends.mps.is_available())"`. The library falls back to CPU automatically.
10275

103-
**"MPS not available"**Check with `python -c "import torch; print(torch.backends.mps.is_available())"`. Falls back to CPU automatically.
76+
**Slow first run**Initial `from_pretrained` downloads ~500MB of weights from Hugging Face Hub. Subsequent runs use the local cache.
10477

105-
**Import errors** — Use module syntax: `python -m examples.02_text_generation`.
78+
## Development
79+
80+
```bash
81+
git clone https://github.com/purohit10saurabh/mamba-ssm-macos.git
82+
cd mamba-ssm-macos
83+
uv sync --extra dev
84+
make test
85+
```
86+
87+
Available `make` targets: `test`, `test-unit`, `test-integration`, `test-quick`, `format`, `format-check`, `clean`. See the [Makefile](Makefile) for details.
10688

10789
## Citation
10890

@@ -141,19 +123,12 @@ Also available via GitHub's "Cite this repository" button ([CITATION.cff](CITATI
141123
## References
142124

143125
- [state-spaces/mamba](https://github.com/state-spaces/mamba) — Original implementation
144-
- [state-spaces/mamba1-130m](https://huggingface.co/state-spaces/mamba1-130m) — Mamba 1 130M Pre-trained model
145-
- [state-spaces/mamba2-130m](https://huggingface.co/state-spaces/mamba2-130m) — Mamba 2 130M Pre-trained model
126+
- [state-spaces/mamba1-130m](https://huggingface.co/state-spaces/mamba-130m) — Mamba 1 130M pre-trained model
127+
- [state-spaces/mamba2-130m](https://huggingface.co/state-spaces/mamba2-130m) — Mamba 2 130M pre-trained model
146128

147129
## Contributing
148130

149-
Contributions are welcome — bug fixes, performance improvements, docs, and new features. Open an issue or submit a PR.
150-
151-
```bash
152-
git clone https://github.com/purohit10saurabh/mamba-ssm-macos.git
153-
cd mamba-ssm-macos
154-
uv sync --extra dev
155-
make test
156-
```
131+
Feel free to [open an issue](https://github.com/purohit10saurabh/mamba-ssm-macos/issues) or [submit a PR](https://github.com/purohit10saurabh/mamba-ssm-macos/pulls). See [Development](#development) for the local setup.
157132

158133
## License
159134

mamba_ssm/__init__.py

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,6 @@
55
except PackageNotFoundError:
66
__version__ = "0.0.0"
77

8-
import platform
9-
import sys
10-
11-
if sys.platform == "darwin" and platform.machine() == "arm64":
12-
print("Mamba SSM macOS: Running on Apple Silicon with MPS acceleration")
13-
148
from mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel
159
from mamba_ssm.modules.mamba2 import Mamba2
1610
from mamba_ssm.modules.mamba_simple import Mamba

mamba_ssm/models/mixer_seq_simple.py

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -232,9 +232,22 @@ def forward(
232232
@classmethod
233233
def from_pretrained(cls, pretrained_model_name, device=None, dtype=None, **kwargs):
234234
config_data = load_config_hf(pretrained_model_name)
235+
if "mamba2" in pretrained_model_name:
236+
mamba2_defaults = {
237+
"layer": "Mamba2",
238+
"d_state": 128,
239+
"d_conv": 4,
240+
"expand": 2,
241+
"headdim": 64,
242+
"ngroups": 1,
243+
}
244+
config_data["ssm_cfg"] = {**mamba2_defaults, **config_data.get("ssm_cfg", {})}
245+
config_data["fused_add_norm"] = False
235246
config = MambaConfig(**config_data)
236247
model = cls(config, device=device, dtype=dtype, **kwargs)
237-
model.load_state_dict(load_state_dict_hf(pretrained_model_name, device=device, dtype=dtype))
248+
model.load_state_dict(
249+
load_state_dict_hf(pretrained_model_name, device=device, dtype=dtype), strict=False
250+
)
238251
return model
239252

240253
def save_pretrained(self, save_directory):

0 commit comments

Comments
 (0)