v0.6.0

v0.6.0 (2024-04-21)

Chore

• chore: enabling pythong 3.12 checks for CI (25526ea)

• chore: setting up precommit to be consistent with CI (18e706d)

Feature

• feat: Added tanh-relu activation fn and input noise options (#77)

• Still need to pip-install from GitHub hufy implementation.

• Added support for tanh_sae.

• Added notebook for loading the tanh_sae

• tweaking config options to be more declarating / composable

• testing adding noise to SAE forward pass

• updating notebook

---

Co-authored-by: David Chanin <[email protected]> (551e94d)

Unknown

• Update proposal.md (6d45b33)

• Merge pull request #96 from jbloomAus/github-templates

add templates for PR's / issues (241a201)

• add templates for PR's / issues (74ff597)

• Merge pull request #95 from jbloomAus/load-state-dict-not-strict

Make load_state_dict use strict=False (4a9e274)

• fix accidental bug (c22fbbd)

• fix load pretrained legacy with state dict change (b5e97f8)

• Make load_state_dict use strict=False (fdf7fe9)

• Merge pull request #94 from jbloomAus/update-pre-commit

chore: setting up precommit to be consistent with CI (6a056b7)

• Merge pull request #87 from evanhanders/old_to_new

Adds function that converts old .pt pretrained SAEs to new folder format (1cb1725)

• Merge pull request #93 from jbloomAus/py-312-ci

chore: enabling python 3.12 checks for CI (87be422)

v0.5.1

v0.5.1 (2024-04-19)

Chore

• chore: re-enabling isort in CI (#86) (9c44731)

Fix

• fix: pin pyzmq==26.0.1 temporarily (0094021)

• fix: typing issue, temporary (25cebf1)

Unknown

• v0.5.1 (0ac218b)

• Merge pull request #91 from jbloomAus/decoder-fine-tuning

Decoder fine tuning (1fc652c)

• par update (2bb5975)

• Merge pull request #89 from jbloomAus/fix_np

Enhance + Fix Neuronpedia generation / upload (38d507c)

• minor changes (bc766e4)

• reformat run.ipynb (822882c)

• get decoder fine tuning working (11a71e1)

• format (040676d)

• Merge pull request #88 from jbloomAus/get_feature_from_neuronpedia

FEAT: Add API for getting Neuronpedia feature (1666a68)

• Fix resuming from batch (145a407)

• Use original repo for sae_vis (1a7d636)

• Use correct model name for np runner (138d5d4)

• Merge main, remove eindex (6578436)

• Add API for getting Neuronpedia feature (e78207d)

v0.5.0

v0.5.0 (2024-04-17)

Feature

• feat: Mamba support vs mamba-lens (#79)

• mamba support

• added init

• added optional model kwargs

• Support transformers and mamba

• forgot one model kwargs

• failed opts

• tokens input

• hack to fix tokens, will look into fixing mambalens

• fixed checkpoint

• added sae group

• removed some comments and fixed merge error

• removed unneeded params since that issue is fixed in mambalens now

• Unneded input param

• removed debug checkpoing and eval

• added refs to hookedrootmodule

• feed linter

• added example and fixed loading

• made layer for eval change

• fix linter issues

• adding mamba-lens as optional dep, and fixing typing/linting

• adding a test for loading mamba model

• adding mamba-lens to dev for CI

• updating min mamba-lens version

• updating mamba-lens version

---

Co-authored-by: David Chanin <[email protected]> (eea7db4)

Unknown

• update readme (440df7b)

• update readme (3694fd2)

v0.4.0

v0.4.0 (2024-04-16)

Feature

• feat: support orthogonal decoder init and no pre-decoder bias (ac606a3)

Fix

• fix: sae dict bug (484163e)

• fix: session loader wasn't working (a928d7e)

Unknown

• enable setting adam pars in config (1e53ede)

• fix sae dict loader and format (c558849)

• default orthogonal init false (a8b0113)

• notebook update (feca408)

v0.3.0

v0.3.0 (2024-04-15)

Feature

• feat: add basic tutorial for training saes (1847280)

v0.2.2

v0.2.2 (2024-04-15)

Fix

• fix: dense batch dim mse norm optional (8018bc9)

Unknown

• format (c359c27)

• make dense_batch_mse_normalization optional (c41774e)

• add warning in run script (9a772ca)

• update sae loading code (356a8ef)

• add device override to session loader (96b1e12)

• update readme (5cd5652)

v0.2.1

v0.2.1 (2024-04-13)

Fix

• fix: neuronpedia quicklist (6769466)

v0.2.0

v0.2.0 (2024-04-13)

Chore

• chore: improving CI speed (9e3863c)

• chore: updating README.md with pip install instructions and PyPI badge (682db80)

Feature

• feat: overhaul saving and loading (004e8f6)

Unknown

• Merge pull request #80 from wllgrnt/will-update-tutorial

bugfix - minimum viable updates to tutorial notebook (e51016b)

• minimum viable fixes to evaluation notebook (b907567)

• Merge pull request #76 from jbloomAus/faster-ci

perf: improving CI speed (8b00000)

• try partial cache restore (392f982)

• Merge branch 'main' into faster-ci (89e1568)

• Merge pull request #78 from jbloomAus/fix-artifact-saving-loading

Fix artifact saving loading (8784c74)

• remove duplicate code (6ed6af5)

• set device in load from pretrained (b4e12cd)

• fix typing issue which required ignore (a5df8b0)

• remove print statement (295e0e4)

• remove load with session option (74926e1)

• fix broken test (16935ef)

• avoid tqdm repeating during training (1d70af8)

• avoid division by 0 (2c7c6d8)

• remove old notebook (e1ad1aa)

• use-sae-dict-not-group (27f8003)

• formatting (827abd0)

• improve artifact loading storage, tutorial forthcoming (604f102)

• add safetensors to project (0da48b0)

• Merge pull request #71 from weissercn/main

Addressing notebook issues (8417505)

• Merge pull request #70 from jbloomAus/update-readme-install

chore: updating README.md with pip install instructions and PyPI badge (4d7d1e7)

• linting (1db0b5a)

• fixed graph name (ace4813)

• changed key for df_enrichment_scores, so it can be run (f0a9d0b)

• fixed space in notebook 2 (2278419)

• fixed space in notebook 2 (24a6696)

• fixed space in notebook (d2f8c8e)

• fixed pickle backwards compatibility in tutorial (3a97a04)

v0.1.0

v0.1.0 (2024-04-06)

Feature

• feat: release (c70b148)

Fix

• fix: removing paths-ignore from action to avoid blocking releases (28ff797)

• fix: updating saevis version to use pypi (dbd96a2)

Unknown

• Merge pull request #69 from chanind/remove-ci-ignore

fix: removing paths-ignore from action to avoid blocking releases (179cea1)

• Update README.md (1720ce8)

• Merge pull request #68 from chanind/updating-sae-vis

fix: hotfix updating saevis version to use pypi (a13cee3)

v0.0.0

v0.0.0 (2024-04-06)

Chore

• chore: adding more tests to ActivationsStore + light refactoring (cc9899c)

• chore: running isort to fix imports (53853b9)

• chore: setting up pyright type checking and fixing typing errors (351995c)

• chore: enable full flake8 default rules list (19886e2)

• chore: using poetry for dependency management (465e003)

• chore: removing .DS_Store files (32f09b6)

Unknown

• Merge pull request #66 from chanind/pypi

feat: setting up sae_lens package and auto-deploy with semantic-release (34633e8)

• Merge branch 'main' into pypi (3ce7f99)

• Merge pull request #60 from chanind/improve-config-typing

fixing config typing (b8fba4f)

• setting up sae_lens package and auto-deploy with semantic-release (ba41f32)

• fixing config typing

switch to using explicit params for ActivationsStore config instead of RunnerConfig base class (9be3445)

• Merge pull request #65 from chanind/fix-forgotten-scheduler-opts

passing accidentally overlooked scheduler opts (773bc02)

• passing accidentally overlooked scheduler opts (ad089b7)

• Merge pull request #64 from chanind/lr-decay

adding lr_decay_steps and refactoring get_scheduler (c960d99)

• adding lr_decay_steps and refactoring get_scheduler (fd5448c)

• Merge pull request #53 from hijohnnylin/neuronpedia_runner

Generate and upload Neuronpedia artifacts (0b94f84)

• format (792c7cb)

• ignore type incorrectness in imported package (5fe83a9)

• Merge pull request #63 from chanind/remove-eindex

removing unused eindex depencency (1ce44d7)

• removing unused eindex depencency (7cf991b)

• Safe to_str_tokens, fix memory issues (901b888)

• Allow starting neuronpedia generation at a specific batch numbe (85d8f57)

• FIX: Linting 'do not use except' (ce3d40c)

• Fix vocab: Ċ should be line break. Also set left and right buffers (205b1c1)

• Merge (b159010)

• Update Neuronpedia Runner (885de27)

• Merge pull request #58 from canrager/main

Make prepend BOS optional: Default True (48a07f9)

• make tests pass with use_bos flag (618d4bb)

• Merge pull request #59 from chanind/fix-docs-deploy

attempting to fix docs deploy (cfafbe7)

• force docs push (3aa179d)

• ignore type eror (e87198b)

• format (67dfb46)

• attempting to fix docs deploy (cda8ece)

• Merge branch 'main' of https://github.com/jbloomAus/mats_sae_training into main (8aadcd3)

• add prepend bos flag (c0b29cc)

• fix attn out on run evals (02fa90b)

• Merge pull request #57 from chanind/optim-tests

Adding tests to get_scheduler (13c8085)

• Merge pull request #56 from chanind/sae-tests

minor refactoring to SAE and adding tests (2c425ca)

• minor refactoring to SAE and adding tests (92a98dd)

• adding tests to get_scheduler (3b7e173)

• Generate and upload Neuronpedia artifacts (b52e0e2)

• Merge pull request #54 from jbloomAus/hook_z_suppourt

notional support, needs more thorough testing (277f35b)

• Merge pull request #55 from chanind/contributing-docs

adding a contribution guide to docs (8ac8f05)

• adding a contribution guide to docs (693c5b3)

• notional support, needs more thorough testing (9585022)

• Generate and upload Neuronpedia artifacts (4540268)

• Merge pull request #52 from hijohnnylin/fix_db_runner_assert

FIX: Don't check wandb assert if not using wandb (5c48811)

• FIX: Don't check wandb assert if not using wandb (1adefda)

• add docs badge (f623ed1)

• try to get correct deployment (777dd6c)

• Merge pull request #51 from jbloomAus/mkdocs

Add Docs to the project. (d2ebbd7)

• mkdocs, test (9f14250)

• code cov (2ae6224)

• Merge pull request #48 from chanind/fix-sae-vis-version

Pin sae_vis to previous working version (3f8a30b)

• fix suffix issue (209ba13)

• pin sae_vis to previous working version (ae0002a)

• don't ignore changes to .github (35fdeec)

• add cov report (971d497)

• Merge pull request #40 from chanind/refactor-train-sae

Refactor train SAE and adding unit tests (5aa0b11)

• Merge branch 'main' into refactor-train-sae ([0acdcb3](0acdcb395aaa42e0344191e...