[Question] Is there any way to train multiple saes at the same time?

[neilwen987]

Questions

Is there any way to train multiple saes at the same time? For example, I want to try saes for “ blocks.0.attn.hook_rot_k
, blocks.0.attn.hook_rot_q”. I notice that activations are only store for per hook name. Is there any way to train multipule saes at the same time ?

[chanind]

That's currently not supported, but would be a nice thing to add if we can do it in a way that does't make the architecture too messy