RWKV-X

[frink]

How hard would it be to run RWKV-X on this?

I know there is work that needs to be done. But how much?

[nkypy]

howard-hou/RWKV-X, this one?
https://github.com/howard-hou/RWKV-X/blob/main/package/src/rwkv_x/model.py#L300
I think the only difference is the attention part. Only need to rewrite the attention part. Not a heavy work.

[frink]

I think that would be awesome if it could be done. The attention stuff is wonderful for what it does.