NDIF python version dependency is overly strict (3.12, not 3.11 or 3.14?)

[davidbau]

Currently NDIF cannot be used if the client python version is not exactly 3.12.

We should relax this restriction as far as possible.

By requiring researchers to downgrade or upgrade their whole environment to match the exact point release of the server, we are inviting problems if a researcher encounters some other critical dependency in some other part of their experimental setup. It makes NDIF a very risky dependency to take on.

[JadenFiotto-Kaufman]

@davidbau So yes the client version needs to be the same as the server version for python (colab uses 3.12 so that's what we use).

This gets into a discussion about how interventions should be packaged up and sent to NDIF. Right now, we compile your intervention fn locally (like we do when you use nnsight locally) and use cloudpickle to serialize the fn itself to send to NDIF which of course, requires the same python version. Ray uses cloudpickle as well to execute code remotely and needs the worker nodes to have the same python version as the head (minor versions are usually okay).

Previous versions of nnsight used a custom json serialization format which 1.) wasn't very efficient and 2.) , requires us to manually add support for every type we'd like to serialize which is a problem when someone says "I want to use this transformers object or this pytorch object"

So its a tradeoff between:

Pickle:
Pro - Arbitrary types. Cloudpickle recursively grabs all your local code referenced in your interventions.
Con - Need the same major python version

Custom:
Pro - Any python version
Con - Need to manually support objects

I'd love to hear your thoughts.

[davidbau]

I think we should not use cloudpickle, which is designed for a different purpose - it's really designed for within-system use, for moving objects between parts of a system made by the same developer - not loosely-coupled API-level use, like we are trying to build.

Cloudpickle also doesn't solve some other problems we would like to solve, such as executing functions and classes on the server that the server doesn't already have defined: when using pickling, if the client has some library that the server can't import, it can't be unpickled.

The solution is to do automatic serialization with a different philosophy: instead of pickling, we should implement inlining. We would want to think carefully about how code should be written to allow new code to be moved to the server that isn't already there. This is a very different philosophy than cloudpickle and we would need to implement it ourselves.

The beautiful thing is that, Nnsight is already a leading innovator in this approach. NNsight's 0.5 trace support is source-code-serialization based, which I think is an awesome insight. We just need to extend source-code-based serialization so solve 90% of the things that cloudpickle solves, but in a python-version-agnostic way. It will be beautiful.

More thoughts here:

https://github.com/davidbau/nnsight/blob/source-serialization-design/docs/nnsight-source-serialization-design.md

I'll try to vibe code a draft PR on that branch, and we can see if it is overcomplex or not.

[davidbau]

I've submitted PR #573 to discuss and for review. It's an attempt at being a pretty complete+exhaustive source-code-based serialization approach that solves the above issues.