|
| 1 | +.. _threading: |
| 2 | + |
| 3 | +Using library in threaded environments |
| 4 | +====================================== |
| 5 | + |
| 6 | +Network namespaces |
| 7 | +------------------ |
| 8 | + |
| 9 | +In order to run a separate socket in network namespace `A` while keeping |
| 10 | +the Python process in namespace `B`, the library performs these steps: |
| 11 | + |
| 12 | +1. spawn a child process |
| 13 | +2. execute `netns.setns()` in the child |
| 14 | +3. create a socket |
| 15 | +4. send the file descriptor using `socket.send_fds()` back to the parent |
| 16 | +5. terminate the child process |
| 17 | +6. create a socket with `socket(fileno=...)` |
| 18 | + |
| 19 | +As a result, in the parent process we get a socket that belongs to another |
| 20 | +network namespace, but we can use it natively like any other socket, both in |
| 21 | +sync and async way. |
| 22 | + |
| 23 | +Start a child: os.fork() |
| 24 | +------------------------ |
| 25 | + |
| 26 | +By default, pyroute2 uses `os.fork()` to create the child. When using it |
| 27 | +in multithreaded processes, no threads will be recreated, and the child |
| 28 | +will only continue the thread where `os.fork()` was called in. This leaves |
| 29 | +the GC in a corrupted state. |
| 30 | + |
| 31 | +This should not be an issue since the socket creation routine stops the GC, |
| 32 | +and doesn't rely on any shared data. But still there is some risk. |
| 33 | + |
| 34 | +That's why pyroute2 provides a configuration option |
| 35 | +`config.child_process_mode`. By default it is `"fork"`, but you can change |
| 36 | +the option to `"mp"`, and then pyroute2 will use `multiprocessing` to |
| 37 | +create and control the child process. |
| 38 | + |
| 39 | +Start a child: multiprocessing |
| 40 | +------------------------------ |
| 41 | + |
| 42 | +Using `multiprocessing` may or may not rely on `os.fork()`, it depends on |
| 43 | +`multiprocessing.set_start_method()`. In Python version < 3.14 the default |
| 44 | +is `"fork"`, but starting with 3.14, the default is `"spawn"`. |
| 45 | + |
| 46 | +Using `"spawn"` is safer, but significantly slower. Beside of that, `"spawn"` |
| 47 | +implies additional limitations by pickling the target method and its |
| 48 | +arguments. This prevents passing lambdas as the target, and make impossible |
| 49 | +to pass the `libc` instance to the child process. |
| 50 | + |
| 51 | +The `multiprocessing` start method is out of scope for pyroute2, thus no |
| 52 | +way to set it using `config.child_process_mode`, where you can specify |
| 53 | +only `"mp"`. You have to run `multiprocessing.set_start_method()` by |
| 54 | +yourself somewhere else in your program then. |
0 commit comments