Skip to content

Conversation

slyon
Copy link
Collaborator

@slyon slyon commented Jun 26, 2025

Description

Refactor Netplan's generate binary to be a proper systemd.generator, according to spec "FO165 – Netplan generator architecture".

New testing of a simulated systemd sandbox is implemented in tests/generator/test_sd-generator.py.

  • Logic that writes systemd unit files (e.g. .service units or override.conf drop-in configs) is now generated in src/gen-*.c and run during daemon-reload.
  • Writing of normal systemd-networkd/NetworkManager/wpa_supplicant/udev/Open vSwitch/SR-IOV config files remains in src/{networkd,nm,sriov,openvswitch}.c.
  • New tests added, utilizing a systemd-run sandbox (TestSystemdGenerator.test_sandbox), that make sure the usr/libexec/netplan/generate binary does not write files outside the allowed scope for a systemd-generator.
  • Adding a new --networkmanager-only parameter, to improve NM integration, by not touching any systemd[-networkd] files
  • Introducing new _netplan_state_get/set_flags() API (internal, for now) to do "validation-only" runs over our NetDef data

Checklist

  • Runs make check successfully.
  • Retains code coverage (make check-coverage).
  • New/changed keys in YAML format are documented.
  • (Optional) Adds example YAML for new feature.
  • (Optional) Closes an open bug in Launchpad. LP#2071747

This should also fix https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/2083129 by not re-generating NM configuration on the fly. And https://bugs.launchpad.net/ubuntu/+source/netplan.io/+bug/2090848 by using the new /usr/libexec/netplan/configure --networkmanager-only flag.

@slyon slyon force-pushed the generator-split-sd-1997 branch 8 times, most recently from 8287a28 to 9105838 Compare July 7, 2025 14:34
@slyon slyon force-pushed the generator-split-sd-1997 branch 2 times, most recently from 15db9ad to 320629c Compare July 8, 2025 14:29
@slyon
Copy link
Collaborator Author

slyon commented Jul 10, 2025

Packaging TODOs

  • re-generate config in .postinst using usr/libexec/netplan/configure binary as the (new) sd-generator does not touch any systemd-networkd configuration (or its permission).

    Setting up libnetplan1:amd64 (1.1.2-7+DEV) ...
    Setting up python3-netplan (1.1.2-7+DEV) ...
    Setting up netplan-generator (1.1.2-7+DEV) ...
    Removing 'diversion of /lib/systemd/system-generators/netplan to /lib/systemd/system-generators/netplan.usr-is-merged by netplan-generator'
    Bail out! ERROR:../src/gen-openvswitch.c:533:_netplan_state_finish_sd_ovs_write: assertion failed: (generator_dir != NULL)
    WARNING: Netplan could not re-generate network configuration. Please run 'netplan generate' to see details.
    Created symlink '/etc/systemd/system/sysinit.target.wants/netplan-configure.service' → '/usr/lib/systemd/system/netplan-configure.service'.
    Setting up netplan.io (1.1.2-7+DEV) ...
    
  • Adopt NetworkManager capabilities, or avoid NM writing systemd-networkd configuration with root permissions, or rather call configure with the (new) --networkmanager-only parameter:

    /usr/libexec/netplan/configure --networkmanager-only
    
    printf "[Unit]\nCapabilityBoundingSet=CAP_CHOWN\n" | systemctl edit --stdin NetworkManager.service 2>/dev/null
    
    mkdir -p /etc/systemd/system/NetworkManager.service.d && echo "[Service]\nCapabilityBoundingSet=CAP_CHOWN\n" > /etc/systemd/system/NetworkManager.service.d/override.conf && systemctl daemon-reload
    
  • Install systemd-resolved for cloud-init & autostart DEP-8 tests

@slyon slyon force-pushed the generator-split-sd-1997 branch from 320629c to 4f36a61 Compare July 10, 2025 14:41
@slyon slyon force-pushed the generator-split-sd-1997 branch 5 times, most recently from b507c0d to d2bd8a8 Compare September 16, 2025 11:32
@slyon slyon force-pushed the generator-split-sd-1997 branch 4 times, most recently from 869fc6e to 1ee0b9f Compare September 29, 2025 15:58
@slyon slyon force-pushed the generator-split-sd-1997 branch 7 times, most recently from 42d4420 to 14a12f9 Compare October 8, 2025 12:05
@slyon slyon force-pushed the generator-split-sd-1997 branch from 14a12f9 to c1cd723 Compare October 9, 2025 10:58
slyon added 29 commits October 21, 2025 12:16
…/configure

Run using:
LD_LIBRARY_PATH=_build/src NETPLAN_DBUS_CMD=_build/dbus/netplan-dbus NETPLAN_GENERATE_PATH=$(pwd)/_build/src/generate G_DEBUG=fatal_criticals PYTHONPATH=_build/python-cffi:. pytest -s -v tests/generator/test_sd-generator.py
…rator and network-configurator

This is to make them fail under the same circumstances, e.g. do not generate
systemd units corresponding to network interfaces that we cannot write any
network configuration files for, due to some late-stage validation error.

Adding new (internal, for now)  _netplan_state_get/set_flags() API.

This is enabled by passing a new NETPLAN_STATE_VALIDATION_ONLY flag to
NetplanState, to skip writing out files.
- The sd-generator (./generate) should not write any network configuration, but still needs to run through its late-stage validations, to fail if the network-configuration would fail.
- The network-configuration should not write any systemd units, but still needs to run trhough its late-stage validations, in order to fail if the sd-generator would fail.

Future improvement:
Ideally, there should not be any late-stage validation inside file writing logic (e.g. [gen-]networkd.c, [gen-]openvswitch.c, ...), but all validation should already be done in the central validation.c stage. Unfortunately, such change is out of scope for this PR.
…#2090848)

This avoids conflicts with AppArmor confinement when re-generating the network
configuration from within NetworkManager.
It introduces unrelated failures and wastes resources, as we're not currently
affected by an ABI breaking change.
…nerator

This was originally implemented to generate & start systemd units just-in-time
during the boot transaction (canonical#162).

With the implementation of a proper systemd-generator, Netplan generates the
corresponding units in /run/system/generator* and automatically re-loads and
re-calculates dependencies during "daemon-reload". Therefore, we do not need
to inject them manually.

This is covered by the "cloud-init" autopkgtest.
@slyon slyon force-pushed the generator-split-sd-1997 branch from 6b93b00 to 49cfdcd Compare October 21, 2025 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant