Updated Frontend and Settings

[bwintermann]

During the last weeks when I had some time I worked on updating our frontend. @LinusJungemann had mentioned that we probably shouldn't hardcode the dependencies into the code, and I thought that the settings management was a bit difficult to understand.

With that in mind, and due to the several research topics that we are currently working on (that will need to be merged and configured), I reworked mainly the following things:

Implemented Features and Fixes
(Updated 03.11.25, 14.11.25, 19.11.25, 03-05.12.25)

Settings

• Settings are now managed by a dedicated Settings-class (FINNSettings) that can be used like any other class and will automatically adjust environment variables
  • The settings are being validated by Pydantic
• Globally available FINNSettings, replacing most uses of os.environ[...]
• The default location for FINN_DEPS is now at finn_deps instead of deps
  • The previous value would create a directory deps in site-packages. This could potentially cause confusion with other packages and doesn't mark the directory belonging to FINN, thus the change

Dependencies

• Dependencies are defined in a YAML file, defaulting to external_dependencies.yaml. This can be configured in settings and environment via FINN_DEPS_DEFINITIONS
  • finn_xsi is part of the dependencies now, not requiring an extra setup
  • Despite the installation being in DependencyManager now, finn_xsi still needs to be loaded in prepare_finn to make sure it's available when skipping dependency updates
• finn deps update got two more options:
  • --accept-defaults to accept missing settings files (explained below again)
  • --force to delete the dependency directory and freshly install all dependencies again

Commands / Click

• Reworked run_finn.py. Any click function is described by decorators that tell the name of the argument that they add to the function.
• Fixed several smaller issues (finn config subcommands for example)
• Verify input and verify output, as well as a output directory overwrite option (Add optional override of path settings from build config via CLI options #130)
• Added utility commands: finn deps {edit,show}
• Added utility commands: finn config {create,edit,show}
• Wizards to setup FINN+ and flow configurations (automatically started upon missing settings, but skippable via --accept-defaults)
  • The configuration wizard can also be manually started with finn wizard config or finn config create
  • The flow config wizard can only be started manually using finn wizard flow
• By default includes all DataflowOutputTypes, so you never accidentally forget them
• The model can now be passed in the config in the field model_path. This way it's possible to run finn build cfg.yaml
• New command finn auto tries to run FINN by automatically finding a flow config and a model. It searches for {cfg,config}.{yaml,yml,json} and model.onnx primarily, but if one isnt found defaults to the first available .yaml/yml/json/onnx file
  • So if you have config.yaml and model.onnx in the same directory, simply running finn auto will find these and start FINN. Note however, that this command does not have any of the other configuration options. If no settings are found this will also automatically start the setup wizard as well.

Tests

• Single tests can be executed from the CLI now as well: finn test --variant custom --name <name>, where name can be a test specification in pytest syntax (like test_dir/complex_tests.py::ComplexTest::test_function, etc.)
• Removed tests concerning build_dataflow_directory, since the function itself is removed as well
• Updated session fixture to use the new global settings. When calling finn test, the main functions sets FINN_SETTINGS to the given file so that the tests receive these settings as well

Other

• Introduced the --batch parameter. This disables the wizards, forces default values if no settings were found and disables interactive / live displays
• Fixed a bug where when setting output_dir: ~ in the flow config, a directory named None would be placed in ~. This is fixed now
• Fixed a bug where in certain cases freshly installed packages were not immediately available. This bug might have existed before the PR already but was possibly shadowed by something else
• Docstrings for various methods that needed them after editing the matching files (@LinusJungemann)
• GHA don't add new comments everytime the docstring checker notes something, and instead modifies the existing comment (@LinusJungemann)

Backwards Compatibility
No:

• Since finn can now start the wizards when some files are missing (and you have to specifically pass --accept-defaults to avoid this), some automated scripts might break.
• finn deps and finn config commands were renamed
• Renaming of a few command line parameters

Open

• --skip-dependency-updates causes XSI to not be loaded when running finn test
• Finish --batch option implementation for all commands that can use it
  • NOTE: Remember to add this in futures PRs with interactive console behaviour as well

[bwintermann]

On @fpjentzsch suggestion moved our frontend to use Pydantic. There are still some bits that need fixing, but it does start normally. We can simply and read from and assign to settings and it gets saved into the environment directly as well. Also simplifies our main run_finn.py quite a bit.

[bwintermann]

@fpjentzsch I fixed several issues. The PR is ready for merging. I had the thought of making FINNSettings available everywhere in FINN to completely replace the environment variables, however a singleton wouldn't work with multiple FINN instances running, and passing the settings object down requires a lot of (compatibility breaking) changes. If there is another way to do it, I'd gladly implement it.

@LinusJungemann I am unsure, but I think the Builder Config Documentation Check might be broken, atleast it doesn't list the undocumented fields (I didn't add any either), and it fails with a syntax error on the match clause (Python version too old?)

Edit: Actually providing a settings singleton (and some nicer output along the way) might be possible. Ill see the next weeks if its doable.