Fatbuildr v2.1.0 is available!

[rezib]

Rackslab is happy to announce the release of Fatbuildr v2.1.0!

Packages are available for easy installation on many Linux distributions including:

• RHEL8
• RHEL9
• Debian 12 stable « bookworm »
• Debian 13 testing « trixie »
• Debian unstable « sid »
• Ubuntu 22.04 LTS « jammy »
• Ubuntu 24.04 LTS « noble »
• Fedora core 39
• Fedora core 40

Follow the quickstart guide!

For reference, the full (and huge!) set of changes:

Added

• Add man command in OSI format container image.
• Additionally to *.tar images, publish OSI registry all other images formats supported by mkosi (Support publishing more OSI formats in registry #112).
• Give the possibility to run OSI builds directly on host instead of container to support images formats for which mkosi utility require access to loop devices (Support building OSI images outside of container #111).
• Support multiples OSI images in the checksum file published in the same registry derivative directory (Support multiples OSI files in checksum file in a registry directory #114).
• Possiblity to define patches templates with variables to replace (Make patches templatable #128).
• Support generation of prescript tarballs with subdirectory at any depth in source tree (Support subdirs for prescript tarballs #154).
• Support distribution and format specific tokens of prescript rules to allow definition of different values for different distributions and formats (Distribution specific prescript deps #156).
• Support installation of DNF modules as prescript dependencies for RPM packages builds (Add possibility to install RPM module in mock environment with prescript #155).
• Automatically exclude from generated archive files untracked by git (ex: referenced in .gitignore) when building or managing patches from local source tree with initialized git repository (Add possibility to ignore files in source tree when generating local source archive #153).
• Possibility to declare in instance pipelines definitions DNF modules to enable in Mock build environments (Possibility to enable modules in mock environments #163).
• Add reference to build task ID and instance name in RPM/deb package changelog entries (Add task ID in RPM/deb package changelog entries #15).
• Add result exportable field to all runnable tasks.
• Possibility to execute hook program before and after tasks are run (Hooks for tasks start and end #19).
• Add prep templating variable for RPM spec files as shorthand for prep_sources and prep_patches (Add common {{ prep }} token for RPM spec files #89).
• Bash completion script for fatbuildrctl (Add bash completion for fatbuildrctl #28).
• cli:
  • Possibility to list artifacts in registries in remote instance with the REST API (Add possibility to list registries content with fatbuildrctl on REST API #141)
  • Possibility to remove artifacts from registries in remote instance with the REST API (Add possibility to remove artifact from registries with REST API and fatbuildrctl #142)
  • Add --include-git-untracked option to build and patches commands to avoid automatic exclusion from generated archive of files untracked by git in local source tree.
  • Possibility to execute command in arguments of images shell and images env-shell (Accept command to run with fatbuildrctl images #98).
  • Support $FATBUILDR_URI environment variable to override value in user preferences file (Override URI in user preferences with environment variable #124).
  • Add git message template for patches to help filling and formatting patches metadata with expected fields and values (Preformat messages in fatbuildrctl patches commits #30).
  • Add --batch option to fatbuildrctl keyring command for keyring creation and renewal tasks.
  • Add size attribute to artifacts found in registry.
• web:
  • Report fatbuildr version in footer of fatbuildrweb HTML pages (Report fatbuildr version in web application HTML pages #108).
  • Support file listing and folders browsing in registries (Add list dir features in fatbuildrweb registry content #65).
  • List RPM/Deb binary artifacts content (Get list of files in binary artefacts in web app #22).
  • Display artifacts sizes in derivatives and artifacts pages of the web interface (Get artefact sizes in web app #23).
• api:
  • Add edit-registry permission action.
  • Possibility to remove artifact from registries with DELETE verb on artifact route.
  • Introduce ArtifactContent objects.
  • Add content key in ArtifactDescription object associated to a list of ArtifactContent objects for Deb and RPM binary packages.
  • Add size attribute to Artifact objects.
• conf:
  • Add containerized parameter in [format:osi] section.
  • Add exec_cmd and exec_tmpfile parameters in [format:deb] and [format:rpm] sections.
  • Add support of multiple build environments initialization commands.
  • Add optional env_default_modules parameter in [format:rpm] section.
  • Add listing boolean parameter in [web] section to control activation of listing feature in fatbuildrweb.
  • Add hook parameter in [tasks] section.
• pkgs:
  • Add patch to define upstream version at build time.
  • Install tasks hooks examples uncompressed.
  • Install default system-wide git commit message template for patches in both Deb and RPM packages.
  • Install fatbuildrctl bash completion script in both Deb and RPM packages.
  • Install fatbuildr tmpfiles.d configuration.
  • Add bootstrap-icon additional source archive.
  • Install bootstrap icon fonts in Deb and RPM packages.
• dbus:
  • Add ArtifactDeleteAs method to org.rackslab.Fatbuildr.Instance object to submit artifact deletion task with another user identity.
  • Add ArtifactContent method to org.rackslab.Fatbuildr.Instance object to retrieve the content of binary artifacts.
  • Add command argument to ImageShell method.
  • Add command argument to ImageEnvironmentShell method.
• polkit: Add org.rackslab.Fatbuildr.edit-registry-as action.
• docs:
  • Mention containerized parameter in [format:osi] section of system configuration.
  • Mention exec_cmd and exec_tmpfile parameters in [format:deb] and [format:rpm] sections of system configuration.
  • Mention new env_default_modules optional parameter in [format:rpm] section of system configuration.
  • Mention listing parameter in [web] section of system configuration.
  • Mention hook parameter in [tasks] section of system configuration.
  • Mention new modules optional parameter for RPM distributions in instance pipelines definitions.
  • Document patches DEP-3 metadata support with Fatbuildr specific fields and their management with fatbuildrctl patches subcommand.
  • Mention patches deb822 Template field and the patches templating feature.
  • Mention possibility of missing JWT signing key for REST API HTTP/404 response code.
  • Document impact of HTTP reverse proxies buffering on tasks output streams with mention of possible configuration settings.
  • Document HTTP reverse proxies settings of interest to allow submission of large artifact builds and avoid timeout in live task output.
  • Mention org.rackslab.Fatbuildr.edit-registry-as Polkit action.
  • Mention edit-registry REST API permission action.
  • Mention REST API route to delete artifact.
  • Mention syntax, usage and behaviour of distribution and format specific tokens of prescript rules.
  • Mention possibility to install DNF modules as prescript dependencies with module: prefix.
  • Mention new --include-git-untracked option for build and patches commands in fatbuildrctl manpage.
  • Mention new command option for images shell and images env-shell commands in fatbuildrctl manpage.
  • Mention support of $FATBUILDR_URI environment variable in fatbuildrctl manpage.
  • Explain upstream debian/ directory in present artifact archives is removed and replaced by Fatbuildr during Deb packages builds.
  • Add two examples of tasks hooks in Python:
    • Basic hook to send syslog message.
    • Advanced hook to send custom notifications messages (based on templates) on Matrix room.
  • Add Tasks Hooks page.
  • Mention new prep templating variable in artifact definition reference documentation for RPM spec files.
  • Mention new commit_template parameter in user preferences documentation in fatbuildrctl manpage.
  • Mention new --batch option for fatbuildrctl keyring command in manpage.
  • Mention content section in binary artifact web interface page.
  • Mention content key in ArtifactDescription object and introduce ArtifactContent object in REST API reference documentation.
  • Mention new size property of Artifact object in REST API reference documentation.

Changed

• Rename Mock and Cowbuilder build environments to add fatbuildr- prefix.
• Support mkosi v22 in osi image (Support mkosi 22 for osi artifacts #173).
• Factorize RPM repository updates by running it once for all packages that share the same architecture (Factorize RPM repository update when publishing multiple package #49).
• Replace fatbuildrd service runtime directory by fatbuildr tmpfiles.d configuration with POSIX ACL to give access to subdirectories to fatbuildr system user and benefit from automatic cleanup by systemd-tmpfiles.
• cli: Watch task output by default when submitting tasks. The -w, --watch option is replaced by the opposite --batch option to submit tasks in background (Make --watch the default and add command to submit task in background #123).
• conf:
  • Rename init_cmd→init_cmds parameter in [format:deb] and [format:rpm] sections of system configuration.
  • Bump Fedora release from 38 to 40 in rpm and osi container images.
  • Install all mkosi 22 optional dependencies required to get all features and assign newuidmap/newgidmap ranges for Fatbuildr system user in osi container image.
  • Install podman for Mock in rpm container image.
• docs:
  • Update table of available remote features with new registry content listing and artifact deletion possibilities.
  • Mention possibility of HTTP/404 response code on supported derivative in REST API reference.
  • Split documentation of images command options by subcommands in manpage.
  • Replace -w, --watch option by opposite --batch option in manpage.
  • Remove mention of --watch options in various pages of documentation.
  • Use fatbuildrctl shell images <command> in troubleshooting page.
  • Rename init_cmd→init_cmds parameter in [format:deb] and [format:rpm] sections of system configuration.
  • Mention usage of %autopatch macro behind prep_patches templating variable in artifact definition reference documentation for RPM spec files.
  • Update supported Linux distributions in quickstart guide (drop Debian 11, Fedora 37-38, add Debian 13, Ubuntu 24.04, RHEL9, Fedora 38-40).
• pkgs:
  • Use new prep variable in Fatbuildr RPM package spec file template.
  • Bump boostrap version to 5.3.3.

Fixed

• Fix crash on client side when loading artifact definition for OSI builds (Crash on client side during OSI build #100).
• Fix crash due to concatation with incompatible types when defining the full release for OSI build on server side (Crash with fullrelease() during OSI build #101).
• Add apt command in container for OSI image to meet Debian and Ubuntu based images build requirements (Debian-based OSI image build error due to missing apt-get command #102).
• Fix systemd-nspawn execution error through mkosi in OSI format container caused by unavailability of DBus system session (Command to remove documentation fail when building OSI images #103).
• Fix GPG keys unsupported filetype error when running apt-get update on build of Debian OSI images due to missing cmp command (Unsupported GPG key format error when building OSI images with apt #104).
• Search mkosi output image and checksum file in distro~release subdirectory in OSI builds (Unable to find SHA256SUMS file generated by mkosi --checksum after an OSI build #105).
• Fix permission error on OSI artifacts produced by mkosi by faking sudo environment (Permission error to sign mkosi SHA256SUMS #106).
• Fix AttributeError in fatbuildrd when publishing OSI artifacts (AttributeError when publishing OSI artifacts #107).
• Make RegistryOsi ensure instance registry directory exists (RegistryOsi does not ensure instance registry directory exists #109).
• Add missing chattr command in OSI container image (OSI build might fail because of missing chattr command #110).
• Fix wrongly filtered out files containing debian or .git words in their path when building local archive on build submission (Some files are wrongly discarded when building local archive #113).
• Fix unusable version.dist variable in packaging code templates (The value of version.dist cannot be used in templates #116).
• Fix crash on build on unsupported derivative in instance pipelines (Crash on build on unsupported derivative #117).
• Fix crash on retrieving derivative version in artifact definition (Crash on retrieving derivative version #118).
• Fix crash when main archive source is not defined on build of Deb or RPM package (Crash when deb or rpm main archive source is not defined #119).
• Fix crash on missing checksum for a specific version in artifact definition file (Server crash when checksum is missing for a specific version in artifact definition file #125).
• Fix crash in fatbuildrweb 404 error handler when HTTP request view arguments are not defined due to error during view matching (Variable definition bug in 404 error handler #126).
• Fix retrieval of instances list in DBus and fatbuildrweb when default instance is not defined (Fatbuildrweb is unable to list available instances when instance default does not exist #127).
• Fix crash in fatbuildrctl when user preferences file is missing (fatbuildrctl crash when preferences file not present #130).
• Fix crash in fatbuildrd when running containers with empty init_opts in site configuration file (Unable to define empty containers init_opts in site configuration #131).
• Return HTTP/404 with appropriate error message instead of HTTP/500 (internal error) when trying to access exported armored public key on nonexistent keyring with fatbuildrweb (Fatbuildrweb fails on keyring.asc HTTP request when keyring does not exist #133).
• Report error properly with fatbuildrctl when remote HTTP instance replies with internal error and HTTP/500 (Avoid fatbuildrctl crash when fatbuildrweb gives HTTP/500 #136).
• Report meaningful error message instead of generic HTTP/500 internal error when authenticating with JWT token on unexisting remote HTTP instance (Avoid meaningless HTTP/500 generic error when submitting JWT token against unexisting instance #137).
• Fix crash with when watching streamed tasks output with HTTP REST API (Error when watching task output on HTTP server #138).
• Avoid buffering with HTTP response headers on reverse proxies (Task output streaming latency due to buffering with reverse proxies #139).
• Fix crash of fatbuildrweb on builds with unsupported derivative (fatbuildrweb crash due to unsupported derivative #149).
• Fix crash of fatbuildrctl on unexpected end of task output (fatbuildctld crash on StopIteration on unexpected end of task output #147)
• Fix crash of fatbuildrctl on task output connection closed by HTTP reverse proxy (fatbuildrctl chunk encoding error caused by timeout of upstream server on HTTP reverse proxy #148).
• Sanitize PRESCRIPT_TARBALLS names for correct detection by Debian build system.
• Enable network access to run prescript in Mock during RPM build.
• Prescript failure due to missing groupadd/useradd commands (passwd package) in Debian sid build environment (Prescript fail due to missing passwd in sid build environment #169).
• Conflict between fatbuildrctl --uri option and fatbuildrctl tokens save --uri option that prevent fatbuildrctl tokens generate from connecting to an instance other than the default (fatbuildrctl tokens --uri option conflict #168).
• Support artifact archive with existing debian/ folder. During deb packages builds, this upstream debian/ folder is removed and replaced by one generated with the artifact definition (Handle artifact archive with debian/ folder #174).
• Detect console unix socket closed by server, generally due to unexpected fatbuildrd error, in order to avoid endless loop and properly stop the console on client side (fatbuildrctl and fatbuildrweb) with error message.
• Check OSI artifact checksum file is properly created by mkosi or raise task execution error to report in task journal.
• Check container image and build environment exist or fail with appropriate error at the beginning of build tasks (Check image and build environment exist before build #17).
• Fix fatbuildrctl crash when RPM spec file is not found (Avoid crash of fatbuildrctl when spec not found #165).
• Use modern %autopatch macro instead of loop of %patchN to avoid deprecated syntax error during RPM packages builds (Fix RPM %patchN macro deprecation error #170).
• Fix crash of fatbuildrctl on missing source definition in YAML artifact definition file (Crash on missing source in artifact.yml #171).
• Handle JSON decode error in fatbuildrctl in case of unexpected failure in fatbuildrweb (fatbuildrctl crash on JSONDecodeError when server does not respond JSON #146).
• Report error instead of crashing in fatbuildrctl when unable to find format corresponding to a given distribution with fatbuildrweb REST API (Crash on missing format in remote instance #172).
• Validate keyring renewal duration format in server-side fatbuildrd and client-side fatbuildrctl (Check format of fatbuildrctl keyring renew duration input #31).
• Fix reporting of keyring errors during keyring creation and renewal tasks execution.
• Drop off artifact and sources archives tarballs in subdirectory of fatbuildr runtime directory to avoid potential collisions between multiple builds in queue (fatbuildrd crash due to archive file not found #179).
• Fix flask.helpers locked_cached_property import error in recent versions of Flask (Import error on flask.helpers locked_cached_property #190).
• docs:
  • Add missing path parameter in REST API to retrieve artifact information.
  • Add missing optional architectures parameter in instances pipelines definitions reference documentation.
• pkgs: Add missing dependency on patch package (Missing dependency on patch command #145).

---

This discussion was created from the release v2.1.0.