Skip to content

chore: add Flox development environment#251

Open
nvidiajeff wants to merge 9 commits intomainfrom
chore/add-flox-environment
Open

chore: add Flox development environment#251
nvidiajeff wants to merge 9 commits intomainfrom
chore/add-flox-environment

Conversation

@nvidiajeff
Copy link
Copy Markdown
Contributor

@nvidiajeff nvidiajeff commented Feb 28, 2026

Summary

  • Add a Flox (Nix-based) development environment as an alternative to make tools-setup
  • The .flox/env/manifest.toml tracks tools from tools/setup-tools and .settings.yaml on a best-effort basis
  • Provides an isolated, reproducible dev shell with a single flox activate command
  • Update DEVELOPMENT.md with Flox usage instructions and guidance for keeping the manifest in sync with .settings.yaml

Test plan

  • flox activate -- make tools-check verifies tools are available
  • Review DEVELOPMENT.md rendering for clarity

Add a Flox (Nix-based) environment as an alternative to `make tools-setup`
for setting up development dependencies. The manifest mirrors all tools
from `tools/setup-tools` and `.settings.yaml`.

Update DEVELOPMENT.md with Flox usage instructions and a prompt for keeping
the Flox manifest in sync when `.settings.yaml` tool versions change.
@github-actions
Copy link
Copy Markdown

github-actions bot commented Feb 28, 2026

Coverage Report ✅

Metric Value
Coverage 73.8%
Threshold 70%
Status Pass
Coverage Badge
![Coverage](https://img.shields.io/badge/coverage-73.8%25-green)

No Go source files changed in this PR.

Copy link
Copy Markdown
Member

@mchmarny mchmarny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We had Flox before as an alternative and since it was really used by one contributor it often lagged behind the frequently evolving main settings approach that allows the entire project (dev and CI/CD side to sync on versions while enable different deployment mechanism).

Some of the constraints on the GH actions that can be used in NVIDIA org means that the CI side for Flox also had to do some non-idiomatic stuff installation/updates). Even in this PR you can already see the coverage gaps between manifest.toml and .settings.yaml. The manifest claims to mirror .settings.yaml, but they're out of sync:

Tool In .settings.yaml In manifest.toml Notes
kwok v0.7.0 ❌ missing Used by make kwok-test-all
karpenter v1.8.0 ❌ missing
hugo 0.155.3 ❌ missing Docs builds
node 22 ❌ missing Docs builds
awscli2 ❌ not listed ✅ present Where does this come from?

A developer running flox activate would be missing tools that make tools-setup provides, which undermines the "drop-in alternative" claim.

Also, the DEVELOPMENT.md currently says .settings.yaml changes "propagate everywhere automatically." That's no longer true with Flox — the manifest requires manual updates. The sync instructions ("use an AI assistant prompt") are creative but fragile. Maybe could either:

  • Generating manifest.toml from .settings.yaml (a script in tools/ that templates it), or
  • Being explicit in the docs that Flox is best-effort and may lag behind make tools-setup

Version pinning

Only Go has a version constraint (^1.26). Every other tool is unpinned in the manifest — the lock file pins them, but flox update would pull whatever's latest in nixpkgs, potentially mismatching .settings.yaml. Adding version constraints (even just major version) would reduce surprise drift.

Housekeeping

Per project policy, please remove the "Generated with Claude Code" line from the PR description (see CLAUDE.md).

Summary

Larger, meta point, given the early stage of this project and the velocity with which it evolves, we want to keep things as aligned to the default, idiomatic OSS Go project experience.

I’m looking at projects like Kubernetes, Helm, Terraform, Docker, or Prometheus and I am not seeing any of them using Flox. While Flox does have some nice properties, unless we have consensus from other maintainers in this project to use Flox themselves and ensure ongoing support, I'm incline to recommend we hold back on introducing yet another tool for now.

@mchmarny mchmarny added the do-not-merge PR should not be merged or auto-closed label Feb 28, 2026
@dims
Copy link
Copy Markdown
Collaborator

dims commented Feb 28, 2026

I'm incline to recommend we hold back on introducing yet another tool for now.

I am in agreement with @mchmarny

@nvidiajeff
Copy link
Copy Markdown
Contributor Author

Sounds good - it was useful to me and thus I wanted to share. We can close this and hopefully revisit it once Flox/Nix are in more widespread usage.

It is incredibly useful to have an isolated environment for installing all these tools without conflicting with the requirements of other projects developers work on.

@nvidiajeff nvidiajeff closed this Feb 28, 2026
Add missing tools (kwok, hugo, nodejs), pin package versions, remove
awscli2, and fix DEVELOPMENT.md to clarify Flox is best-effort and
requires manual sync with .settings.yaml.
@nvidiajeff nvidiajeff reopened this Feb 28, 2026
@nvidiajeff
Copy link
Copy Markdown
Contributor Author

Re-opening 🙏 - I have made a best effort to address the feedback @mchmarny / @dims - if this is sufficient, please consider merging, if not I am ok if we want to close the PR and address it at a later date.

Aside from that, if I may offer my opinion:

I understand introducing a new paradigm for environment management is not a trivial thing, but the industry is indeed moving this direction and it is in part our duty to innovate.

Ideally we would use Flox OSS for all environments across dev/CI rather than maintain shell scripts that do not provide an isolated environment.

For those unfamiliar - Flox provides Python Virtualenv-like environments for all dev/ci dependencies while guaranteeing that the exact build of each tool is used end-to-end in all environments.

Replace 'flox install' with 'flox activate' in the sync instructions,
as activate is what re-resolves the lock from an edited manifest.
@mchmarny mchmarny added the lifecycle/stale Marked stale by automation label Mar 9, 2026
@lockwobr
Copy link
Copy Markdown
Contributor

I want to add my thoughts to this discussion. This week I found and fixed several issues in our current tools-setup, and it's making me wish it were easier to add and test new dependencies. Let me outline the problems I've been running into and how I think Flox could help address them.

Issues with the current approach:

  • Linux support — Several deps didn't work on Linux at all. Since most core devs are on Mac, the Linux side doesn't get the same exercise. Flox uses Nix packages under the hood, which are built and tested for both Linux and macOS from the same definition — no platform-specific install logic to maintain.
  • System-level dependencies — Some tools depend on system packages like Python (e.g., yamllint). Depending on your Linux distro, you may need to set up Python separately. I added apt support, but if you're on a Red Hat variant it just warns and skips. Flox resolves the full dependency tree, so yamllint would bring in its own Python without touching the system or caring about the distro's package manager.
  • No version locking on macOS — Many installs use Homebrew, which (as far as I know) doesn't have a good way to pin versions. That means running setup-tools today gives you different versions than someone who ran it weeks ago. Flox has a lockfile (manifest.lock) that pins exact package builds, so every developer gets the same versions until someone explicitly runs flox update.
  • Staleness — Twice recently I ran a command that failed because my system was missing something new. I had to manually track it down and add it. To @mchmarny point, Flox can also drift from .settings.yaml — but the current approach already has this problem, and debugging it is harder because you're dealing with platform-specific shell scripts rather than a single declarative manifest.

Related PRs from this week:

@nvidiajeff nvidiajeff removed the lifecycle/stale Marked stale by automation label Mar 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docs do-not-merge PR should not be merged or auto-closed size/XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants