add an option for skipping compilation of hybrid packages

[daylinmorgan]

TL;DR add support for --srcOnly flag so install, setup don't compile binaries of hybrid packages

I rely on nimble setup (always in local mode) to fetch dependencies. But for none of the hybrid libraries I've used as dependencies have I ever needed the binaries those packages define/provide (for example karax).

I think it would be useful to offer this an opt-in feature for those cases when users are aware they are installing a hybrid package and have no need for the binary.

Relatedly, I maintain a nix flake that is based on nimble and ingesting nimble.lock files. In this case the compiled outputs are noise that at best consume disk space and at worst a possible failure mode given the build system quirks of nix when they are likely not relied on for a package that is using libraries as a dependency.

Granted nix problems are not nimble's problems but I still think this QOL feature would be useful when developing and installing dependencies.

To avoid this problem for my own hybrid packages I've taken to defining binaries in a subdirwith their own .nimble to keep library installation limited to src files.

I am not aware of any library that requires both its src and binary to function as a library but I imagine there is a way to do that using after install tasks. And if you give folks the ability to something they are sure to find a way, so maintaining the current behavior with an explicit option to disable it seems safer.

[arnetheduck]

I am not aware of any library that requires both its src and binary to function as a library

https://github.com/status-im/nim-testutils/ is one such example, or more generally build tooling (code generators etc)..

Arguably, this would be a property of the package rather than a flag that the user passes - if I depend on testutils I don't want an arbitrary (and global for all packages) skipBins flag to mess that up.

[daylinmorgan]

AH! Thanks for the use case!

Arguably, this would be a property of the package

I'm not sure what you mean?

if I depend on testutils I don't want an arbitrary (and global for all packages) skipBins flag to mess that up.

But if you don't use the --skipBin flag the behavior wouldn't change.
Unless you mean you don't want to ALLOW a user to accidentally do this (but a subsequent nimble install or nimble setup would resolve this).

[arnetheduck]

Unless you mean you don't want to ALLOW a user to accidentally

I mean that the requirement might be somewhere deep in the (transitive) deps and the user is unlikely to know -> not a great candidate for a flag.

Imagine a dep graph a->b->c and c creates a tool that b uses.. the developer of b knows that we need the tool from c (or not) and should be the one choosing whether the tool gets built - the user that is installing a shouldn't have to care - either b needs c with the binary or it doesn't, it's not like it changes from day to day or from user to user.

Further, imagine that a also depends on d and d installs a binary that a doesn't need - in this case, there's no correct setting for --skipBins: either b breaks or you get unnecessary d binaries.

To solve for both cases, either b needs to say "I need the binaries of c" somehow or there needs to be a special category for "library binaries" so that "c" can say "install these binaries if you depend on me as a library" (similar to developer deps / feature flags).

[daylinmorgan]

I mean that the requirement might be somewhere deep in the (transitive) deps and the user is unlikely to know -> not a great candidate for a flag.

They would know after nimble install a --skipBin fails and that information could be front-loaded by a message from nimble saying `warning "skipping compilation of binary provided by deps: c"

Further, imagine that a also depends on d and d installs a binary that a doesn't need - in this case, there's no correct setting for --skipBins: either b breaks or you get unnecessary d binaries.

For this case --skipBins could accept a comma separated list of binaries so one could run nimble setup --skipBins:d?
Then you would get the needed compilation of c without the unneeded compilation of d.

But yes without giving nimble additional information in the form of some kind of package metadata (requiresBins?) users would need to encounter a failure mode, to learn that a transitive dep is compiling code that a library uses.

nim-testutils sounds like an example of something I don't want or need to compile (with nimble setup) if it's just a transitive dep (not gated by the new "features" feature (pun intended)) by a library I use tests.

[arnetheduck]

They would know after

Unlikely (once the Nim ecosystem reaches any kind of maturity) - ie in developed ecosystems, a typical package has tens if not hundreds of (transitive) deps and all the user will know is "it didn't work, there's some annoying warning on screen too about something I don't care about".

sounds like

Well, it's also used for all kinds of code gens, ie afair futhark compiles itself then uses that binary to generate nim mappings for c files based on what the user has installed locally - the point is that it's a legitimate use case.

Whether you need testutils in particular or not in a transitive dep in particular different matter - ie if c wast testutils and it was only needed to run tests, b should declare it as a dev requirement so it's not considered.

Anyway. A flag will likely not work for long, I'd rather see this fixed properly at the source rather than adding another flag that must be maintained.

But yes without giving nimble additional information

this is the way.

[daylinmorgan]

the point is that it's a legitimate use case.

Sorry if I made it seem like I thought this wasn't a valid use case, it absolutely is!

I think the cause and effect of running nimble build --skipBin:opir followed by the message BUILD FAILED: opir not found is fairly straightforward for someone, but I may not be properly considering the novice.

I think I'm also mainly concerned about the current status quo in the ecosystem in which libraries with "extra" tools already exist and function independent (like karax), which I think is a different case from nim-testutils and futhark, but I guess upstream could also just make use of the new "features".

I'd rather see this fixed properly at the source rather than adding another flag that must be maintained.

The need for this make sense for the futhark or nim-testutils situation that we declare in metadata the binary is "not optional" and if desired can fail at the nimble build|install|setup but for something like karax, in no way does using the library require the defined binary.

Features would help for transitive dependencies but not when a direct or downstream dep doesn't use them (as I doubt many libraries currently make use of feature "dev" or dev in their nimble files, hopefully we can see it employed more for these situations)

---

I recognize the potential for increased build errors from user misuse but the failure mode feels straightforward when they go out of their way to opt-in to not building a particularly binary, feels like they are accepting the consequences.