Generating meson.build files at configure time, is this "allowed"

[LunarLambda]

Ignoring for a moment the obvious insanity of doing this, is something like this valid insofar that meson won't at some point decide to break the following:

project('foo')

# Write a meson.build file to `.foo`
configure_file(output: 'dummy', command: ['sh', '-c', '''cd $MESON_SOURCE_ROOT; mkdir -p .foo; echo "message('test')" > .foo/meson.build'''])

# Execute it
subdir('.foo')

To explain the intended use of this, take for example more complicated asset processing pipelines or build steps that depend on declarative data outside of meson and/or may produce a non-fixed number of outputs, or non-deterministic filenames (e.g., content-addressed/hashes). These things are not possible to statically express in meson with things like generator or custom_target, or would be extremely verbose or time-intensive to do so, leaving the best option at a kind of meta-build step. And to avoid the user needing to manually rerun said step, it is integrated directly into meson's logic by using configure_file and subdir, allowing meson to automatically reconfigure/regenerate the project when needed.

While I understand this falls firmly into "meson is not intended to be used this way", the alternatives aren't much better: extremely convoluted makefiles with questionable dependency tracking, or convoluted and bespoke Cmake macros/abstractions, or writing a wholly bespoke build system or meta-build system altogether...

This at least somewhat retains a degree of reproducibility/debuggability, which is why it would be nice if it came with a kind of uneasy "you shouldn't do this, but we won't break it either" peace ^^

[dcbaker]

I'm against any attempt to guarantee this, as I could not possibly guarantee it with the architecture of meson++.

I would really like to see what you're attempting to do, as I suspect it could be done with a clever combinations of fs.read(), a foreach loop, and set_variable() and get_variable()

[LunarLambda]

I'm trying to simplify the inclusion of assets in my meson toolkit for homebrew game development.

As it stands, the existing tools I'm integrating with are difficult to use directly and somewhat verbose to invoke from meson, due to needing complicated commands to put files in the right place (i.e., where meson expects them, as an example I've had some issues with using placeholders like @INPUT@ in a way that meson would accept), or things like turning the built ROM images into flat binaries with custom_target, etc.

There is also the question of linking scripts, which have a lot of potential "options" the user might want to set, such as which RAM areas to use, etc. Which presents a combinatorial problem probably most easily solved by generating the final script from a template and including it

As such I would like to provide a wrapper tool that takes a simplified declarative description of all the assets, determines/generates the proper commands and (ideally) produces a meson dependency object with all of the linker arguments, asset files & generated headers, etc., such that the user can simply add it to their executable and everything gets included properly.

I figured a good way to do this might be to have the tool generate a meson.build with all of the appropriate targets, seeing as meson lacks functions or the ability to load modules "out of tree", as it were

[virtuald]

I have a similar problem and after looking at the problem hard for a month or two I decided that the simplest route would be to generate meson.build files before running meson -- it's really the only approach that fits with meson's design. meson intentionally makes it difficult to do dynamic behaviors like this, and instead of fighting that just go with it and generate your meson.build.

My approach was to put all the generated meson.build files in subdirectories that way I can still write normal meson build rules by hand elsewhere.

[eli-schwartz]

Ignoring for a moment the obvious insanity of doing this, is something like this valid insofar that meson won't at some point decide to break the following:

As @dcbaker alluded to, the most likely practiceal breakage would be if a meson implementation parsed the build files in parallel and then attempted to implement different configure_files and run_commands and compiler checks and whatnot, in parallel as long as the AST can be lowered and determined that that particular node is definitely going to be executed.

Its unlikely at the moment that the reference implementation in Python will gain such abilities, though, so I guess I'd say "we don't currently have any plans that are likely to have the side effect of breaking this loophole".

It's possible that using a run_command for this instead of a configure_file, and then descending into the subdir based on the result of checking the return code of the run_command(), would be sufficient to fool even such alternative parallel approaches. Reason being, would e.g. meson++ actually open and parse a subdir() even for static AST generation, if the subdir() call itself is not yet guaranteed to run? I would assume not, because for example that would mean that if the subdir file has an error in it then meson.py would ignore the error but meson++ would be unable to actually do anything about the error until it rechecks that the code actually runs. (Or it would incorrectly fail the build!)There's not really anything to parallelize until it is known.

[dcbaker]

Let me be more clear then. This will absolutely not work in Meson++ right now and I could never realistically make it work without giving up some of the advantages it gives.

Meson++ does, in fact, resolve all subdir calls statically at AST creation time, unless it cannot determine what subdir to open (consider loops of subdir() or selecting a subdir like x = host_machine.system() == 'windows' ? 'window' : 'posix'; subdir(x). This latter part is a really annoying because it means that you have to do some amount of mid level IR reduction, then create new AST, lower that, insert it into the tree, and then recalculate SSA values. It also creates AST out of all branches of conditionals where Meson and Muon don't, which means it can find more bugs for you :D

[eli-schwartz]

"easy" solution, subdir(runresult.stdout().strip())

ducks and runs for the hills. Hey officer it's not my fault, this thread was about finding ways to do the uneasily insane!

which means it can find more bugs for you :D

Two-edged blade. 😜

[dcbaker]

Indeed, it's definately a two edged blade.

Honestly, solving the subdir that cannot be resolved at AST time is a big pain for me with Meson++, because it's really complicated. That and how to handle loops are probably the two most difficult things I have left to do!