"Internal" binding specs in TH mode

[edsko]

Suppose a user wants to use bindgen in TH mode, and they want to generate multiple modules, perhaps corresponding to multiple main haeders in a given library. If header B uses types defined in header A, we need some kind of "internal" binding spec, that we could pass from module A to module B. A similar situation arises if a user wants to emulate the preprocessor in TH mode, and generate a .Safe and .Unsafe module.

We had previously considered and then dismissed the generation of binding specs in TH mode (#1046), on the grounds that TH binding specs "don't really exist"; the spec is not useful, because there is no re-usable set of bindings that could be used elsewhere and needs a description. This ticket is more narrowly focused: this is using bindings from one module in another, within the same project.

[TravisCardwell]

Using external (and prescriptive) binding specifications in TH is fine, but generating binding specification is problematic because TH should not write files, especially when users use an IDE (HLS). The best workaround that I know of would be to only write the generated binding specification if the file does not already exist. After changes, users would need to remove a binding specification in order for it to be regenerated.

My understanding is that TH users can create multiple modules by creating the .hs files for those multiple modules and use TH in each module to generate bindings using the same settings except that each module specifies a different "category." Is this (still) accurate? As long as users follow the same naming conventions, perhaps we do not need binding specifications for this case.

---

Is there a way that we could pass binding specifications without using the filesystem? One idea is to export the external binding specifications from the module itself. (TH users must have hs-bindgen as a dependency anyway.) For example, A.hsBindgenExtBindingSpec could be used by moduleB. (We can always generate hsBindgenExtBindingSpec, and users can decide if they want to export it or not.)

[edsko]

Is this (still) accurate? As long as users follow the same naming conventions, perhaps we do not need binding specifications for this case.

Discussing this with @TravisCardwell , let's clarify: this could perhaps work for generating multiple modules for the same header, with the same settings, etc., but just selecting out different binding categories. For example, if module A has all the types, then we still generate everything for module B, without an external binding spec, but then only "choose" the bindings generated for functions. We'd basically need to reference types that are "not" in scope as far as we know, and the user would need to add the necessary imports to the TH module to bring them into scope (that is, the TH code we'd generate would need to use unbound names, not bound names).

This is obviously a bit of a hack, and it won't work for the general case. Just to spell that out:

Suppose we have main.h

struct Vector {

};

and extension.h

struct Extension {
  struct Vector v;
};

then for Extension.hs we'd want to select struct Extension but not struct Vector.

[edsko]

Is there a way that we could pass binding specifications without using the filesystem?

I quite @TravisCardwell 's suggestion of having the external binding spec as part of the module itself; that's worth exploring.

There are also worse ideas: TH could in principle maintain state in some global IORef. This could work as long as we have a single ghc invocation, which I think will always be the case. But it's certainly a lot less appealing, and more brittle. If @TravisCardwell 's suggestion works, it would be much better. (For one thing, I think this would work when ghc gets called by cabal, but it's less clear how this will work in the hls case. It may or may not be workable.). This is definitely a plan B. Or C. Or D.