Including both the front-end specific and the multiplexing header

[kranzj]

Including both the front-end-specific and the multiplexing header currently requires the programmer to obey a specific inclusion order - the front-end specific header version is required to be included before the generic header.

Header files should not put dependencies on one another. Every inclusion order should be valid.

[axel-simon]

To me the right way to view this is that every front-end is an independent module and should be emitted in its own "namespace" by prefixing all the function names. The multiplex library then gives all the different front-end a unified interface which has yet another set of names.
In this view, it is the task of the multiplex library to make the adjustment. One easy adjustment is to cast the pointer to a generic structure to the one specific to the backend. Checks that at least the sizes of the structures in the generic interface and the back end are the same, can give this interface some resilience.

jucs@ proposed to add a "module" system in which we somehow tell the compiler which of the structures are the "generic" interface part that should not be prefixed. Besides requiring modifications to the DSL compiler and the decoders, this would still pose linking problems when different definitions of the "generic" names in several decoders are linked together. We would have to restrict the "interface" to declarations only and guard against double declarations which seems non-trivial.

[kranzj]

Prefixing would complicate the development of software that does not use the multiplexing library extremely, since this software would need to call different functions depending on the front-end. Currently, all you need to do is link your software against a different library to change the front-end. Additionally, introducing casts almost everywhere results in not easy-to-read code. From my point of view, this change would "kill" the standard way of interfacing with GDSL and reduce the interface to using the multiplexing library.

For applications that use the multiplexing library, there could not be any size checking since the multiplexing library is independent of the front-ends. It only sees the front-end as object file when it is loaded. Of course you could invoke the compiler (e.g. gcc) whenever you load a specific front-end and parse errors, but I'm not sure this is a good idea and it would also require some development effort.

In my module approach, there would be no linking problems. The reason is that if modules were known to the compiler, it would not emit any function more than ones. Instead, it would know the module the function has been emitted in and refer to it. The GDSL compiler itself would therefore output multiple header files, one for each module. We could than compile the "common module" as a dependency of the multiplexing library. Since we'd need to be able to compile the modules independently, the compiler would also need to emit some internal interface representation it can later use to interface against (since reading in the C header file is not really an option) - a nice reference is the implementation of C++ modules by Microsoft. I see that this is not easy to implement, but this is the clean solution to the problem.

[axel-simon]

It might be good to support a few use cases well than all uses cases in a mediocre way:
If anybody potentially uses more than one front end, they should use the multiplex library and the generic interface.
If an application really only wants a specific frontend, they should use the prefixes in the function names.
The use case where an application links in a single front end with the generic names should use the multiplex library instead.
Even if we had the manpower to implement some kind of modules in the compiler, this would be a major time investment with very little impact: there is nobody who compiles in a single front end as using the generic interface (except for demo applications maybe). So why bother supporting this?

[kranzj]

Well, some people want to easily change the front-end but still not use the multiplexing library. One example is the way I'm using it right now. In fact, this would break my stubbing approach since I'd need to statically link against different names. I see that I can solve this by simply compiling the stub library multiple times (and changing the name using macros), however it would still complicate things.

Also there might be other people who want to use GDSL directly without the need to change the multiplexing library. From my point of view, the multiplexing stuff makes the interface way more heavy-weight. You need to understand an additional layer of abstraction to make changes. As far as I remember we decided against the the prefixing stuff for that very reason, I think it's still valid.

Altogether, I think this would be way worse than putting a restriction on the inclusion order of two header files. This is only a minor trade-off...

Additionally: Only few people will recognize this. Usually, people will either use the binding libraries or not use the multiplexing library. Thus, they won't run into the problem.

[axel-simon]

We also argued against trying to invent a module system in GDSL.
I'm ok with spitting out an x86 frontend with prefixes and one without prefixes so that the letter has the "generic" names.
In fact, why don't we build special, prefixed front-ends for the multiplex library?

[kranzj]

Yes, I know, but the reason was that a module system was considered too much work for not too much benefit. However, I've always had the opinion that with everything else we have to accept some hacks (since, in fact, we want to use modules although they're not there).

I remember the main argument against prefixes to be that people would need to adapt code to each front-end. For the multiplex library it would not hurt, but cause additional work with no real benefit...

[axel-simon]

With "additional work with no benefit" for the multiplex library you mean that multiplex needs to do the casts to the different prefixed data structures? I don't see how these casts would be a big problem.
So suppose we stick with the current solution. Maybe a good way would be to write some script that removes all definitions from a decoder that make up the "generic interface". We could run that script on all decoders and replace them with an #include of the generic interface. That way, we know that the definitions are the same across all decoders and we don't need to guard against multiple declarations?

[kranzj]

Not only cast, we also need to change the code to add the prefix to the strings generated to load the different callbacks. True, this is not much work.

The script would surely be possible. However, I still find it more intrusive to let a script run (which would be a hack anyway, since we would need to hardcode the common part) than to rely on inclusion order. Even with the script we would not win anything from a safety point of view - if the data structures don't fit together, that might still only strike at run-time and result in undefined behavior (at least as long as we don't require the cut out part of the front-ends to be char-wise equal to the common part, but that seems way to hacky to me).