Question: What are the steps to integrate a custom programming language?

[lolgab]

I would love to integrate Scala Native in nginx unit.
It:

• Supports C ABI for calling functions passing structs (only by pointer) and set callbacks
• Can't generate dynamic libraries
• Isn't thread-safe ( you can't have callbacks that allocate memory with the GC from other threads than the main thread )

Since it can't export dynamic languages I guess the only option is the External language module.
Is there any documentation of the process to integrate a new language?
What is the rough idea behind the external language module mode?
Is it possible to integrate such a language given the constraints I listed above?

Thank you very much if you want to answer.

[mar0x]

Hello,

Thank you for interesting Unit!

To communicate with applications Unit uses UNIX domain socket pairs and shared memory. The protocol is too complicated and modified too often to try to document it. Instead, we've introduced a library libunit which hides the details of implementations and simplifies the language module creation process.

The external and internal (aka language module) modes differs only by the way the application starts. Interpreted languages needs an interpreter which can be linked with language module and initialises both language environment and Unit application environment. External applications need to be started by Unit main process to inherit required file descriptors for further communication with Unit. So, the only difference is initialisation steps and libunit supports both modes.

Currently libunit compiled as a static library because we don't have a requirements yet to make it dynamic. Your application need to be linked with library of the same version as Unit to ensure compatibility.

Library does not starts any threads. The application may start threads, but callbacks will be called in same thread as library functions called.

Summing the above, I would say all listed constraints are satisfied.

Unfortunately, the library is not documented yet, but you may start learning it by reading sample C application source code src/test/nxt_unit_app_test.c. You can build it using --tests configure flag and make tests target. Next, you may read other language modules and ask questions here.

[lolgab]

Thank you very much for your explanation @mar0x.
I think all the requirements are satisfied for having this working.
I will read the examples and let's see where I arrive 😃

[lolgab]

I wanted to let you know that I started working on my Unit integration for Scala Native and I have to say it is fantastic! In my opinion it is by far the easiest way to implement a http / websocket server on a new language that has interoperability with C.
In a weekend I got a basic hello world working that gave me 60k requests per second!
Looking forward to make my thin library a little better and publish it :-)