Provide access to ModelicaUtilities in external functions compiled as dynamically linked libraries

[casella]

Problem statement

The Modelica Specification should allow library developers to write external functions that include calls to ModelicaUtilities functions, with their binary code provided by dynamically linked / shared libraries. Such external functions should be usable in the same contexts and with the same input variables as their regular Modelica function counterparts, to fully enable the replaceable/redeclare mechanism.

These are the detailed requirements:

1. The interface (input and output variables) of two functions that do the same job, one implemented by a Modelica algorithm and the other one implemented as an external function, should be exactly the same. Rationale: this is required so they can be swapped by the replaceable/redeclare mechanism.
2. The external functions should be callable both at Modelica compile time and at simulation run time. Rationale: regular Modelica functions defined by a Modelica algorithm can be evaluated at Modelica compile time, e.g. when they define the value of structural parameters or when the annotation(Evaluate = true) is set. Hence, corresponding external Modelica functions should be usable in the same contexts.
3. External functions should be able to call functions from ModelicaUtilities. Rationale: this is required by the MLS Section 12.9.6.
4. The binary code of external functions should be provided by one dynamically linked / shared library for each OS. Rationale: standard interfaces for .dll/.so libraries are defined by operating systems such as Windows and Unix, hence only one binary library file must be provided for each supported OS. Conversely, statically linked libraries are not guaranteed to work with versions of the C compiler other than the one that was used to compile them; this forces library developers to provide compiled binaries for each version of each compiler that can potentially be used by the final user to compile the simulation executable, constantly updating them as new compiler versions are shipped; this is a developer's nightmare. Also, calling functions in statically linked libraries at (Modelica) compile time is much less straightforward than calling functions in a .dll/.so library, e.g., via libffi.
5. A library containing external functions compiled into dynamically linked/shared libraries that call ModelicaUtilities functions should run in all Modelica tools and under all OSs. Rationale: external functions and ModelicaUtilities are both part of the MLS, so any library built following those rules should be runnable in any Modelica tool and operating system out of the box.
6. It should be possible to generate FMUs of Modelica models using such libraries. In this case, the behaviour of the ModelicaUtilities functions called by the external C function should be suitably mapped into the equivalent logging an error-handling mechanisms of the FMI standard.
7. How one can write external functions compiled in a dynamically linked/shared library that access ModelicaUtilities should be clearly explained in the Language Specification and should preferably not involve writing tricky C code in every new external function. Rationale: Modelica is a high-level modelling language, writing a Modelica library which involves some external C code should not require advanced or exotic C programming skills.

Minimal working example

The following library exemplifies the above-stated requirements in a MWE.

package ExternalFunctionsModelicaUtilities
  package BaseFunctions "Common interface declarations for functions"
    function size "Returns size of arrays based on name input"
      input String name;
      output Integer N;
    end size;
    
    function compute "Compute a function of x based on the name input"
      input String name;
      input Real x;
      output Real y;
    end compute;
  end BaseFunctions;

  package ModelicaFunctions "Modelica implementation"
    extends BaseFunctions;
    redeclare function extends size
    algorithm
      if name == "foobar" then
        N := 3;
      else
        assert(false, "Unsupported name");
      end if;
    end size;
    
    redeclare function extends compute
      input String name;
      input Real x;
      output Real y;
    algorithm
      if name == "foobar" then
        y := 1 + x^2;
        assert(y <= 10, "Value of x = " + String(x) + " could be too high", AssertionLevel.warning);
        assert(y <= 17, "Value of x = "+ String(x) + " is too high");
      else
        assert(false, "Unsupported name");
      end if;
    end compute;
  end ModelicaFunctions;
  
  package ExternalFunctions
    extends BaseFunctions;

    redeclare function extends size
      external "C";
    annotation(
      LibraryDirectory="modelica://ExternalFunctionsModelicaUtilities/Resources/Source/build",
      Library="functions",
      IncludeDirectory="modelica://ExternalFunctionsModelicaUtilities/Resources/Source",
      Include="#include \"functions.h\"");
    end size;
    
    redeclare function extends compute
      external "C";
    annotation(
      LibraryDirectory="modelica://ExternalFunctionsModelicaUtilities/Resources/Source/build",
      Library="functions",
      IncludeDirectory="modelica://ExternalFunctionsModelicaUtilities/Resources/Source",
      Include="#include \"functions.h\"");
    end compute;
  end ExternalFunctions;

  partial model M "The mother of all test models"
    replaceable package P = BaseFunctions;
    parameter String name = "undefined";
    parameter Integer N = P.size(name) annotation(Evaluate = true);
    Real y[N];
  equation
    for i in 1:N loop
      y[i] = P.compute(name, time/i);
    end for;
  end M;
  
  model M1M "Should fail at compile time with 'Unsupported name' error"
    extends M(redeclare package P = ModelicaFunctions);
    annotation(experiment(StopTime = 10));
  end M1M;
  
  model M2M "Shoud simulate until time = 4 with warning at time = 3, then fail"
    extends M(redeclare package P = ModelicaFunctions, name = "foobar");
    annotation(experiment(StopTime = 10));
  end M2M;

  model M3M "Same as M2M but N is provided by a literal, not by a function"
    extends M(redeclare package P = ModelicaFunctions, name = "foobar", N = 3);
    annotation(experiment(StopTime = 10));
  end M3M;

  model M1E "Same as M1M with external function implementation"
    extends M1M(redeclare package P = ExternalFunctions);
    annotation(experiment(StopTime = 10));
  end M1E;
  
  model M2E "Same as M2M with external function implementation"
    extends M2M(redeclare package P = ExternalFunctions);
    annotation(experiment(StopTime = 10));
  end M2E;

  model M3E "Same as M3M with external function implementation"
    extends M3M(redeclare package P = ExternalFunctions);
    annotation(experiment(StopTime = 10));
  end M3E;
end ExternalFunctionsModelicaUtilities;

/* functions.c */
#include "functions.h"
#include <string.h>
#include <stdio.h>
#include "ModelicaUtilities.h"

int size(const char *name)
{
  if (!strcmp(name, "foobar"))
    return 3;
  else
    ModelicaError("Unsupported name");
}

double compute(const char *name, double x)
{
  double y;
  char str[100];
 
  if (!strcmp(name, "foobar"))
    {
	  y = 1 + x*x;
	  if (y > 10.0)
	  {
	    snprintf(str, 100, "Value of x = %f could be too high", x);
		ModelicaWarning(str);
	  }
	  if (y > 17.0)
	  {
	    snprintf(str, 100, "Value of x = %f is too high", x);
		ModelicaError(str);
	  }
	  return y;
	}
  else
    ModelicaError("Unsupported name");
}

/* functions.h */
#pragma once

int size(const char *name);
double compute(const char *name, double x);

The binaries can be compiled using CMAKE with the following CMakeLists.txt file:

cmake_minimum_required(VERSION 3.5)
set(CMAKE_BUILD_TYPE Release)
project(functions)
add_library(functions SHARED functions.c)

The full MWE source code is provided here. The ModelicaUtilities.h file was copied from the latest version provided by the MSL 4.1.0.

Motivation

The above-described MWE is motivated in particular (but not exclusively!) by the ExternalMedia library. The goal of this library is to provide Modelica.Media compatible medium models, which use external code (e.g. CoolProp) to compute the medium properties. Note that the Modelica.Media library provides a purely functional interface, so one can write the following generic model:

model M
  replaceable package Medium = Modelica.Media.Interfaces.PartialTwoPhaseMedium;
  Medium.Density d = Medium.density_pT(1e5, 300);
end M;

Model M can be instantiated to use either a built-in Modelica.Media medium model, whose functions are defined in by Modelica algorithms, or the ExternalMedia medium models, whose functions are defined as external functions:

model S
  M m1(redeclare package Medium = Modelica.Media.Water.StandardWater);
  M m2(redeclare package Medium = ExternalMedia.Examples.WaterCoolProp);
end M;

Of course, proper error handling via ModelicaError must be provided to correctly cope with property functions called outside their domain of validity during simulation, or to report invalid medium description strings with proper error messages.

Current status

The code shown above is valid, according to the latest MLS draft specification. When running CMAKE under Windows, using Visual Studio 2019, a linker error is issued, complaining that the ModelicaError and ModelicaWarning symbols are unresolved. When running CMAKE under Linux, the binaries are compiled successfully, because Linux defers checking the availablity of those symbols at runtime.

In fact, all the example models in the MWE package run in OpenModelica under Linux. At compile time, OMC uses the libffi library to access the external size() function and exports the ModelicaError and ModelicaWarning symbols, which are then called by the external function, triggering errors that are shown in the GUI. At simulation run time, the simulation executable loads the function.so shared library and exports the ModelicaError and ModelicaWarning symbols, so that the external functions can call them, triggering the appropriate response by the simulation executable. This can point to a possible solution strategy (see below) but currently does not fulfill requirement 5.

Bottom line, AFAIU, it is not possible to fullfill the six stated requirements with the current MLS.

Candidate solutions

At the moment, there are several possible candidate solution:

• One is proposed by myself and @fedetftpolimi in ModelicaUtilities for shared object / dynamic link library #2191 here and here. The idea is to mandate that Modelica tools and simulation runtimes export the ModelicaUtilities symbols. This is enough to the get the MWE as shown working on Linux. On Windows, you need some tricky C code to explicitly handle the importing of the symbols; luckily this code can be written once and for all and became part of Modelica.Utilities, so it does not put too much of a burden to the library developer. This proposal fulfills the six requirements (6. is a bit stretched on Windows) and was tested successfully on OMC.
• One is proposed by @t-sommer in #4476 here. The idea is to define a ModelicaUtilityFunction object in the (EDIT) ModelicaServices library (I guess that's the right place for it), which is implemented by each tool vendor. One can then use this external object to get the pointers to the ModelicaUtilities functions. It is not clear to me at the moment if this proposal fullfills the six requirements listed above, in particular number 2. Also, one potential issue that is pointed out by the MWE is that if you have a purely functional interface, there is no place to instantiate the external object. However, as pointed out in the current draft of Section 12.9.7: "External objects may be a protected component (or part of one) in a function. The constructor is in that case called at the start of the function call, and the destructor when the function returns, or when recovering from errors in the function.". So, one could write a "glue" function in Modelica that internally declares this external object and then passess it to the actual external function. If this works, it would be better than the previous solution from the point of view of requirement 6. I'm not sure it would fulfill requirement 2.
• One is sketched in #4476 by @HansOlsson. It involves a statically linked "glue" C function providing the pointers to ModelicaUtilities functions to a dynamically linked function with the actual external function code. The proposed C-code of the glue external functions looks a bit tricky to me, so requirement 6. is not fully fulfilled. Also, it is not clear to me if requirement 2. can be fulfilled at all.
• Last, but not least, ModelicaUtilities for shared object / dynamic link library #2191 reports the first proposal made by @beutlich on the old trac system. The idea is that tools provide ModelicaExternalC.so/ModelicaExternalC.dll shared libraries which export the ModelicaUtilities symbols, so they can be linked by the external functions. There is some discussion there, I'm not sure if this proposal is actually superseded by the first candidate solution in this list.

Proposed action

To me, the six requirements posted above are more than reasonable and I hope there is agreement on them. If so, we should investigate which of the proposed solutions can fulfill them and then choose the best one.

[casella]

After sleeping over this for one night, I came up with a proposal to solve this issue that actually takes inspiration from all the current proposals listed above.

As @t-sommer suggested in modelica/ModelicaStandardLibrary#4476, we can define in the ModelicaUtilities library the interface of a ModelicaUtilityFunctions external object, which will then be implemented by each tool vendor:

class ModelicaUtilityFunctions
  extends ExternalObject;

  function constructor
    output ModelicaUtilityFunctions functions;
  end constructor;

  function destructor
    input ModelicaUtilityFunctions functions;
  end destructor;

end ModelicaUtilityFunctions;

The constructor returns a pointer to this C struct, that collects all the pointers to callback ModelicaUtilities functions and whose definition should be added to ModelicaUtilities.h

typedef struct {

	void  (*ModelicaMessage)(const char *string);
	void  (*ModelicaFormatMessage)(const char *string, ...);
	void  (*ModelicaVFormatMessage)(const char *string, va_list);
	void  (*ModelicaError)(const char *string);
	void  (*ModelicaFormatError)(const char *string, ...);
	void  (*ModelicaVFormatError)(const char *string, va_list);
	char* (*ModelicaAllocateString)(size_t len);
	char* (*ModelicaAllocateStringWithErrorReturn)(size_t len);
        // ...

} ModelicaUtilityFunctions_t;

The external functions of the MWE package ExternalFunctionsModelicaUtilities shown above could then be implemented like this:

package ExternalFunctions
    extends BaseFunctions;

    redeclare function extends size
    protected
      ModelicaUtilitiesFunctions callbacks;
    algorithm
      N := size_callbacks(name, callbacks);
    end size;

    protected function size_callbacks
      extends BaseFunctions size;
      input ModelicaUtilitiesFunctions callbacks;
      external "C"
      annotation(
        LibraryDirectory="modelica://ExternalFunctionsModelicaUtilities/Resources/Source/build",
        Library="functions",
        IncludeDirectory="modelica://ExternalFunctionsModelicaUtilities/Resources/Source",
        Include="#include \"functions.h\"");
    end size_callbacks;
...

with the code in functions.c looking like this:

/* functions.c */
#include "functions.h"
#include <string.h>
#include <stdio.h>
#include "ModelicaUtilities.h"

int size_callbacks(const char *name, void *callbacks)
{
  if (!strcmp(name, "foobar"))
    return 3;
  else
    ((ModelicaUtilityFunctions_t *)callbacks)->ModelicaError("Unsupported name");
}

This basically provides the "glue" function mechanism proposed by @HansOlsson at a much higher level of abstraction (nice for Modelica!), without involving any statically linked function with arcane C code.

Tool vendors could then implement the ModelicaUtilityFunctions external object as proposed by @fedetftpolimi, i.e, by exporting and importing symbols appropriately. The advantage is that the ugly code required for importing the ModelicaUtilities symbols on Windows would be completely hidden from the library developers, which only see a high-level external object interface, as shown above. Of course tool vendors could choose any other implementation of the external object they deem fit, as long as it returns the required function pointers.

As I understand, this proposal nicely fullfills all the six requirements listed above.

Additionally, and most importantly, this solution does not require any modification to the MLS. It only requires

• adding one more class to ModelicaUtilities
• adding the definition of the ModelicaUtilityFunctions_t struct to ModelicaUtilities.h.

As I understand, both additions are entirely within the compentence of MAP-Lib.

[maltelenz]

To be able to experiment more easily I created https://github.com/maltelenz/ExternalFunctionsModelicaUtilities with the zip file from the first post, as represented by the main branch.

The branch wsm has a bunch of fixes towards better (more valid) Modelica, as well as an improvement making it work on macOS (with System Modeler).

I also adjusted the test models to what I assume was meant? EDIT: Never mind, I misread the package, I reverted this part.

[fedetftpolimi]

I fully agree with all the requirements a solution should have. After reading the summary provided by @casella I have another possible solution to propose, which combines my importer solution with @beutlich's solution.

The problem with my original solution is it relies on some tricky code to import symbols of functions at run-time that should be used by library developer, thus violating requirement 6. The problem with @beutlich's proposal of making a single libModelicaExternalC.dll to be placed in the MSL is that we need to find some tool vendor independent implementation of ModelicaUtilities functions to place in there, otherwise it violates requirement 5.

However, if we place the tricky code to import the symbols of functions from each separate tool at run-time inside a tool-vendor-independent libModelicaExternalC.dll and place it in the MSL, then we can make a truly tool vendor independent solution without requiring any external object.

Proposal

We place in the MSL a single implementation of libModelicaExternalC.dll (and its source code) that implements all the ModelicaUtilities functions by importing a corresponding "implementation function symbol" at run-time from the tool vendor and calling it.

For example, consider ModelicaError:

• libModelicaExternalC.dll provides the ModelicaError function symbol, so all Modelica libraries will just have to link against the libModelicaExternalC.dll in the MSL and be done from the library developer perspective
• libModelicaExternalC.dll internally implements the ModelicaError function by importing a tool-vendor-specific ModelicaErrorImpl function symbol and calling it, therefore hiding the tricky code necessary to import a symbol at run-time on Windows away from both library developers and tool vendors
• all tool vendors only need to export the ModelicaErrorImpl function symbol (the name should be standardized so that it's called with the same name regardless of the tool vendor), both at compile-time and at run-time

Of course there will be a ModelicaWarningImpl function symbol to implement ModelicaWarning and so on... so all ModelicaUtilties functions provided by the tool-vendor-independent libModelicaExternalC.dll call into the tool vendor code by means of impl functions resolved at run-time.

I can already provide a proof-of-concept implementation of such a libModelicaExternalC.dll to be placed in the MSL, based on the importer code me and @mahge wrote. It can be optimized, but that's for later.

ModelicaExternalC.zip

We can also choose if we want to provide a libModelicaExternalC.so and libModelicaExternalC.dylib for Linux an Mac OS, but if we do it will be just for consistency. The Linux and Mac OS linkers have no problem with leaving a symbol undefined and automatically look it up at runtime, thus unlike with Windows, to achieve tool-vendor-independence, tool vendors only need to export the ModelicaUtilities function symbols directly.

Advantages

The beauty of this approach is that from the library developer there is no more magic involved, at all. Not from the Modelica side, nor from the C/C++ side, regardless of the operating system.

An external function such as size in @casella's example can be written in Modelica simply as

redeclare function extends size
      external "C";
    annotation(
      LibraryDirectory="modelica://ExternalFunctionsModelicaUtilities/Resources/Source/build",
      Library="functions",
      IncludeDirectory="modelica://ExternalFunctionsModelicaUtilities/Resources/Source",
      Include="#include \"functions.h\"");
    end size;

without the need for creating a wrapper function and passing any external object.

From the C/C++ code perspective, library developers can just #include "ModelicaUtilities.h" and call the corresponding functions

#include "ModelicaUtilities.h"

int size(const char *name)
{
  if (!strcmp(name, "foobar"))
    return 3;
  else
    ModelicaError("Unsupported name");
}

again, without the need for additional void pointers to be dereferenced into pointers to structs containing pointers to ModelicaUtilities functions.

Then, when making the shared library, library developers just need to link with the tool-vendor-independent libModelicaExternalC.dll present in the MSL.

cmake_minimum_required(VERSION 3.5)
set(CMAKE_BUILD_TYPE Release)
project(functions)
link_directories(${CMAKE_SOURCE_DIR})
add_library(functions SHARED functions.c)
target_link_libraries(functions ModelicaExternalC) # <-- link with libModelicaExternalC.dll

[fedetftpolimi]

Following @maltelenz example I made a repository (https://github.com/fedetftpolimi/ModelicaExternalC) to experiment with the proposal of a libModelicaExternalC for possible inclusion in the MSL.

[maltelenz]

I want to start out with the fact that I'm not an expert at these things (especially not the Windows details), so feel free to disregard my comment if I have misunderstood something :)

Could a variant of the latest proposal by @fedetftpolimi be, that libModelicaExternalC.dll instead introduces MAPLib-owned convenience functions, which "translate" these into the already MLS-specified functions?

To rewrite the example from the proposal above:

For example, consider ModelicaError:

• libModelicaExternalC.dll provides the MAPLibModelicaError function symbol, so all Modelica libraries will just have to link against the libModelicaExternalC.dll in the MSL and use the MAPLibModelicaError function from the library developer perspective
• libModelicaExternalC.dll internally implements the MAPLibModelicaError function by importing the MLS-specified ModelicaError function symbol and calling it, therefore hiding the tricky code necessary to import a symbol at run-time on Windows away from both library developers and tool vendors
• all tool vendors only need to export the ModelicaError function symbol (the name is already standardized so that it's called with the same name regardless of the tool vendor), both at compile-time and at run-time

(I haven't done any actual testing, just reading and thinking.)

[fedetftpolimi]

@maltelenz there is no reason your proposal can't work with the Windows linker, since only the function names change.
Both you and @casella mentioned the "competence of MAP-Lib" but that part I don't fully understand, so just like you, there may be some misunderstanding in the rest of my reply.

The rationale behind my proposal is that it prioritizes not breaking existing "expectations" from library developers. The MSL already includes a ModelicaUtilities.h header file which declares functions whose name are ModelicaError, ModelicaWarning and so forth.
Thus I thought of keeping those names in the "public interface" towards libraries. This does have the disadvantage that tool vendors will need to change the names of the functions they export (if they already do so...), but there are comparatively less tool vendors than library developers and of these tool vendors, only a fraction currently exports symbols for shared libraries to call into so my current proposal is written like that. If this is an issue maybe we can look if there's a way for libModelicaExternalC.dll to lookup a symbol with the same name in other libraries. Maybe with some more linker obscure magic it would be possible, I have never tried.

Your proposal seems to me it prioritizes not making changes in tool vendors (if the motivation is different, let me know maybe I did not understand it) but as a side effect it requires to change the already public ModelicaUtilities.h in the MSL to change the function names. Moreover, if libraries start calling MAPLibModelicaError instead of ModelicaError, then for tools that also want to support static libraries a tool change is still required, as in the case of static libraries there's no forwarding DLL and the tool vendor code for handling static libraries will need to provide a MAPLibModelicaError function instead of a ModelicaError function.

[maltelenz]

@fedetftpolimi My idea was that this was a convenience layer that MAPLib can single-handedly provide in MSL, without any changes needed in the actual language, and ideally no change would be needed by tool vendors either.

But:

only a fraction currently exports symbols for shared libraries to call into

maybe tool vendors need to do something in any case...

it requires to change the already public ModelicaUtilities.h in the MSL to change the function names.

I was somewhat against MSL providing that anyway, because I don't see how it possibly belongs there. If MSL actually provides a layer that does something like your/my proposal, it would make sense to provide a header file (but it should be called something else since this is a reserved name in the MLS).

Moreover, if libraries start calling MAPLibModelicaError instead of ModelicaError, then for tools that also want to support static libraries a tool change is still required, as in the case of static libraries there's no forwarding DLL and the tool vendor code for handling static libraries will need to provide a MAPLibModelicaError function instead of a ModelicaError function.

Could this convenience layer in MSL "resolve" the static case as well (by just calling the ModelicaError in the static case)? Maybe I don't understand enough how these things work.

[fedetftpolimi]

My idea was that this was a convenience layer that MAPLib can single-handedly provide in MSL

From a pragmatic point of view, if it would be easier to get this issue solved by adding a "convenience layer" I am not against it, but it does have a few major drawbacks from a purely software engineering perspective:

• it would deprecate the current interface towards libraries: ModelicaUtlities.h, as we're abandoning hope of making it work with shared libraries
• it would add a new interface composed of a new header file, a new .dll, with new function names different from those in ModelicaUtlities.h, and we should somehow document and "advertise" to library developers that don't necessary follow these bug reports that the tool-vendor-independent interface is no longer ModelicaUtlities.h but MAPLibModelicaUtilities.h or however we want to call it
• still, tool vendors need to export symbols for the "non-MAPLib" ModelicaUtilities functions both at compile-time and at run-time for runtime lookup to succeed. For example, Dymola does not currently export symbols for those functions and as such currently only works with static libraries and no convenience layer can solve this...

My proposal solves a technical problem: "how to make a single tool-vendor-independent DLL that all libraries can link to which in turn calls into vendor-dependent code". It does not solve the political problem of convincing tool vendors to export the symbols required to make shared library work.

Could this convenience layer in MSL "resolve" the static case as well (by just calling the ModelicaError in the static case)? Maybe I don't understand enough how these things work.

A solution is possible, but perhaps not pretty. Or at least I don't see an elegant one. In theory we could add a static library in addition to a dynamic library performing the same mapping, but in practice static libraries are not guaranteed to work with versions of the C compiler other than the one that was used to compile them so we can't just add a static library to the MSL. It would become a maintainability nightmare. We could maybe come up with some header file hack providing inline functions doing the name remapping based on an #ifdef COMPILING_STATIC_LIBRARY but that becomes tricky code that's hard to explain to library developers quickly...

In summary, my opinion is that the hard part will be to convince tool vendors to export symbols. Unfortunately until this is done shared library will never work across tool vendors. Once this hard but unavoidable issue is solved, adding a DLL to implement the functions in ModelicaUtilities.h instead of an convenience layer will occur naturally.

Note that @casella solution has the same problem, in that we need to convince tool vendors to provide the external object with the function pointers.

[HansOlsson]

Note that Modelica models are used in various ways (and there are multiple library developers providing C-libraries for them), and they are deployed in various ways and I'm not sure if all platforms even have dynamic link libraries.

Thus requiring that all models (at least the ones using C-functions) have to use dynamic libraries does not seem like a good idea, and similarly requiring that the simulator exposes dll-export symbol.

We also need to consider the needs of other library developers; as there are a number of other ones - some provide non-dlls, some provide dlls, but many libraries using dlls use externally provided dlls, i.e., without any dependency on anything Modelica-related.

[fedetftpolimi]

I think there's a misunderstanding. This issue is all about adding better support for shared libraries, in particular for calling ModelicaUtilities functions in a portable way.

Doing so does not in any way break support for static libraries, in particular exposing symbols for shared libraries to use does not break static libraries support.

I think my discussion with @maltelenz about renaming functions already converged to the better design decision to keep the same "public API" function names currently present in ModelicaUtilities.h for both static and shared libraries, thus not impacting existing code at all.

There is nothing we lose by improving shared libraries support. Developers who wish to continue to provide static libraries (even though are not guaranteed to work with versions of the C compiler other than the one that was used to compile them) can continue to do so if they wish.

The reason for this issue is that developers who wish to use shared libraries

• because they want their libraries to not have compatibility issues with compilers
• because their libraries in turn depend on other shared libraries
• for whatever design decision that fits their use case

currently are unable to call ModelicaUtilities functions in a tool vendor independent way.

We also need to consider the needs of other library developers;

Yes, and since we can add better support for shared libraries without breaking static libraries, this needs to be fixed.

[fedetftpolimi]

If this is an issue maybe we can look if there's a way for libModelicaExternalC.dll to lookup a symbol with the same name in other libraries. Maybe with some more linker obscure magic it would be possible, I have never tried.

Great news, thanks to the discussion with @maltelenz I had a look if it was possible for libModelicaExternalC.dll to at the same time:

• provide function symbols of ModelicaUtilities.h for all Modelica libraries containing C/C++ code to link to
• do runtime lookup of symbols with the same name to find the tool vendor one and forward to it

and it is possible. All it takes is to get a handle to the current DLL (libModelicaExternalC.dll itself) and exclude symbol lookup there to prevent an infinite recursion.

The updated code is here: https://github.com/fedetftpolimi/ModelicaExternalC

Now we can really expose in the MSL a vendor independent DLL providing the ModelicaUtilities.h function names that calls the same function names in the tool vendor code!

[fedetftpolimi]

Actually, I made some more checks about which tools export ModelicaUtilities function symbols since last time I checked was more than one year ago.

For this I'm using my previous mwe that uses the same importer code as my proposal for extending the MSL with a libModelicaExternalC.dll
Here it is:

mwe.zip

I started with Linux.
OpenModelica exports symbols both at compile time and at run-time.
For Dymola, at compile time I can't get it to evaluate structural parameters that depend on external functions.
At run-time building the model fails for an unrelated reason:

/opt/dymola-2025x-x86_64/bin/dsbuild.sh: Unknown option: -S

apparently the script that is used to build code is not in sync with what options the GUI provides. That makes it impossible to test whether Dymola exports symbols at run-time unless... unless I patch the Dymola build system code.
That's what I did, here's the patch :)

fix-dymola-shared-libraries-support-linux.patch

And lo and behold, Dymola 2025x on Linux already exports symbols for ModelicaUtilities functions! Maybe the next version will have fixed the unrelated build system issue so patching won't even be required.

Then on Windows.
OpenModelica at compile and at runtime works but there are a couple of minor issues that we'll solve by opening tickets to OpenModelica. In any case symbols are correctly exported for shared libraries to use.
For Dymola me and @casella quickly tried to compile it but:

• at compile time just like on Linux we couldn't make it evaluate structural parameters that depend on external functions.
• at run-time we got the error "Some libraries are missing as 32 bit" so we're not sure if Dymola already exports symbols or not on Windows, and we did not have enough time to look into this unrelated issue in detail.

Here's the summary:

Windows OpenModelica CompileTime: Works
Windows OpenModelica RunTime: Works
Windows Dymola CompileTime: Error "Failed to expand the variable x"
Windows Dymola RunTime: Unrelated bug, "some libraries are missing as 32 bit"
Linux OpenModelica CompileTime: Works
Linux OpenModelica RunTime: Works
Linux Dymola CompileTime: Error "Failed to expand the variable x"
Linux Dymola RunTime: Works with patch to dsbuild.sh