Julia support for Cantera

[RSuryaNarayan]

Motivation
With Julia being increasingly turned towards by the scientific computing community, Cantera's support to Julia shall be a valuable feature addition. Two such paths exist to extend Cantera's support to Julia:
Possible solutions

• A complete re-implementation based on the Cantera interpreter like Arrhenius.jl
• Some additions to the source code to make the existing version of Cantera "callable" in Julia.
  A possible example of the second method would be to use the PyCall feature of Julia to make the Python implementation of Cantera available in the Julia environment:
  using PyCall; pyimport('cantera')
  Note that before this can take action, one needs PyCall added using Pkg.add("PyCall") and configure the working environment to include python using ENV["PYTHON"] = "location of the python.exe". Some skepticism exists that such an implementation would be unstable. A better implementation of such a feature would be to use the C++ implementation of cantera directly using Julia ccall()and create an executable.

References-Cantera user's group discussions

1. https://groups.google.com/g/cantera-users/c/AkjunMzAkJs
2. https://groups.google.com/g/cantera-users/c/WGi-PTq9Wgg/m/M5LTO-1HAwAJ

[jiweiqi]

I would suggest we explore both of those two approaches.

On one hand, there seem to be some needs for machine learning research that are not easy to realize by extends Canetra. But this needs more careful discussions.

On the other hand, there are very successful examples of building a wrapper for C++ packages in Julia. See Sundials.jl for Sundials. This might be categorized as different solutions to PyCall or cccall, as discussed in #39

[speth]

The Sundials.jl package uses ccall, as you can see in cvode.jl.

For Cantera, we could wrap the Cantera C interface using ccall, or interact directly with the C++ interface using something like Cxx.jl or CxxWrap.jl. Wrapping the C++ interface directly would be more like what we do for the Python module, and would probably make it easier to translate data structures like the AnyMap class across language boundaries.

[jiweiqi]

I just invited Chris to see if he could share some suggestions on the choice of ccall .vs. C++ wrapper.

[ischoegl]

I believe a hybrid option would be to use Cantera's existing framework to handle loading of mechanisms etc. and expose essential reaction parameters to Julia (not too dissimilar of what ReacTorch does in a targeted way). Conversely, it should be possible to write reaction equations in Julia and call them from the C++ framework (which may incur some overhead, but would be quite useful for testing). Another route would be to leverage existing projects, e.g. ReactionMechanismSimulator.jl or others.

[ChrisRackauckas]

The latest results from ReactionMechanismSimulator.jl show some really nice speedups, so for the most part I plan on spending my time there. But for how to wrap, Sundials.jl uses Clang.jl to automatically generate a full wrapper of the sundials C API, which then gets a Julia wrapper into the DifferentialEquations.jl interface, but can also be used directly if one wished. Cxx.jl and CxxWrap.jl get you to a similar place, but IMO are a bit harder to maintain. But with Clang.jl you cannot make good use of templates, while with Cxx.jl you can JIT compile templated calls (hence the C++ REPL it provides). I like working with Clang.jl because then there's a direct Julia-side equivalent to all structs making the memory matching easy (to me), but YMMV.

[jiweiqi]

For a hybrid option, using Cantera's interpreter is a very mature way and it is something that we can decide inside Cantera communities. Both ReacTorch and Arrhenius.jl are trying to work in this style.

One thing that I am not sure about is whether there is already API in Cantera's C/Python API that one can access every kinetic parameter. For instance, for now, ReacTorch has to interpret from the YAML files directly for parts of parameters. But I believe there are already APIs existing but I didn't know back at that time.

For the equations, more specifically, computing reaction source term, I believe we certainly should build a wrapper to do that. As Chris mentioned, we could have some nice speedups with DifferentialEquations.jl. But I guess that speed up is based on better Jacobian computation with auto-differentiation. Nevertheless, this is something we should do within Canetra's community as we did for Matlab to facilitate Julia users.

Regarding the efforts of re-implementing the equations for computing reaction source terms in native Julia code, I am not sure whether it should be a part of Cantera or a standalone package since it does equivalent functionality as of Cantera.

[ischoegl]

One thing that I am not sure about is whether there is already API in Cantera's C/Python API that one can access every kinetic parameter. For instance, for now, ReacTorch has to interpret from the YAML files directly for parts of parameters. But I believe there are already APIs existing but I didn't know back at that time.

Most parameters should be available for Python via the cython interface, and adding ones that may not be exposed at the moment is relatively straightforward. The C API (clib) is less developed, but it may not be needed if Julia accesses C++ directly. On cython/clib I can provide pointers if you know what is missing.

[jiweiqi]

Thanks @ischoegl , I did try to use the wrapper to access that information. I had some difficulties in using ccall since I haven't written c/c++ code for many years. For me, I would like to join the efforts after someone sets up the environment. I mean figure out what package to use, how to set up the environment for development etc.

For the current time being, the development of ReacTorch and Arrhenius.jl are focusing on doing minimal value products to show combustion communities that those PINN and neural ODE stuff are exciting (which is indeed not easy to convince ... based on my experience). By saying exciting, I mean doing things fundamentally new that we can not do with a conventional method. The current mechanism interpreter is sufficient for those algorithms development. But in the long run, I expect that it can rely more on Canetra's interpreter.

[jiweiqi]

Share some of my recent experience on this topic.

Regarding calling Cantera via PyCall: I think this is a pretty convenient approach if the integration doesn't add much overhead, and we don't need auto-differentiation across the solver. For example, in the package of Arrhenius.jl, it is fairly easy to make a zero-D reactor, although might not robust as Cantera was given the limited time period. But for one-dimensional flame, one has to implement the switching between newton iterations and time-stepping, adaptive grid, etc. Then, it is a good idea to call Cantera to do the calculation of one-d flame. Also, solving one-d flame is usually time-consuming, such that the communication is neglectable. Furthermore, for a steady solution of one-D flame, the sensitivity computation can be easily done offline in Julia by reading the results from Cantera.

looking forward, it will be certainly great to have something like Cantera's Matlab toolbox.

[jiweiqi]

For reference, below are some example codes calling Cantera in Julia.
https://github.com/DENG-MIT/Arrhenius_Flame_1D/blob/4e88cd98dc6de9b573ee4754a197d3b29d52345d/src/ct_flame_1d.jl#L5-L37

It is pretty convenient if the goal is not to using Julia's AD eco-system

[Naikless]

Just out of interest: In a package I am writing, I am currently using the approach suggested above where I access cantera‘s python interface via PyCall to obtain thermodynamic and kinetic data or calculate equilibriums. Any ODEs are solved using OrdinaryDiffEq.jl.

This works quite conveniently, even though it adds a little extra organizational effort in managing the package’s dependencies (ensure python is installed, choose a compatible cantera version…)

But what I am more interested in are the implications regarding performance. I call some cantera routines via pycall inside my ODE function calls and wonder how much of a bottleneck this actually is. Does anyone have a feeling if trying to switch to a pure Julia implementation using e.g. Arrhenius.jl or trying to access cantera‘s c++ interface directly would significantly improve performance here?

[Fastaxx]

Hi everyone,
I am a PhD student working on numerical methods for heat and mass transfer, phase change, and reactive transport in Julia.

For my own research, I would like to use Cantera directly from Julia, especially for coupling thermochemistry and kinetics with my own PDE solvers. To explore this, I have started a prototype Julia interface based on the generated CLib bindings, linking directly to libcantera without using PyCall:

Cantera/cantera#2149

I would be very grateful for any feedback on whether this direction makes sense, and how such work could best align with Cantera.
Thank you.

[Fastaxx]

Follow-up to my post above, now that #2149 has had a round of review.

The Julia interface takes the second path from the original issue : direct ccall into libcantera through the generated CLib, no PyCall. It currently covers thermo, kinetics, transport, reactions, Func1, reactors and reactor networks, connectors (walls and flow devices), 1D flames, multiphase equilibrium, reaction path diagrams and SolutionArray: ~320 exported names, 12 test files (267 tests) checked against reference values from the Python interface, and four examples.

A few things changed since the first post, all from @speth's review. The low-level bindings are now scaffolded by sourcegen, as a julia submodule alongside clib and csharp, with by the same ct*.yaml specs. An interfaces/julia/SConscript runs it during the build, so scons build emits the bindings and using Cantera works with no extra step. The ThermoPhase/Kinetics/Transport views now keep a reference to the owning Solution, so it cannot be finalized while a view is still in use.

Documentation follows the second approach @speth suggested: Documenter generates its own pages, scons builds them alongside the Sphinx and Doxygen ones as part of the docs workflow, and they're linked from the Sphinx API reference index, so they deploy with the rest of the documentation.

One remaining question :

Packaging without building from source. Julia ships compiled dependencies as JLL packages: BinaryBuilder cross-compiles the binary from a build_tarballs.jl recipe in Yggdrasil, publishes it as <Name>_jll, and you depend on it normally : using Cantera_jll hands you the library path. That would replace the current library probing, and the jll could ship the data directory too.
No Cantera_jll exists and there's no Cantera recipe in Yggdrasil, so this might be added.

Costs :

• The recipe must drive Cantera's SCons build under BinaryBuilder's cross-compiler for ~10 platforms (it's happiest with autotools/CMake), against Sundials_jll, fmt_jll, yaml_cpp_jll, Eigen_jll and ....
• Yggdrasil is a separate repository, so each Cantera release needs a follow-on PR there, and the jll has to stay locked to the CLib version the bindings were generated against.

My own use is coupling Cantera's thermochemistry and kinetics to our PDE solvers in Julia, so the parts I expect to keep exercising are SolutionArray, the reactor networks and the property getters in hot loops. I'm happy to keep maintaining this. Guidance on this would be the most useful right now.

[bryanwweber]

Yggdrasil is a separate repository, so each Cantera release needs a follow-on PR there, and the jll has to stay locked to the CLib version the bindings were generated against

I don't have a particular concern with this. We already have a separate conda-forge recipe, so there is prior art. I know this is probably very early to ask, and the answer does not change my support for the external recipe, but do you think you'll be able to continue maintaining the package after your degree is done?