RTNNIgen: Code generation from Keras for sequential neural networks to run them directly in the PLC (TwinCAT3)

This toolbox enables the generation of TwinCAT3 Structured Text from a Keras sequential neural network model. It is driven by two components:

• RTNNI: a PLC library allowing real-time capable neural network inference. This avoids redundancies in the code generation step as well as providing a cleaner (more readable) interface
• nnigen: a Python package converting Keras sequential models to PLC code using the RTNNI library.

The following use-cases are supported:

Contents

• RTNNIgen: Code generation from Keras for sequential neural networks to run them directly in the PLC (TwinCAT3)
  • Contents
  • How to install
    • PLC library RTNNI in TwinCAT3
    • nnigen package in Python:
  • How to use
    • Code generation for Keras sequential model in Python
    • Import the generated code in the TwinCAT project
    • Generate a usage example in Python for TwinCAT
    • Update weights only (e.g. after retraining)
  • Reference

How to install

PLC library RTNNI in TwinCAT3

• Open the Library Manager: double-click on the References object in the PLC project tree
  

• Open the Library Repository : click the button in Library Manager

• Install our library RTNNI: click on the Install... button (shown below), then choose RTNNI.library

  

• Add the necessary libraries: click the button add library in Library Manager and add the following dependencies:

  

nnigen package in Python:

In Python3 the package needs to be installed. This can be done via

# install from cloned repository 
git clone https://github.com/iswunistuttgart/rtnnigen.git
cd rtnnigen
pip install .

# or directly from repo:
pip install git+https://github.com/iswunistuttgart/rtnnigen.git@main

How to use

Code generation for Keras sequential model in Python

from nnigen import nnigen
import keras 

# load saved neural network model
model_file = "test_model.keras"
model = keras.saving.load_model(model_file) 
# (this step could be done differently, which depends on your tensorflow version

# tip: directly use a subfolder in the PLC project, nonexistent folders will be created
folder = "ST_files/"
model_name = "Dense_v1"

# generate structured text
nnigen(model, model_name, folder, overwrite_if_model_exists=True)

then several files have been generated in folder ST_files:

File	Contents
{model_name}_LayersWeights.TcDUT	Struct containing all model weights (the variable part of the model)
{model_name}_weights.dat	Binary serialized weights (corresponding to {model_name}_LayersWeights.TcDUT)
{model_name}_Layers.TcDUT	Struct containing the whole network
FB_{model_name}.TcPOU	Function block for model inference (forward pass). Loads the weights on initialization (first >6 calls). This is the only component of the model that needs to be accessed.

For the code example above, the generated set of files would be:

Import the generated code in the TwinCAT project

Add data types DUT ({model_name}_Layers.TcDUT & {model_name}_LayersWeights.TcDUT) and function block POU (FB_{model_name}.TcPOU)

right click on your PLC project -> choose "add" -> choose "existing item..." -> choose files

![add_function](/resources/pictures/add_function.PNG) 

Warning: The path of the model weights is coded into variable filePath of FB_{model_name}.TcPOU. If you move the weights file after its creation make sure to adapt the path in filePath. Otherwise, loading the weights will fail.

Generate a usage example in Python for TwinCAT

from nnigen import get_example_usage

print(get_example_usage(model, model_name))

Example output :

The following code can be used to call the generated model:
    Assuming declared input/output for model:
    
        input : ARRAY[0..0] OF LREAL;
        result : ARRAY[0..0] OF LREAL;

    Then call as:

        FB_Dense_v1(pointer_input:=ADR(input), pointer_output:=ADR(result));  

Update weights only (e.g. after retraining)

In Python

from nnigen import update_model_weigths

# assuming `model` is a Keras sequential model
# which was previously exported and now its weights were retrained


folder = "ST_files/"
model_name = "Dense_v1"

update_model_weigths(model, model_name, folder)

Warning: If the export location of the weights differs from the folder used for the original export, also adapt the variable filePathof FB_{model_name}.TcPOU to let the PLC know the new weights location.

Reference

If you use RTNNIgen in an academic context, please acknowledge this and cite the following article.

Hinze, C., Zhou, Z., Xu, H., Lechler, A., & Verl, A. (2024, November). RTNNIgen–An Open-Source PLC Library and Python Generator Converting Keras Models to Realtime Capable Structured Text. In IECON 2024-50th Annual Conference of the IEEE Industrial Electronics Society (pp. 1-7). IEEE. DOI: 10.1109/IECON55916.2024.10905775