passthrough_kernel

Passthrough Kernel:

This IRON design flow example, called "Passthrough Kernel", demonstrates a simple AIE implementation for vectorized memcpy on a vector of integers. In this design, a single AIE core performs the memcpy operation on a vector with a default length 4096. The kernel is configured to work on 1024 element-sized subvectors and is invoked multiple times to complete the full copy. The example consists of two primary design files: passthrough_kernel.py and passThrough.cc, and a testbench test.cpp or test.py.

Source Files Overview

1. passthrough_kernel.py: A Python script that defines the AIE array structural design using MLIR-AIE operations. The file generates MLIR that is then compiled using aiecc to produce design binaries (ie. XCLBIN and inst.bin for the NPU in Ryzen™ AI).

2. passthrough_kernel_placed.py: A Python script that defines the AIE array structural design using an alternatives IRON syntax that yields MLIR-AIE operations. The file generates MLIR that is then compiled using aiecc to produce design binaries (ie. XCLBIN and inst.bin for the NPU in Ryzen™ AI).

3. passThrough.cc: A C++ implementation of vectorized memcpy operations for AIE cores. Found here.

4. test.cpp: This C++ code is a testbench for the Passthrough Kernel design example. The code is responsible for loading the compiled XCLBIN file, configuring the AIE module, providing input data, and executing the AIE design on the NPU. After executing, the script verifies the memcpy results and optionally outputs trace data.

5. test.py: This Python code is a testbench for the Passthrough Kernel design example. The code is responsible for loading the compiled XCLBIN file, configuring the AIE module, providing input data, and executing the AIE design on the NPU. After executing, the script verifies the memcpy results and optionally outputs trace data.

Design Overview

This simple example effectively passes data through a single compute tile in the NPU's AIE array. The design is described as shown in the figure to the right. The overall design flow is as follows:

1. An object FIFO called "of_in" connects a Shim Tile to a Compute Tile, and another called "of_out" connects the Compute Tile back to the Shim Tile.
2. The runtime data movement is expressed to read 4096 uint8_t data from host memory to the compute tile and write the 4096 data back to host memory.
3. The compute tile acquires this input data in "object" sized (1024) blocks from "of_in" and copies them to another output "object" it has acquired from "of_out". Note that a vectorized kernel running on the Compute Tile's AIE core copies the data from the input "object" to the output "object".
4. After the vectorized copy is performed, the Compute Tile releases the "objects", allowing the DMAs (abstracted by the object FIFO) to transfer the data back to host memory and copy additional blocks into the Compute Tile, "of_out" and "of_in" respectively.

It is important to note that the Shim Tile and Compute Tile DMAs move data concurrently, and the Compute Tile's AIE Core also processes data concurrently with the data movement. This is made possible by expressing depth 2 in declaring the ObjectFifo, for example, ObjectFifo(line_ty, name="in", default_depth=2) to denote ping-pong buffers. If default_depth is not declared, the default is 2 in reference to this pattern.

Design Component Details

AIE Array Structural Placed Design

This design performs a memcpy operation on a vector of input data. The AIE design is described in a Python module as follows:

1. Constants & Configuration: The script defines input/output dimensions (N, n), buffer sizes in lineWidthInBytes and lineWidthInInt32s, and tracing support.

2. AIE Device Definition: @device defines the target device. The device_body function contains the AIE array design definition.

3. Kernel Function Declarations: passThroughLine is an external function imported from passThrough.cc.

4. Tile Definitions: ShimTile handles data movement, and ComputeTile2 processes the memcpy operations.

5. Object Fifos: of_in and of_out are defined to facilitate communication between ShimTile and ComputeTile2.

6. Tracing Flow Setup (Optional): A circuit-switched flow is set up for tracing information when enabled.

7. Core Definition: The core_body function loops through sub-vectors of the input data, acquiring elements from of_in, processing using passThroughLine, and outputting the result to of_out.

8. Data Movement Configuration: The aie.runtime_sequence operation configures data movement and synchronization on the ShimTile for input and output buffer management.

9. Tracing Configuration (Optional): Trace control, event groups, and buffer descriptors are set up in the aie.runtime_sequence operation when tracing is enabled.

10. Generate the design: The passthroughKernel() function triggers the code generation process. The final print statement outputs the MLIR representation of the AIE array configuration.

AIE Core Kernel Code

passThrough.cc contains a C++ implementation of vectorized memcpy operation designed for AIE cores. It consists of two main sections:

1. Vectorized Copying: The passThrough_aie() function processes multiple data elements simultaneously, taking advantage of AIE vector datapath capabilities to load, copy and store data elements.

2. C-style Wrapper Functions: passThroughLine() and passThroughTile() are two C-style wrapper functions to call the templated passThrough_aie() vectorized memcpy implementation from the AIE design implemented in passthrough_kernel.py. The passThroughLine() and passThroughTile() functions are compiled for uint8_t, int16_t, or int32_t determined by the value the BIT_WIDTH variable defines.

Usage

Compilation

To compile the design:

make

To compile the placed design:

env use_placed=1 make

C++ Testbench

To complete compiling the C++ testbench and run the design:

make run

Python Testbench

To run the design:

make run_py