Skip to content

Add scratchpad run-time parameter functionality#3061

Draft
andrej wants to merge 7 commits into
Xilinx:mainfrom
andrej:params
Draft

Add scratchpad run-time parameter functionality#3061
andrej wants to merge 7 commits into
Xilinx:mainfrom
andrej:params

Conversation

@andrej
Copy link
Copy Markdown
Collaborator

@andrej andrej commented May 8, 2026

Builds on the XRT/firmware scratchpad feature (#3056) to enable more efficient run-time parameters. The XRT scratchpad supports up to 32 parameters in a "StateTable" that can be written to directly from DDR on the host, copied over by the firmware into a local memory, and from there used to write to registers in the NPU partition.

The primary advantage/difference over the existing run-time parameter implementation is that these parameters are truly read from DDR. They are not hard-coded as write32s in the instruction sequence, and therefore do not require instruction sequence patching or different instruction sequences for different parameter values.

This PR abstracts the XRT scratchpad mechanism to make it more user-friendly through an aiex.parameter declaration (at the module level since parameters can be shared across devices), a aiex.sync_parameters_from_host operation that performs the copy from DDR -> command processor memory -> NPU registers/buffers, and a aiex.read_parameter operation that allows reading the parameters from the core. For the host side, a convenience class ScratchpadParameter provides an abstraction to write the parameters to the correct offset in DDR (no manual tracking of offsets, instead we can use named parameters as declared in the MLIR using aiex.parameter).

The mapping of parameter name to DDR offset/StateTable index (where the host C++ writes to and runtime sequence reads from) is exported as additional metadata to be consumed by the runtime libs in a params.txt file. The mapping of parameter name to buffers on NPU cores is kept inline in the MLIR.

    test.cpp (host code) 
          |
          |    ScratchpadParameter abstraction in test_utils
          |    ScratchpadParameter.write("foo", value)
          |
 parameter in DDR 
          |
          |    aiex.sync_parameters_from_host in runtime sequence
          |
  parameter in scratchpad
      /  |  \
    /    |    \      parameters written to NPU cores, one buffer per parameter per core
  /      |      \     (this is also done by aiex.sync_parameters_from_host)
/        |        \
                 aiex.read_parameter("foo")

C++ usage example: NPU-side (MLIR), host-side (C++)

Python usage example: NPU-side (IRON Python), host-side (Python)

There is also another mode to use this where rather than use it as run-time parameters on NPU cores, you can use it to patch DMA BD addresses, C++ example, Python example.

Note: Depends on XRT PR #9813 / XRT PR #9814 for the Python bindings

@andrej andrej changed the title Params Add scratchpad run-time parameter functionality May 8, 2026
@andrej andrej force-pushed the params branch 5 times, most recently from 3c87d83 to dcc57c1 Compare May 14, 2026 15:46
@andrej andrej force-pushed the params branch 2 times, most recently from 1f55992 to ac14b99 Compare May 18, 2026 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant