update isa spec docs

Author: xuhz

specification/aie2ps/isa-spec.html

@@ -792,8 +792,8 @@ <h2 id="wait_tcts-0x06">WAIT_TCTS (0x06)</h2>
 the current job will be blocked until all the TCT arrive. The runtime will proceed with
 execution of other (non-blocked) jobs during that time.
 Use this operation only for tasks which have TCTs enabled and where the TCTs are
-routed to arrive at the uC executing this operation.
-Note that tasks can be enqueued from any jobs, but only one job can call <code>WAIT_TCTS</code>. And
+routed to arrive at the uC executing this operation.</p>
+<p>Note that tasks can be enqueued from any jobs, but only one job can call <code>WAIT_TCTS</code>. And
 waiting for more tcts than enqueued will hang the job forever</p>
 <h2 id="end_job-0x07">END_JOB (0x07)</h2>
 <p>Indicates the end of the current job.</p>
@@ -1179,6 +1179,7 @@ <h2 id="remote_barrier-0x12">REMOTE_BARRIER (0x12)</h2>
 remote barrier, only one job in a column can participate in. Example:</p>
 <pre><code>REMOTE_BARRIER      $rb0, 0xC
 </code></pre>
+<p>Note that it is not necessary all prior jobs have to complete before the barrier matures.</p>
 <h2 id="eof-0xff">EOF (0xff)</h2>
 <p>Indicates the end of the operation sequence.</p>
 <table>
@@ -1378,6 +1379,16 @@ <h2 id="preempt-0x19">PREEMPT (0x19)</h2>
 for each preemption point id, the control code of each uc should have this opcode with same <code>id</code>.
 This opcode should take one whole job but the job can share page with other jobs. All following jobs in same page don't start
 before this preemption job is done</p>
+<p>Note:</p>
+<ol>
+<li>
+<p>Just like REMOTE_BARRIER, the barrier sync done by CERT doesn't require completion of prior jobs</p>
+</li>
+<li>
+<p><code>SAVE</code> and <code>RESTORE</code> control code have self-contained stream switch routing for themselves. The routing information
+will be cleared after <code>SAVE</code> and <code>RESTORE</code> are done</p>
+</li>
+</ol>
 <h2 id="load_pdi-0x1a">LOAD_PDI (0x1a)</h2>
 <p>load pdi</p>
 <table>
@@ -1412,6 +1423,18 @@ <h2 id="load_pdi-0x1a">LOAD_PDI (0x1a)</h2>
 This opcode should take one whole job but the job can share page with other jobs. All jobs following the load pdi job
 don't start before the load pdi job is done.
 In multi-uc case, cert will do barrier sync after the load_pdi job, so each uc needs to have same number of load pdi jobs</p>
+<p>According to AIE spec,</p>
+<pre><code>For full reconfiguration of the cores program memory, the following steps may be taken.
+
+Wait for core to reach a known synchronization point (lock, or Core done) 
+
+Reset, disable core (Section 5.12.2) 
+
+Write PM 
+
+Unreset, enable core
+</code></pre>
+<p>Note that if core elf is part of pdi, reset/unreset, enable/disable core required. This should be part of the generated control code</p>
 <h2 id="load_cores-0x04">LOAD_CORES (0x04)</h2>
 <p>load cores</p>
 <table>
@@ -1439,6 +1462,7 @@ <h2 id="load_cores-0x04">LOAD_CORES (0x04)</h2>
 </tbody>
 </table>
 <p>This essentially is same to LOAD_PDI except that CERT will save the elf info for aie cores to different location than the pdi so that cert can do recovery of both during preemption</p>
+<p>Note that same to LOAD_PDI, reset/unreset, enable/disable core is required to load new core elf. This should be part of generated control code</p>
 <h2 id="load_cores_cp-0x20">LOAD_CORES_CP (0x20)</h2>
 <p>load cores_cp</p>
 <table>
@@ -1462,6 +1486,18 @@ <h2 id="load_cores_cp-0x20">LOAD_CORES_CP (0x20)</h2>
 </tbody>
 </table>
 <p>This is to load control packet format core elfs. Compared to the LOAD_CORES, this opcode assumes the control code to load the control packet through shimdma will not take more than a page so that the current page that holds this opcode will not be overwritten. As a result, we can put this control packet loading part right after this opcode and hold everything in one job. The <code>core_elf_id</code> in this opcode and the one in LOAD_CORES are in same id space so that if there are LOAD_CORES and LOAD_CORES_CP, cert can only save the id for the last of these opcodes.</p>
+<p>Notes:</p>
+<ol>
+<li>
+<p>Same to LOAD_PDI, reset/unreset, enable/disable core is required to load new core elf. This should be part of generated control code</p>
+</li>
+<li>
+<p>Completion of control packet blob shimDMA doesn't mean the completion of the control packet loading. Control code itself should be responsible for checking completion of control packet loading. There are 2 ways doing this, one is to enable the write response of last control packet and the other is to add one extra control packet write to a known register. For former way, routing for response packet should be configured, and WAIT_TCTS is used to check the response. For latter way, POLL_32/MASK_POLL_32 is used to check the register of the extra control packet.</p>
+</li>
+<li>
+<p>In broadcast case, completion check should be performed on all destination cores.</p>
+</li>
+</ol>
 <h2 id="load_last_pdi-0x1b">LOAD_LAST_PDI (0x1b)</h2>
 <p>load last loaded pdi</p>
 <table>
@@ -1484,6 +1520,18 @@ <h2 id="load_last_pdi-0x1b">LOAD_LAST_PDI (0x1b)</h2>
 </table>
 <p>Used in preemption restore case. The info (pdi id and location in host ddr) of the pdi last time loaded is saved
 in firmware. During restore (after a context switch), the last loaded pdi will be loaded with this opcode</p>
+<p>Notes:</p>
+<ol>
+<li>
+<p>CERT saves information of one last pdi and one last core elf</p>
+</li>
+<li>
+<p>The last core elf can be either from LOAD_CORES or LOAD_CORES_CP, whichever is the last.</p>
+</li>
+<li>
+<p>In preemption support, LOAD_PDI is mandatory, that is to say, there is at least one LOAD_PDI before preemption point is inserted. LOAD_CORES/LOAD_CORES_CP are optional.  </p>
+</li>
+</ol>
 <h2 id="save_timestamps-0x1c">SAVE_TIMESTAMPS (0x1c)</h2>
 <p>save the timestamps</p>
 <table>
@@ -1534,6 +1582,7 @@ <h2 id="sleep-0x1d">SLEEP (0x1d)</h2>
 </table>
 <p>Sleep instruction makes the current job busy waiting <code>target</code> microseconds. This opcode is for test purpose
 only. Calling this operation doesn't yield control to another job.</p>
+<p>Note that this opcode is not supported in release CERT. Dtrace action sleep() is recommended for this test purpose.</p>
 <h2 id="save_register-0x1e">SAVE_REGISTER (0x1e)</h2>
 <p>Reads value from specified register address and saves it in Shared Data Memory.</p>
 <table>

specification/aie2ps/isa-spec.md

@@ -325,6 +325,7 @@ the current job will be blocked until all the TCT arrive. The runtime will proce
 execution of other (non-blocked) jobs during that time.
 Use this operation only for tasks which have TCTs enabled and where the TCTs are
 routed to arrive at the uC executing this operation.
+
 Note that tasks can be enqueued from any jobs, but only one job can call `WAIT_TCTS`. And
 waiting for more tcts than enqueued will hang the job forever
 
@@ -574,6 +575,8 @@ remote barrier, only one job in a column can participate in. Example:
 REMOTE_BARRIER      $rb0, 0xC
 ```
 
+Note that it is not necessary all prior jobs have to complete before the barrier matures.
+
 
 ## EOF (0xff)
 
@@ -701,6 +704,13 @@ for each preemption point id, the control code of each uc should have this opcod
 This opcode should take one whole job but the job can share page with other jobs. All following jobs in same page don't start
 before this preemption job is done
 
+Note:
+
+1. Just like REMOTE_BARRIER, the barrier sync done by CERT doesn't require completion of prior jobs
+
+2. `SAVE` and `RESTORE` control code have self-contained stream switch routing for themselves. The routing information
+will be cleared after `SAVE` and `RESTORE` are done
+
 
 ## LOAD_PDI (0x1a)
 
@@ -719,6 +729,21 @@ This opcode should take one whole job but the job can share page with other jobs
 don't start before the load pdi job is done.
 In multi-uc case, cert will do barrier sync after the load_pdi job, so each uc needs to have same number of load pdi jobs
 
+According to AIE spec,
+```
+For full reconfiguration of the cores program memory, the following steps may be taken.
+
+Wait for core to reach a known synchronization point (lock, or Core done) 
+
+Reset, disable core (Section 5.12.2) 
+
+Write PM 
+
+Unreset, enable core
+```
+
+Note that if core elf is part of pdi, reset/unreset, enable/disable core required. This should be part of the generated control code
+
 
 ## LOAD_CORES (0x04)
 
@@ -730,6 +755,8 @@ load cores
 
 This essentially is same to LOAD_PDI except that CERT will save the elf info for aie cores to different location than the pdi so that cert can do recovery of both during preemption
 
+Note that same to LOAD_PDI, reset/unreset, enable/disable core is required to load new core elf. This should be part of generated control code
+
 
 ## LOAD_CORES_CP (0x20)
 
@@ -741,6 +768,14 @@ load cores_cp
 
 This is to load control packet format core elfs. Compared to the LOAD_CORES, this opcode assumes the control code to load the control packet through shimdma will not take more than a page so that the current page that holds this opcode will not be overwritten. As a result, we can put this control packet loading part right after this opcode and hold everything in one job. The `core_elf_id` in this opcode and the one in LOAD_CORES are in same id space so that if there are LOAD_CORES and LOAD_CORES_CP, cert can only save the id for the last of these opcodes.
 
+Notes:
+
+1. Same to LOAD_PDI, reset/unreset, enable/disable core is required to load new core elf. This should be part of generated control code
+
+2. Completion of control packet blob shimDMA doesn't mean the completion of the control packet loading. Control code itself should be responsible for checking completion of control packet loading. There are 2 ways doing this, one is to enable the write response of last control packet and the other is to add one extra control packet write to a known register. For former way, routing for response packet should be configured, and WAIT_TCTS is used to check the response. For latter way, POLL_32/MASK_POLL_32 is used to check the register of the extra control packet.
+
+3. In broadcast case, completion check should be performed on all destination cores.
+
 
 ## LOAD_LAST_PDI (0x1b)
 
@@ -753,6 +788,14 @@ load last loaded pdi
 Used in preemption restore case. The info (pdi id and location in host ddr) of the pdi last time loaded is saved
 in firmware. During restore (after a context switch), the last loaded pdi will be loaded with this opcode
 
+Notes:
+
+1. CERT saves information of one last pdi and one last core elf
+
+2. The last core elf can be either from LOAD_CORES or LOAD_CORES_CP, whichever is the last.
+
+3. In preemption support, LOAD_PDI is mandatory, that is to say, there is at least one LOAD_PDI before preemption point is inserted. LOAD_CORES/LOAD_CORES_CP are optional.  
+
 
 ## SAVE_TIMESTAMPS (0x1c)
 
@@ -779,6 +822,8 @@ sleep instruction.
 Sleep instruction makes the current job busy waiting `target` microseconds. This opcode is for test purpose
 only. Calling this operation doesn't yield control to another job.
 
+Note that this opcode is not supported in release CERT. Dtrace action sleep() is recommended for this test purpose.
+
 
 ## SAVE_REGISTER (0x1e)
 

specification/aie2ps/isa-spec.yaml

@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: MIT
-# Copyright (C) 2023-2025 Advanced Micro Devices, Inc. All rights reserved.
+# Copyright (C) 2023-2026 Advanced Micro Devices, Inc. All rights reserved.
 ---
 platform:
   architecture: aie2ps
@@ -225,6 +225,7 @@ operations:
       execution of other (non-blocked) jobs during that time.
       Use this operation only for tasks which have TCTs enabled and where the TCTs are
       routed to arrive at the uC executing this operation.
+
       Note that tasks can be enqueued from any jobs, but only one job can call `WAIT_TCTS`. And
       waiting for more tcts than enqueued will hang the job forever
   - mnemonic: END_JOB
@@ -495,6 +496,8 @@ operations:
       ```
       REMOTE_BARRIER      $rb0, 0xC
       ```
+
+      Note that it is not necessary all prior jobs have to complete before the barrier matures.
   - mnemonic: EOF
     opcode: 0xff
     regular: False
@@ -633,6 +636,13 @@ operations:
       for each preemption point id, the control code of each uc should have this opcode with same `id`.
       This opcode should take one whole job but the job can share page with other jobs. All following jobs in same page don't start
       before this preemption job is done
+
+      Note:
+
+      1. Just like REMOTE_BARRIER, the barrier sync done by CERT doesn't require completion of prior jobs
+
+      2. `SAVE` and `RESTORE` control code have self-contained stream switch routing for themselves. The routing information
+      will be cleared after `SAVE` and `RESTORE` are done
   - mnemonic: LOAD_PDI
     opcode: 0x1a
     regular: True
@@ -657,6 +667,21 @@ operations:
       This opcode should take one whole job but the job can share page with other jobs. All jobs following the load pdi job
       don't start before the load pdi job is done.
       In multi-uc case, cert will do barrier sync after the load_pdi job, so each uc needs to have same number of load pdi jobs
+
+      According to AIE spec,
+      ```
+      For full reconfiguration of the cores program memory, the following steps may be taken.
+
+      Wait for core to reach a known synchronization point (lock, or Core done) 
+
+      Reset, disable core (Section 5.12.2) 
+
+      Write PM 
+
+      Unreset, enable core
+      ```
+
+      Note that if core elf is part of pdi, reset/unreset, enable/disable core is required. This should be part of the generated control code
   - mnemonic: LOAD_CORES
     opcode: 0x04
     regular: True
@@ -674,6 +699,8 @@ operations:
     brief: load cores
     description: |
       This essentially is same to LOAD_PDI except that CERT will save the elf info for aie cores to different location than the pdi so that cert can do recovery of both during preemption
+
+      Note that same to LOAD_PDI, reset/unreset, enable/disable core is required to load new core elf. This should be part of generated control code
   - mnemonic: LOAD_CORES_CP
     opcode: 0x20
     regular: True
@@ -686,6 +713,14 @@ operations:
     brief: load cores_cp
     description: |
       This is to load control packet format core elfs. Compared to the LOAD_CORES, this opcode assumes the control code to load the control packet through shimdma will not take more than a page so that the current page that holds this opcode will not be overwritten. As a result, we can put this control packet loading part right after this opcode and hold everything in one job. The `core_elf_id` in this opcode and the one in LOAD_CORES are in same id space so that if there are LOAD_CORES and LOAD_CORES_CP, cert can only save the id for the last of these opcodes.
+
+      Notes:
+
+      1. Same to LOAD_PDI, reset/unreset, enable/disable core is required to load new core elf. This should be part of generated control code
+
+      2. Completion of control packet blob shimDMA doesn't mean the completion of the control packet loading. Control code itself should be responsible for checking completion of control packet loading. There are 2 ways doing this, one is to enable the write response of last control packet and the other is to add one extra control packet write to a known register. For former way, routing for response packet should be configured, and WAIT_TCTS is used to check the response. For latter way, POLL_32/MASK_POLL_32 is used to check the register of the extra control packet.
+
+      3. In broadcast case, completion check should be performed on all destination cores.
   - mnemonic: LOAD_LAST_PDI
     opcode: 0x1b
     regular: True
@@ -696,6 +731,14 @@ operations:
     description: |
       Used in preemption restore case. The info (pdi id and location in host ddr) of the pdi last time loaded is saved
       in firmware. During restore (after a context switch), the last loaded pdi will be loaded with this opcode
+
+      Notes:
+
+      1. CERT saves information of one last pdi and one last core elf
+
+      2. The last core elf can be either from LOAD_CORES or LOAD_CORES_CP, whichever is the last.
+
+      3. In preemption support, LOAD_PDI is mandatory, that is to say, there is at least one LOAD_PDI before preemption point is inserted. LOAD_CORES/LOAD_CORES_CP are optional.  
   - mnemonic: SAVE_TIMESTAMPS
     opcode: 0x1c
     regular: True
@@ -724,6 +767,8 @@ operations:
     description: |
       Sleep instruction makes the current job busy waiting `target` microseconds. This opcode is for test purpose
       only. Calling this operation doesn't yield control to another job.
+
+      Note that this opcode is not supported in release CERT. Dtrace action sleep() is recommended for this test purpose.
   - mnemonic: SAVE_REGISTER
     opcode: 0x1e
     regular: True