Simulation does not complete with custom job files (hw/tb/tb_floo_axi_mesh.tb) #128
Description
I've created custom job files for the testbench under hw/tb/tb_floo_axi_mesh.tb that tests a 4x4 NoC mesh with AXI support, and has experienced stalling or hanging behavior . The custom job files is provided within the attachments.**
The custom job files are tested under release v0.6.1 or the latest release of FlooNoC as of 05/19/2025.
Revalidate the Hanging Behavior:
I've followed the initial setup procedures provided under README.md. These are some additional commands that creates the following files necessary for the testing of the 4x4 NoC test bench tb_floo_axi_mesh.sv.
generated/floo_axi_noc.svgenerated/floo_axi_noc_pkg.svgenerated/floo_nw_noc.svgenerated/floo_nw_noc_pkg.svgenerated/floo_single_cluster_noc.svgenerated/floo_single_cluster_noc_pkg.sv
make sources
floogen -c floogen/examples/axi_mesh_src.yml -o ./generated
floogen -c floogen/examples/nw_mesh_src.yml -o ./generated
Additional lines listed below were added to Bender.yml to include the additional files. This is done to include necessary files pertaining to the NoC during the compilation of the ModelSim simulation.
# Level 3
- generated/floo_axi_mesh_noc_pkg.sv
- generated/floo_axi_mesh_noc.sv
- hw/tb/tb_floo_axi_mesh.sv
- generated/floo_nw_mesh_noc_pkg.sv
- generated/floo_nw_mesh_noc.sv
- hw/tb/tb_floo_nw_mesh.sv
Copy of Bender.yml is provided below as Bender.yml.txt
Bender.yml.txt
The custom job files are then transferred into hw/test/jobs.
Execution of custom job files:
Once job files are inserted, with files named hw/test/jobs/mesh_0.txt ... mesh_15.txt, tests are ran with these commands:
make compile-vsim
make run-vsim-batch TB_DUT=tb_floo_axi_mesh JOB_NAME=mesh
Results and Observations:
These are the observed results from the ModelSim simulation:
vsim -c -64 -t 1ps -sv_seed 0 -quiet -work work +JOB_NAME=mesh tb_floo_axi_mesh -do "run -all; quit"
Reading pref.tcl
# 2021.3
# vsim -c -64 -t 1ps -sv_seed 0 -quiet -work work "+JOB_NAME=mesh" tb_floo_axi_mesh -do "run -all; quit"
# Start time: 13:26:47 on May 19,2025
# ** Note: (vsim-3812) Design is being optimized...
# // ModelSim SE-64 2021.3 Jul 13 2021 Linux 4.18.0-553.33.1.el8_10.x86_64
# //
# // Copyright 1991-2021 Mentor Graphics Corporation
# // All Rights Reserved.
# //
# // ModelSim SE-64 and its associated documentation contain trade
# // secrets and commercial or financial information that are the property of
# // Mentor Graphics Corporation and are privileged, confidential,
# // and exempt from disclosure under the Freedom of Information Act,
# // 5 U.S.C. Section 552. Furthermore, this information
# // is prohibited from disclosure under the Trade Secrets Act,
# // 18 U.S.C. Section 1905.
# //
# run -all
# [DMA1] Reading from hw/test/jobs/mesh_0.txt
# [DMA2] Reading from hw/test/jobs/mesh_1.txt
# [DMA3] Reading from hw/test/jobs/mesh_2.txt
# [DMA4] Reading from hw/test/jobs/mesh_3.txt
# [DMA5] Reading from hw/test/jobs/mesh_4.txt
# [DMA6] Reading from hw/test/jobs/mesh_5.txt
# [DMA7] Reading from hw/test/jobs/mesh_6.txt
# [DMA8] Reading from hw/test/jobs/mesh_7.txt
# [DMA9] Reading from hw/test/jobs/mesh_8.txt
# [DMA10] Reading from hw/test/jobs/mesh_9.txt
# [DMA11] Reading from hw/test/jobs/mesh_10.txt
# [DMA12] Reading from hw/test/jobs/mesh_11.txt
# [DMA13] Reading from hw/test/jobs/mesh_12.txt
# [DMA14] Reading from hw/test/jobs/mesh_13.txt
# [DMA15] Reading from hw/test/jobs/mesh_14.txt
# [DMA16] Reading from hw/test/jobs/mesh_15.txt
# [DMA16] Launching 4 jobs copying a total of 1722 B ( 1 kiB - 0 MiB)
# [DMA16] Using default injection ratio of 1.0
# [DMA15] Launching 4 jobs copying a total of 2080 B ( 2 kiB - 0 MiB)
# [DMA15] Using default injection ratio of 1.0
# [DMA14] Launching 4 jobs copying a total of 2256 B ( 2 kiB - 0 MiB)
# [DMA14] Using default injection ratio of 1.0
# [DMA13] Launching 4 jobs copying a total of 1575 B ( 1 kiB - 0 MiB)
# [DMA13] Using default injection ratio of 1.0
# [DMA12] Launching 4 jobs copying a total of 2935 B ( 2 kiB - 0 MiB)
# [DMA12] Using default injection ratio of 1.0
# [DMA11] Launching 4 jobs copying a total of 1495 B ( 1 kiB - 0 MiB)
# [DMA11] Using default injection ratio of 1.0
# [DMA10] Launching 4 jobs copying a total of 2491 B ( 2 kiB - 0 MiB)
# [DMA10] Using default injection ratio of 1.0
# [DMA9] Launching 4 jobs copying a total of 1975 B ( 1 kiB - 0 MiB)
# [DMA9] Using default injection ratio of 1.0
# [DMA8] Launching 4 jobs copying a total of 2349 B ( 2 kiB - 0 MiB)
# [DMA8] Using default injection ratio of 1.0
# [DMA7] Launching 4 jobs copying a total of 2048 B ( 2 kiB - 0 MiB)
# [DMA7] Using default injection ratio of 1.0
# [DMA6] Launching 4 jobs copying a total of 1693 B ( 1 kiB - 0 MiB)
# [DMA6] Using default injection ratio of 1.0
# [DMA5] Launching 4 jobs copying a total of 2506 B ( 2 kiB - 0 MiB)
# [DMA5] Using default injection ratio of 1.0
# [DMA4] Launching 4 jobs copying a total of 1984 B ( 1 kiB - 0 MiB)
# [DMA4] Using default injection ratio of 1.0
# [DMA3] Launching 4 jobs copying a total of 1623 B ( 1 kiB - 0 MiB)
# [DMA3] Using default injection ratio of 1.0
# [DMA2] Launching 3 jobs copying a total of 1308 B ( 1 kiB - 0 MiB)
# [DMA2] Using default injection ratio of 1.0
# [DMA1] Launching 4 jobs copying a total of 2041 B ( 1 kiB - 0 MiB)
# [DMA1] Using default injection ratio of 1.0
# [DMA2] Launched all Transfers.
# [DMA11] Launched all Transfers.
# [DMA9] Launched all Transfers.
# [DMA14] Launched all Transfers.
# [DMA7] Launched all Transfers.
# [DMA4] Launched all Transfers.
# [DMA3] Launched all Transfers.
# [DMA15] Launched all Transfers.
# [DMA8] Launched all Transfers.
# [Monitor dma_2_2][Read] Latency: 632.46 +- 215.10, BW: 10.78 Bits/cycle, Util: 16.85%
# [Monitor dma_2_2][Write] Latency: 73.08 +- 44.59, BW: 10.83 Bits/cycle, Util: 16.92%
# [DMA16] Launched all Transfers.
# [DMA1] Launched all Transfers.
# [DMA10] Launched all Transfers.
# [DMA13] Launched all Transfers.
# [DMA5] Launched all Transfers.
# [Monitor dma_0_1][Read] Latency: 880.20 +- 506.36, BW: 6.98 Bits/cycle, Util: 10.91%
# [Monitor dma_0_1][Write] Latency: 180.54 +- 166.44, BW: 6.50 Bits/cycle, Util: 10.16%
# [Monitor dma_0_2][Read] Latency: 580.18 +- 261.00, BW: 7.12 Bits/cycle, Util: 11.13%
# [Monitor dma_0_2][Write] Latency: 76.95 +- 35.74, BW: 4.46 Bits/cycle, Util: 6.96%
# [DMA6] Launched all Transfers.
# [Monitor dma_1_2][Read] Latency: 537.80 +- 181.26, BW: 8.47 Bits/cycle, Util: 13.23%
# [Monitor dma_1_2][Write] Latency: 282.36 +- 121.87, BW: 9.72 Bits/cycle, Util: 15.19%
# [Monitor dma_3_2][Read] Latency: 307.05 +- 122.11, BW: 12.22 Bits/cycle, Util: 19.10%
# [Monitor dma_3_2][Write] Latency: 113.37 +- 110.82, BW: 8.02 Bits/cycle, Util: 12.54%
# [DMA12] Launched all Transfers.
# [Monitor dma_2_1][Read] Latency: 1208.38 +- 634.90, BW: 7.47 Bits/cycle, Util: 11.67%
# [Monitor dma_2_1][Write] Latency: 326.48 +- 285.57, BW: 5.71 Bits/cycle, Util: 8.92%
# [Monitor dma_0_0][Read] Latency: 709.72 +- 343.76, BW: 7.61 Bits/cycle, Util: 11.90%
# [Monitor dma_0_0][Write] Latency: 310.19 +- 165.50, BW: 6.69 Bits/cycle, Util: 10.45%
# [Monitor dma_3_0][Read] Latency: 881.88 +- 461.33, BW: 5.26 Bits/cycle, Util: 8.21%
# [Monitor dma_3_0][Write] Latency: 326.88 +- 240.91, BW: 4.40 Bits/cycle, Util: 6.88%
# [Monitor dma_2_0][Read] Latency: 973.71 +- 501.01, BW: 6.42 Bits/cycle, Util: 10.03%
# [Monitor dma_2_0][Write] Latency: 181.24 +- 110.33, BW: 6.16 Bits/cycle, Util: 9.63%
^C# Break key hit
# Break at an unknown location
# Stopped at tb_floo_axi_mesh.sv line 39
# quit
# End time: 14:17:41 on May 19,2025, Elapsed time: 0:50:54
# Errors: 0, Warnings: 0, Suppressed Warnings: 40
Within the trace logs from ModelSim, it's clear that the simulation is stalling or hanging at an elapsed time of 0:50:54.I've compared the simulation time of uniform traffic type (Job files for uniform traffic was created using the command: make jobs TRAFFIC_TYPE=uniform), which was around 0:01:20. This indicates that the traffic created by the custom job files exhibited a 50x increase in elapsed time compared to uniform traffic.
I would love your feedback as to the cause of this simulation stall or hang. The custom job files are atached to this issue below.
Thank you