Skip to content

hud-evals/verilog-template

Repository files navigation

Verilog Template

This template expects the open source Verilog tools to be installed on your machine and available on PATH.

Reference Tasks

Design track: tasks/stream_arb_fifo_repair. This is a repair/debug task where the agent fixes a two-lane stream FIFO RTL implementation, with grading from hidden Verilator simulation, Yosys synthesis and latch detection, Verilator lint, and a weighted functional/synthesis/lint rubric.

Verification track: tasks/stream_arb_fifo_cocotb_dv. The DUT wrapper uses a vendored BaseJump STL FIFO as its storage primitive, while the agent edits a CocoTB Python testbench that must pass on the golden DUT, catch hidden mutant DUTs, hit coverage targets, and satisfy the weighted hygiene rubric.

Formal track: tasks/stream_arb_fifo_formal. This task reuses the same BaseJump-backed FIFO wrapper pattern, but the agent edits Yosys-supported SystemVerilog formal checks. The grader runs bounded SymbiYosys/Yosys proofs on the golden DUT, hidden mutant DUTs, and cover scenarios, then applies a weighted golden-pass/mutant-kill/cover/hygiene rubric.

The deployed image includes the Python DV stack: cocotb, cocotb-bus, cocotb-coverage, pyuvm, and hypothesis. Formal tasks require sby, Yosys, and an SMT solver such as Z3.

The BaseJump-backed tasks vendor the minimum required files under their local vendor/basejump_stl directories. Runtime tasks do not fetch third-party HDL libraries from the internet.

Third-party HDL notices are listed in THIRD_PARTY_NOTICES.md. The vendored BaseJump STL subsets are licensed under the Solderpad Hardware License, Version 0.51, with the upstream license preserved in each task's vendor/basejump_stl/LICENSE.

Run the Design-track grader checks:

cd tasks/stream_arb_fifo_repair
python3 scripts/check_calibration.py

Expected scores:

buggy_baseline: reward=0.000
golden_reference: reward=1.000
functional_fixed_latch_remaining: reward=0.700

Run the CocoTB DV grader checks:

cd tasks/stream_arb_fifo_cocotb_dv
uv run python scripts/check_calibration.py

Expected scores:

starter_testbench: reward=0.250000
reference_solution: reward=1.000000

Run the Formal-track grader checks:

cd tasks/stream_arb_fifo_formal
python3 scripts/check_calibration.py

Expected scores:

starter_properties: reward=0.350000
reference_solution: reward=1.000000

Run the hidden grader against the current agent-facing RTL:

python3 donotaccess/grade.py --root . --pretty

In a deployed HUD environment, donotaccess/ should be copied to a protected root-owned path and hidden from the agent. It remains checked in here so vendors can inspect the reference grader structure.

Build the example container layout:

docker build -t verilog-template:stream-fifo .

The container stores hidden grader files and task baselines under root-owned /donotaccess/<task_id>. At scenario setup time, only the selected task's agent-facing files are copied into /workdir/<task_id>. Human-facing README files and local calibration scripts are not copied into the agent worktree; task requirements are delivered through the HUD prompt. The native HUD v5 environment exposes only bash and edit as agent tools.

Local HUD wrapper checks:

uv venv --python 3.12
uv pip install -e /Users/rose/dev/hud/hud-python -e .
VERILOG_TEMPLATE_LOCAL_TOOLS=1 uv run python env.py

The tool list should be exactly:

['bash', 'edit']

Local Toolchain Setup

Use OSS CAD Suite for local macOS or Linux development. It includes the tools normally needed for Verilog/SystemVerilog labs and FPGA workflows, including:

  • verilator
  • iverilog
  • gtkwave
  • yosys
  • nextpnr
  • iceprog / Project IceStorm tools

Download an archive matching your OS from the latest releases page:

https://github.com/YosysHQ/oss-cad-suite-build/releases/latest

Choose the matching archive for your machine and extract it to a location of your choice. The examples below assume it was extracted to $HOME/oss-cad-suite:

tar -xzf oss-cad-suite-*.tgz -C "$HOME"

On macOS, if execution is blocked by quarantine, clear the downloaded archive's quarantine attribute before extraction, or run ./activate once inside the extracted oss-cad-suite directory:

xattr -d com.apple.quarantine oss-cad-suite-*.tgz
cd "$HOME/oss-cad-suite" && ./activate

For Linux and macOS, either add the bin directory to your shell startup file or source the provided environment script. On macOS the default shell is usually zsh:

echo 'export PATH="$HOME/oss-cad-suite/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc

On Linux, use the startup file for your shell. For a typical Bash setup:

echo 'export PATH="$HOME/oss-cad-suite/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

Alternatively, activate the full OSS CAD Suite environment for the current shell:

source "$HOME/oss-cad-suite/environment"

If you extracted it somewhere other than $HOME/oss-cad-suite, use that path in the export PATH=... line.

Verify the install:

verilator --version
iverilog -V
gtkwave --version
yosys -V

Python Dependencies

Keep Python dependencies in pyproject.toml if this template adds Python tests or helper scripts later. Do not use pyproject.toml as the main way to install Verilator: the Python package exposes verilator-cli, while standard Verilog build scripts usually expect the executable to be named verilator.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors