Skip to content

Integrate embench-iot as benchmark suite#724

Open
yc199911 wants to merge 1 commit intosysprog21:masterfrom
yc199911:master
Open

Integrate embench-iot as benchmark suite#724
yc199911 wants to merge 1 commit intosysprog21:masterfrom
yc199911:master

Conversation

@yc199911
Copy link
Copy Markdown

@yc199911 yc199911 commented Mar 19, 2026

embench-iot provides 19 benchmarks designed for embedded systems, offering more representative coverage than the existing dhrystone. This complements the current benchmark infrastructure and enables more thorough performance evaluation of rv32emu.

  • Add embench-iot as a git submodule under tests/embench-iot
  • Add rv32emu board support (boardsupport.c/h) in the fork
  • Add wrapper Makefile to bridge rv32emu build system with scons
  • Register embench-iot in TEST_SUITES in mk/artifact.mk
  • Add scons to CI install dependencies
  • Update CI to detect changes under tests/embench-iot

Note: This integration requires sysprog21 to create an official fork of embench-iot (similar to sysprog21/ansibench) and update the submodule URL accordingly before merging.

Close #28


Summary by cubic

Integrates embench-iot as a new benchmark suite for rv32emu, adding 19 embedded-focused tests to improve coverage beyond Dhrystone. Hooks it into the build, CI, and artifact pipeline for repeatable runs.

  • New Features

    • Add embench-iot as a git submodule under tests/embench-iot.
    • Add rv32emu board support and a wrapper Makefile to build via scons.
    • Register the suite in TEST_SUITES in mk/artifact.mk.
    • CI: install scons and detect changes under tests/embench-iot.
  • Migration

    • Create an official sysprog21/embench-iot fork and update the submodule URL before merging.

Written for commit 6f1ea41. Summary will update on new commits.

embench-iot provides 19 benchmarks designed for embedded systems,
offering more representative coverage than the existing dhrystone.
This complements the current benchmark infrastructure and enables
more thorough performance evaluation of rv32emu.

- Add embench-iot as a git submodule under tests/embench-iot
- Add rv32emu board support (boardsupport.c/h) in the fork
- Add wrapper Makefile to bridge rv32emu build system with scons
- Register embench-iot in TEST_SUITES in mk/artifact.mk
- Add scons to CI install dependencies
- Update CI to detect changes under tests/embench-iot

Note: This integration requires sysprog21 to create an official
fork of embench-iot (similar to sysprog21/ansibench) and update
the submodule URL accordingly before merging.

Close sysprog21#28
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files


Since this is your first cubic review, here's how it works:

  • cubic automatically reviews your code and comments on bugs and improvements
  • Teach cubic by replying to its comments. cubic learns from your replies and gets better over time
  • Add one-off context when rerunning by tagging @cubic-dev-ai with guidance or docs links (including llms.txt)
  • Ask questions if you need clarification on any suggestion

@jserv
Copy link
Copy Markdown
Contributor

jserv commented Mar 26, 2026

Note: This integration requires sysprog21 to create an official fork of embench-iot (similar to sysprog21/ansibench) and update the submodule URL accordingly before merging.

This approach should be avoided initially. The only valid reason to fork an existing benchmark suite is if it has not been maintained for some time. If you encounter build or runtime errors in embench-iot, please submit a pull request upstream first.

@jserv jserv requested review from vacantron March 26, 2026 02:03
@yc199911
Copy link
Copy Markdown
Author

yc199911 commented Mar 26, 2026

Submitted two PRs to upstream embench-iot to avoid maintaining a separate fork:

  1. fix(wikisort): replace custom bool typedef with stdbool.h
    fix(wikisort): replace custom bool typedef with stdbool.h embench/embench-iot#215
  2. Add rv32emu board support
    Add rv32emu board support embench/embench-iot#216

While testing embench-iot with a GCC 15 toolchain (which defaults to C23),
I encountered a build failure in wikisort caused by typedef uint8_t bool
since bool is now a language keyword in C23.
embench-iot#215 addresses
this by replacing the custom bool definitions with #include <stdbool.h>.

Once these are merged upstream, I will update this PR to point the
submodule directly at the upstream repo, removing the need for a fork.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Integrate embench-iot for benchmarking

2 participants