Add Benchmarks for RRT#67
Conversation
|
|
||
|
|
||
| def solve_many(scene: Scene, rrt: RRT, iterations: int = 10): | ||
| for i in range(7): |
There was a problem hiding this comment.
Should be range(iterations) yeah?
sea-bass
left a comment
There was a problem hiding this comment.
I feel like I want this to land before the continuous joint expanding PR to know how much worse it's going to make the results (if even noticeable)
4f698e4 to
16ae8c3
Compare
| # TODO: Decide on a method for automated benchmark performance testing. | ||
| - name: Run benchmarks | ||
| run: | | ||
| pixi run test_benchmarks |
There was a problem hiding this comment.
I've played around with, but haven't actually connected, two options here for continuous benchmark monitoring (assuming we want this)...
-
https://github.com/benchmark-action/github-action-benchmark, which can store benchmarking results in gh-pages. It'll be a little annoying to configure so if we go this route it's probably easier to just wait until the repo is public. On the plus side we wouldn't need a separate app and it supports more benchmarking suites out of the box. On the down side it's more to configure.
-
https://github.com/CodSpeedHQ/action, third part app to which benchmarking results are uploaded and are viewable on their dashboard. It's pretty easy to setup and use but requires granting access to our repo.
Thoughts? Or do we just want the benchmarks in place and then can come back to this later and rely on checking manually?
There was a problem hiding this comment.
Sample output here.
https://github.com/open-planning/roboplan/actions/runs/18103021233/job/51510609862?pr=67#step:6:27
If this is enough for now then we can call it enough for now...
There was a problem hiding this comment.
I think waiting for the repo to be public is good for me. We can just make this a non-required check for now.
As for 1 or 2, I think whichever is easier tbh
Not really what I was supposed to be doing but at least it would catch if things are broken. Plus I think they would be useful...