Skip to content

Conversation

@a-nogikh
Copy link
Collaborator

@a-nogikh a-nogikh commented Dec 1, 2025

Once a day, generate .jsonl.gz archives of bug infos for the selected namespaces. Refactor the dashboard code to directly reuse the public json API code.

The archives will let us more easily do more a involved syzbot stats analysis.

Cc #5439

@a-nogikh a-nogikh requested a review from tarasmadan December 1, 2025 09:30
@tarasmadan
Copy link
Collaborator

Let's start from the requirements part.
I'm trying to understand https://github.com/google/syzkaller/tree/master/tools/syz-db-export role here.

@a-nogikh
Copy link
Collaborator Author

a-nogikh commented Dec 1, 2025

Ah, didn't know we already do it regularly. Then we can probably only merge the first two commits, or just the second one.

@a-nogikh a-nogikh force-pushed the features/export-bug-archives branch from ad15896 to 75b437f Compare December 1, 2025 10:25
@a-nogikh
Copy link
Collaborator Author

a-nogikh commented Dec 1, 2025

Dropped the third commit (and archived it).

I think the refactoring done in the first commit is still valuable as is - the handleBug function has become way too big as is doing too much.

@a-nogikh a-nogikh changed the title dashboard: export bug archives dashboard: bug json export changes Dec 1, 2025
@tarasmadan
Copy link
Collaborator

Could you please explain how will this additional information be used?
Why archived export is the best way to do the things?

@a-nogikh
Copy link
Collaborator Author

a-nogikh commented Dec 1, 2025

I plan to write a Colab python script that would download the archive, transform into a pandas dataset and do some calculations / draw some graphs for the LPC presentation(s).

If datastore were a more traditional database, some of that could be done by simple SQL queries, but in our case we can only export from Datastore and import somewhere else where we can properly filter/aggregate the data.

Separate the code that loads all details about a particular bug from the
code that is only necessary for rendering a bug page on Web UI.

These can be used in mass bug exports.
Add status, crash, fix, close and commit dates.
@a-nogikh a-nogikh force-pushed the features/export-bug-archives branch from 75b437f to 2eee5ef Compare December 29, 2025 10:35
@a-nogikh
Copy link
Collaborator Author

@tarasmadan PTAL

@a-nogikh a-nogikh requested a review from tarasmadan December 29, 2025 10:41
@a-nogikh a-nogikh added this pull request to the merge queue Dec 29, 2025
Merged via the queue into google:master with commit 4086874 Dec 29, 2025
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants