This is still a Work In Progress, the below screenshot shows the design I had in mind.
I assumed that the button on the left allows to open an internal issue form (with only summary field) or eventually annotate in case some issue exist with the same summary (this is handled by the backend automatically).
If the number of occurrences > 3, the usual button to open an bug in bugzilla is enabled at the right of the internal issue suggestion.
@Archaeopteryx if you have any suggestion, or idea about this it would help me going on with the implementation.

In the current design, the user has to focus on two things:

• match failure line against an appropriate bug: focus is near the start of the line to the left
• check if a bug needs to be filed (occurrence count, new bug button): focus is to the right
  While the first action should be done first, there is a risk the pin button to the left is used without checking the occurrence count. Suggestion: Keep everything to the left with a line like this

<pin button>i<internal issue count> (<occurrence/classification count><pin icon, not a button, title to explain this as count of already existing classifications>) internal issue summary <button to force filing a bug>

• An attempt to use an internal issue and create its third classification should open the bug filer.
• The structured log needs to be linked in the bug filer UI because the dialog is modal and often more information is needed from the log. Else the users need to close the dialog, open the log, and attempt again to classify it (and retrigger the bugfiler prompt).

I'm actually stuck by an obscure JS error (that seems related to webpack build & react) that happens while submitting the form (when user is logged on treeherder):

Objects are not valid as a React child (found: object with keys {}). If you meant to render a collection of children, use an array instead.

@Archaeopteryx does that mean anything to you?
I still need to patch JS tests failures, then we should be done. I suggest doing the internal/bugzilla issue merge in another PR.

Objects are not valid as a React child (found: object with keys {}). If you meant to render a collection of children, use an array instead.

Could this be the callback which currently adds the bug number returned by the bug filer to the pinboard?

I recorded a video using the UI to create and reference an internal issue: https://github.com/user-attachments/assets/ce94f6b0-b2a8-4fba-a5d9-fc24e27cdd13.
It was not easy to record (due to redis cache, login against after a flush, summary matching and existing bugzilla ID…) but shows quite accurately the workflow. Here are the steps:

1. Log in to Treeherder
2. Initial internal issue creation (i1940976)
3. Flush redis cache and display the annotated internal issue
4. Annotation on another job from the "Pin" button
5. Annotation directly via the interna ID
6. Reusing internal issue with a similar summary (on other job)
7. Flush redis cache, there are 3 occurrences
8. Opening the bug filer via the bug icon
9. Opening the bug filer by referencing internal issue with occurrences >= 3

Next (and last for this feature) work is to link a bugzilla ID to the internal issue once it gets creates, then update its attributes when Bugscache is automatically filled.

Thank you for the video and the description.

Does a third classification with a bug automatically reopen the bug filer?
Step "6. Reusing internal issue with a similar summary (on other job)" shows the attempt to create a new internal issue which reuses the existing one. Correct?

Thank you for the video and the description.

Does a third classification with a bug automatically reopen the bug filer? Step "6. Reusing internal issue with a similar summary (on other job)" shows the attempt to create a new internal issue which reuses the existing one. Correct?

Actually it's the fourth classification that opens the bug filer (which is shown at the end of the video).
This check is based on already fetched suggestions, so it actually only works if the UI know the number of occurrences. I'll patch it so it triggers on the third occurrence. A better approach would be that the backend returns the number of occurrences upon internal issue creation, but in this case the UI would eventually miss the suggestion entry to open the bug filer (or the serializer would also return a suggestion ?).

About step 6, indeed it reuses the existing internal issue, then the UI also create a classification with the selected job.

I rebased this branch, it is fine by me.

I think I reproduced the bug I mentioned on thursday. It only happens with some special summary (e.g. Traceback (most recent call last):), so there should be an explanation somewhere (the internal issue exists but is not listed as a suggestion by the backend).

Getting back to this tomorrow.

Thank you. The changes have been deployed to prototype.

Thank you. The changes have been deployed to prototype.

I played a bit on prototype: https://prototype.treeherder.nonprod.cloudops.mozgcp.net/jobs?repo=autoland&revision=cc5967ffe3435273d130a6b4e992606e1948fe81&selectedTaskRun=aX1fdZkFQyuwljxSLA6e4Q.0
and was able to open a bug at https://bugzilla-dev.allizom.org/show_bug.cgi?id=1854375.

Remaining issues from testing the stack and PR 8595:

• Internal issues do not get suggested for identic failures in tasks which completed after the internal issue had been created. This is due to the caching of the suggested bugs for a failure line. Should the occurrence count be retrieved in a second query on the server and not be cached while the mapping of failure lines to bugs remains cached?
• Flow to force filing a bug if no internal issue exists. Button to file a bug needs to be always shown.
• Pinning with an internal issue which has 2 occurrences should highlight a bug needs to be filed. The positions of the button to classify with the existing internal issue and to file it as a bug could be switched for this.
• Character count for internal issue summary during bug creation needs to be shown similar to when a bug gets filed. Failure lines can be longer than the allowed 255 characters for the field and the user needs to know if they need to shorten the suggested summary and when it is short enough. Example: failures lines for this task

Internal issues do not get suggested for identic failures in tasks which completed after the internal issue had been created. This is due to the caching of the suggested bugs for a failure line. Should the occurrence count be retrieved in a second query on the server and not be cached while the mapping of failure lines to bugs remains cached?

If I understand correctly, cache for suggestions happens at job level in treeherder: https://github.com/mozilla/treeherder/blob/master/treeherder/model/error_summary.py#L125. I see no summary based cache for the suggestions query. To my understanding already opened jobs should not show the internal issue, but a new job with a TextLogError matching the summary should be visible.

I'm implementing the 3 other points.

I implemented the suggestions, except for the last point (number of occurrences is reached via pinning). I'll work on it next week.

I had trouble to log in from my local application (making UI devs more complex to test…). It was working before, now I have the above error at https://auth.mozilla.auth0.com:

I rebased on top of master and finally managed to trigger bug suggestion from a pinned bug, through a custom event (when the suggestion is available).
The issue with logging in to the local instance seems to be fixed! I could not reproduce today.

@aryx can you have a look once again please?