Skip to content

[code_review] Switch to GPT-4.1 #4966

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

marco-c
Copy link
Collaborator

@marco-c marco-c commented Apr 17, 2025

Fixes #4965

@suhaibmujahid could you run the evaluation?

@suhaibmujahid
Copy link
Member

could you run the evaluation?

Here is the report:

--------------------
Variant Name: llm-gpt-4.1
--------------------
New Comments: 316
New Valid Comments: 42
New Invalid Comments: 84
New Unevaluated Comments: 190
--------------------
Old Comments: 272
Old Valid Comments: 83
Old Invalid Comments: 189
--------------------
Recalled comments: 51.10294117647059
Recalled valid comments: 50.602409638554214
Recalled invalid comments: 51.32275132275132
--------------------
Missed valid comments: 49.39759036144578
Missed invalid comments: 48.67724867724868

This can be compared to #4827 (comment):

--------------------
Variant Name: New Filtering
--------------------
New Comments: 520
New Valid Comments: 57
New Invalid Comments: 115
New Unevaluated Comments: 348
--------------------
Old Comments: 272
Old Valid Comments: 85
Old Invalid Comments: 187
--------------------
Recalled comments: 68.38235294117648
Recalled valid comments: 67.05882352941175
Recalled invalid comments: 68.98395721925134
--------------------
Missed valid comments: 32.94117647058823
Missed invalid comments: 31.016042780748666

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

[code_review] Switch to GPT-4.1
2 participants