Skip to content

Added Fine tuning a VLM for Object Detection Grounding using TRL recipe - #315

Merged
merveenoyan merged 3 commits into
huggingface:mainfrom
sergiopaniego:add-object-detection-grounding
Jul 18, 2025
Merged

Added Fine tuning a VLM for Object Detection Grounding using TRL recipe#315
merveenoyan merged 3 commits into
huggingface:mainfrom
sergiopaniego:add-object-detection-grounding

Conversation

@sergiopaniego

Copy link
Copy Markdown
Member

What does this PR do?

Adds a new recipe 🧑‍🍳 for fine tuning a VLM for object detection + visual grounding using TRL.

Fixes #314

Who can review?

@merveenoyan and @stevhliu

@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@review-notebook-app

review-notebook-app Bot commented Jul 17, 2025

Copy link
Copy Markdown

View / edit / reply to this conversation on ReviewNB

merveenoyan commented on 2025-07-17T15:58:52Z
----------------------------------------------------------------

you could write bbox format for this dataset (is it xywh or xyxy?)


@review-notebook-app

review-notebook-app Bot commented Jul 17, 2025

Copy link
Copy Markdown

View / edit / reply to this conversation on ReviewNB

merveenoyan commented on 2025-07-17T15:58:52Z
----------------------------------------------------------------

not sure of the function name tbh 😅


@review-notebook-app

review-notebook-app Bot commented Jul 17, 2025

Copy link
Copy Markdown

View / edit / reply to this conversation on ReviewNB

merveenoyan commented on 2025-07-17T15:58:53Z
----------------------------------------------------------------

it recognizes two girls, we could just say if we train further it will be better


@merveenoyan merveenoyan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nice! left only three comments, otherwise LGTM :)

@stevhliu stevhliu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome job, really nice to read your notebooks again! 😄

@sergiopaniego

Copy link
Copy Markdown
Member Author

Thanks a lot for the comments! ❤️
I've updated it accordingly and it's now ready to be merged if you consider 😄

@merveenoyan
merveenoyan merged commit 35d27fa into huggingface:main Jul 18, 2025
1 check passed
@sergiopaniego
sergiopaniego deleted the add-object-detection-grounding branch July 18, 2025 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🧑‍🍳 New Fine tuning a VLM for Object Detection Grounding using TRL recipe

4 participants