Skip to content

Add option to store template instruction in user role and not system role and added granite thinking example#1667

Merged
yoavkatz merged 18 commits intomainfrom
evaluate_different_formats
Jun 10, 2025
Merged

Add option to store template instruction in user role and not system role and added granite thinking example#1667
yoavkatz merged 18 commits intomainfrom
evaluate_different_formats

Conversation

@yoavkatz
Copy link
Member

No description provided.

@yoavkatz yoavkatz marked this pull request as draft March 10, 2025 21:05
@yoavkatz
Copy link
Member Author

yoavkatz commented Apr 6, 2025

Debug completed. Main difference are due to temperature.
Impact of System prompt:

Uploading image.png…

@yoavkatz yoavkatz closed this Apr 6, 2025
@yoavkatz yoavkatz changed the title Test branch to evaluate impact of different format Add option to store template instruction in user role and not system role and valuate impact of different formats Jun 8, 2025
@yoavkatz yoavkatz reopened this Jun 8, 2025
@yoavkatz
Copy link
Member Author

yoavkatz commented Jun 8, 2025

There is a new need to set template instruction in user role and not ion the system role. This in some models (e.g. granite) a user system overrides the default system prompt (E.g. one thast supports thinking).

yoavkatz added 3 commits June 8, 2025 14:59
…s to highlight where instructions are placed.

Signed-off-by: Yoav Katz <katz@il.ibm.com>
Signed-off-by: Yoav Katz <katz@il.ibm.com>
@yoavkatz yoavkatz changed the title Add option to store template instruction in user role and not system role and valuate impact of different formats Add option to store template instruction in user role and not system role and added granite thinking example Jun 8, 2025
yoavkatz added 6 commits June 9, 2025 12:15
Removed 'add_target_prefix' which is not related.

Improved example.

Signed-off-by: Yoav Katz <katz@il.ibm.com>
Signed-off-by: Yoav Katz <katz@il.ibm.com>
Signed-off-by: Yoav Katz <katz@il.ibm.com>
Signed-off-by: Yoav Katz <katz@il.ibm.com>
Signed-off-by: Yoav Katz <katz@il.ibm.com>
@yoavkatz yoavkatz marked this pull request as ready for review June 10, 2025 08:07
yoavkatz and others added 2 commits June 10, 2025 11:08
@yoavkatz yoavkatz merged commit b55532f into main Jun 10, 2025
18 of 19 checks passed
@yoavkatz yoavkatz deleted the evaluate_different_formats branch June 10, 2025 10:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants