Skip to content

Feature/autoprompting evaluation script#82

Open
zhuravlevvik wants to merge 17 commits intostagefrom
feature/autoprompting_evaluation_script
Open

Feature/autoprompting evaluation script#82
zhuravlevvik wants to merge 17 commits intostagefrom
feature/autoprompting_evaluation_script

Conversation

@zhuravlevvik
Copy link
Collaborator

No description provided.

return "f1"
case Task.GENERATION:
return "meteor"
return "bertscore"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Решил, что пора переходить на более современные подходы :)

max_bucket_size=10
)

self.model = ChatOpenAI(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

возможно мы хотим поддержать openrouter через base_url параметр конфига, мб стоит добавить в конце **config или вроде того?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

можно сразу принимать модель (либо создавать дефолтную с конфигом), чтобы не морочиться с тонкой настройкой

Copy link
Collaborator

@Mr-Cool-in-Git Mr-Cool-in-Git left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Оставил комменты + разберись с передачей класса model

}


def code_to_text_preproc(sample, size: int = None):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

протипизируй sample и добавь аргументы в докстринг (во всем скрипте)

common_gen = load_dataset("allenai/common_gen")
ag_news = load_dataset("fancyzhx/ag_news")
xsum = load_dataset("yairfeldman/xsum")
load_dataset("cardiffnlp/tweet_eval", 'emotion')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

почему здесь нет присвоения к переменной?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

кстати почему мы рассматриваем только emotion?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants