SCIDOCS coread coview dataset questions

[psw0021]

Hello,
I have some questions regarding SCIDOCS subset data from scirepeval benchmark.

First, it seems that when I have tried to access several queries and candidates' abstract and title data using huggingface scidocs_view_cite_read data from scidocs_view and scidocs_read, some of them don't seem to exist in scidocs_view_cite_read_data, which contains document id, its title and corresponding abstracts. Since paper details do not exist in scidocs_view and scidocs_read split, I tried to access document details using the document id in scidocs_view and scidocs_read split and match them with ones in scidocs_view_cite_read_data. I was curious if I am using the dataset in a right way, or whether the dataset was originally constructed in such way. It seems that scidocs_cite and scidocs_cocite have perfect matches, and scidocs_view, scidocs_read are the splits that seem to be problematic.

Secondly, it seems that several candidates do not contain abstract and only have the titles. I am trying to make sure that such setting is okay.

Sincerely
Thank You.

[amanpreet692]

Hello, thanks for using our benchmark!
Yes, it is possible that some of the entries have only the title and not the abstract and such a setting is ok since we train the model on such data as well.
As for the missing ids, you seem to be using the dataset correctly with doc id from scirepeval_test and the paper data in scirepeval. If some of the ids are missing, I'd suggest to skip those as it's possible they got in by mistake.