quotes.test.txt

[Sharplinger]

Two questions,

From the looks of it the "quotes.test.txt" used by test_speaker is a just combined version of .quotes / .tokens / .entities in a single file, right?
Is there a script for generating this file?

And is there a script for making all the combined .csv files that are inside each novel folder, (the files that are not part of the pdnc repo).
Doesn't look like they're actually used anywhere but could be really useful to generate these anyways.

(Chances are that I am missing something but seem I cannot find what generated any of these)

Regards, Simon

[gasmichel]

Dear Simon,

Sorry for the late reply. The python script to generate the quotes.test.txt file is located in the folder quotation_attribution/training/data/gen_training_pdnc.py.

You might have to tweak some of the parameters such as absolute paths to PDNC. In terms of other parameters, we used it this way to generate the pdnc-no-minor-w100_full and test-pdnc-no-minor-w100_full folders.

Hope that did not come to late!

Best,
Gaspard

[Sharplinger]

Thank you, I figured it out eventually.

Your test-process uses data from the PDNC so I am working on reproducing the quotes file only using the raw booknlp output.
Most of the data for the qoutes seems to exist in the booknlp output but automatically creating the character list/dictionary will not be comparable to the manual PDNC character list - so I belive raw/nlp data inference results will somewhat differ from the test case.