missing files, extra files

[eshellman]

the files table in the database contains >170,000 entries for files that have gone missing. these are marked with filestatus=5. these include the etext/* files as well as 1/2/3 files that have been moved.

there is at least one file marked filestatus=5 because a stat failed, probably due to its large size? https://www.gutenberg.org/files/673/673.txt

[gbnewby]

No, I don't think we need to keep track perpetually. However, it might be useful for diagnostic purposes to find out when a file went missing, such as if it was deleted accidentally.

My suggestion is to flush all the entries.

A more sophisticated suggestion would be to flush all the entries more than x days old. A good value for x might be 180.

[eshellman]

There are also old files with ".debug." in their names which are not used and should be removed in sync with the files db.