Crash when opening a specific project

[Biep]

Manuskript 0.16.1 on Ubuntu 24.10

One project (Markdown compile <1.5 MB, each file under 1500 words, <350 pages ePub on my e-reader) suddenly crashes Manuskript when I try to open it. When doing this starting from the command line I get this:

biep@Puk-1:~$ manuskript
QSocketNotifier: Can only be used with threads started with QThread
CRITICAL> An unhandled exception has occurred!
Traceback (most recent call last):
  File "/usr/share/manuskript/manuskript/ui/welcome.py", line 148, in loadRecentFile
    self.mw.loadProject(act.data())
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
  File "/usr/share/manuskript/manuskript/mainWindow.py", line 936, in loadProject
    self.loadDatas(project)
    ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/share/manuskript/manuskript/mainWindow.py", line 1235, in loadDatas
    errors = loadSave.loadProject(project)
  File "/usr/share/manuskript/manuskript/loadSave.py", line 68, in loadProject
    return v1.loadProject(project, zip=isZip)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/manuskript/manuskript/load_save/version_1.py", line 919, in loadProject
    root = ET.fromstring(files["revisions.xml"])
  File "src/lxml/etree.pyx", line 3307, in lxml.etree.fromstring
  File "src/lxml/parser.pxi", line 1995, in lxml.etree._parseMemoryDocument
  File "src/lxml/parser.pxi", line 1882, in lxml.etree._parseDoc
  File "src/lxml/parser.pxi", line 1164, in lxml.etree._BaseParser._parseDoc
  File "src/lxml/parser.pxi", line 633, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 743, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 672, in lxml.etree._raiseParseError
  File "<string>", line 3965
lxml.etree.XMLSyntaxError: internal error: Huge input lookup, line 3965, column 1830
biep@Puk-1:~$ 

Typically I leave the project open, and when Manuskript starts it shows it immediately, but once in a while I close it - which I did earlier today, after writing a bit of text and doing nothing else.
I can still open other projects, but this is my main one, on which I had hoped to work a lot these days.

[Biep]

I see that the Revisions file is 10.1 MB - did that trigger some limit? I can't find anything else that could cause a "huge input". The whole project folder is 12.4 MB.

[peter88213]

There is actually a size limit for XML files in lxml that is around 10 MB. It looks like you are frequently saving revisions.
For now, you can try moving the revisions.xml file somewhere else so that the application does not try to load this file, which has apparently become too large.
Then try to find a setting for smart deletion of revisions so that not too much accumulates in the future.

I hope this helps.

[TheJackiMonster]

It is also recommended not to use the revision feature until it is reworked. Luckily the idea was to replace current revisions with some versioning system like GIT or SVN anyway internally. So that should not run into some arbitrary limit like this.

[Biep]

Even with revisions off it crashes. Strangely enough, merely copying everything into a new project (throwing out the contents of that project) solved it.

I discovered the logs. Here is the ending of one - it just finishes loading everything, but finds "Strange things" everywhere.
I changed some file and folder names.

2025-09-11 16:54:50,005 - manuskript.load_save.version_1 - DEBUG - * Adds [Last folder] to Root (folder)
2025-09-11 16:54:50,006 - manuskript.load_save.version_1 - DEBUG - Strange things in file [Last folder]
2025-09-11 16:54:50,006 - manuskript.load_save.version_1 - DEBUG - * Adds 21-Appendix to Root (folder)
2025-09-11 16:54:50,006 - manuskript.load_save.version_1 - DEBUG - * Adds 0-Appendix1.md to Appendix (file)
2025-09-11 16:54:50,009 - manuskript.load_save.version_1 - DEBUG - * Adds 1-Appendix2.md to Appendix (file)
2025-09-11 16:54:50,011 - manuskript.load_save.version_1 - DEBUG - * Adds 2-Appendix3.md to Appendix (file)
2025-09-11 16:54:50,013 - manuskript.load_save.version_1 - DEBUG - Strange things in file 21-Appendix
2025-09-11 16:54:50,013 - manuskript.load_save.version_1 - DEBUG - * Adds 22-Klad to Root (folder)
2025-09-11 16:54:50,014 - manuskript.load_save.version_1 - DEBUG - * Adds 0---.md to Klad (file)
2025-09-11 16:54:50,016 - manuskript.load_save.version_1 - DEBUG - * Adds 1-Fragments.md to Klad (file)
2025-09-11 16:54:50,018 - manuskript.load_save.version_1 - DEBUG - * Adds 2-Data.md to Klad (file)
2025-09-11 16:54:50,021 - manuskript.load_save.version_1 - DEBUG - Strange things in file 22-Klad
2025-09-11 16:54:50,365 - manuskript.logging - CRITICAL - An unhandled exception has occurred!
Traceback (most recent call last):
  File "/usr/share/manuskript/manuskript/ui/welcome.py", line 148, in loadRecentFile
    self.mw.loadProject(act.data())
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
  File "/usr/share/manuskript/manuskript/mainWindow.py", line 936, in loadProject
    self.loadDatas(project)
    ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/share/manuskript/manuskript/mainWindow.py", line 1235, in loadDatas
    errors = loadSave.loadProject(project)
  File "/usr/share/manuskript/manuskript/loadSave.py", line 68, in loadProject
    return v1.loadProject(project, zip=isZip)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/manuskript/manuskript/load_save/version_1.py", line 919, in loadProject
    root = ET.fromstring(files["revisions.xml"])
  File "src/lxml/etree.pyx", line 3307, in lxml.etree.fromstring
  File "src/lxml/parser.pxi", line 1995, in lxml.etree._parseMemoryDocument
  File "src/lxml/parser.pxi", line 1882, in lxml.etree._parseDoc
  File "src/lxml/parser.pxi", line 1164, in lxml.etree._BaseParser._parseDoc
  File "src/lxml/parser.pxi", line 633, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 743, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 672, in lxml.etree._raiseParseError
  File "<string>", line 3965
lxml.etree.XMLSyntaxError: internal error: Huge input lookup, line 3965, column 1830

What does Strange things mean? In the new, not-crashing, project, it also finds them in every folder.

[peter88213]

The error log shows that the application still tried to read the "too big" revisions.xml.
As I pointed out, you'd better remove this file. Just switching revisions off is not enough, as it looks.
In this case, you wouldn't have to copy things into another project, which seems to have messed up something.
So you might want to restore your original project, and just remove revisions.xml. Then turn revisions off in the settings.
Does this work?

[peter88213]

I just had a look at the code. The debug message Strange things in file ... seems to indicate that something in the project structure is broken. So better don't tamper with your project data outsides the application.

[Biep]

Thanks!
I have "tampered" with the character order, and in one little place in the (extended) folder structure of the text (because the move-up bug had bitten me, so I had to restore from a backup that in other places was outdated) - but this message comes with each folder of the text, from the very first (which was never tampered with).
I can check how far back that goes, but whether I'll be able to reproduce what I did at that time is quite doubtful.

[Biep]

In this case, you wouldn't have to copy things into another project, which seems to have messed up something.

No - the copying repaired it. The original project crashed; the copy doesn't.
But I have removed revisions.xml. I won't return to my old project, which is quite outdated by now, but will do the test.
Thanks for all your detection work!

[SpeedyDoggo]

This does remind me of #1147 (which has links to other similar bugs as well.)

What is interesting to me is that you were also leaving the program open for large periods of time when the error occurred. In my case, I never had revisions on but had the same problem and solution. I don't tend to leave Manuskript running for extended periods of time anymore and haven't had this particular issue in a while, but if you still do, then I'd be curious to see if this recurs.

"Strange things in file..." shows up even in files that haven't been touched; they're all over my project that I've actively avoided altering from outside the program. Issue #1338 asks about it and the actual guess from Jackie is that it's just some oddity related to unicode and cross platform development.

[Kristof5933]

The merged pull request #1400 solves this issue ("lxml.etree.XMLSyntaxError: internal error: Huge input lookup").