Convert more of the main code to async, and write to markdown as results arrive. This means that, in the case of a crash, some results are still saved, and we can resume.
By implication, we might want to track "last updated" by document. Then we can avoid reprocessing highlights more than once. We could do that in .memorymarker.
OTOH, this might also be a lot of complexity for benefit only at times where we process large batches.
Convert more of the main code to async, and write to markdown as results arrive. This means that, in the case of a crash, some results are still saved, and we can resume.
By implication, we might want to track "last updated" by document. Then we can avoid reprocessing highlights more than once. We could do that in
.memorymarker.OTOH, this might also be a lot of complexity for benefit only at times where we process large batches.