Running index.insertItem in parallel results in a corrupted database

[poetsmeniet]

To speed up the process of embedding a large-ish document, I run into issues with a corrupted database.

The error is "Unexpected number in JSON at position x"

The json file (vector database) is clearly corrupted.

Is it even possible to run index.insertItem in parallel?

[Stevenic]

Yes... You need to call index.beginUpdate() first followed by index.endUpdate() to commit your changes to disk. Here's some code:

index.beginUpdate();
try {
  // Do your updates

  // Commit changes to disk
  index.endUpdate();
} catch (err) {
  // Abort changes
  index.cancelUpdate();
  throw err;
}

That will generally speed up mass updates anyway.