Search with 300k record fails

[miyuru-vs1bv]

Describe the bug
After adding 300k reocrds search does not work. BSONObj size: 26823947 (0x1994D0B) is invalid. Size must be between 0 and 16793600(16MB) First element: _id: ObjectId('680f0b064df205001d3f78e3') on search endpoint.

Which feature of OpenCRVS your bug concern?
Search

To Reproduce
Steps to reproduce the behaviour:

1. Insert large number of records. (300k)
2. Search for a record

Expected behaviour
Successful search.

Actual behaviour
No records in UI, and error in Kibana

Screenshots
If applicable, add screenshots to help explain your problem.

OpenCRVS Core Version:

• v1.8.0, + Fix migration issue discovered when restoring an OpenCRVS instace with a large number of records #9398

Country Configuration Version:

• v1.8.0

Desktop (please complete the following information):

NA

Smartphone (please complete the following information):

NA

Root cause
MongoDB large document

Additional context
Only happens with large number of records, small records work without issue.

Possible fixes
https://github.com/opencrvs/opencrvs-core/blob/develop/packages/search/src/features/records/service.ts

Reproducible demo

NA

[euanmillar]

It seems this is a bug with a custom data migration script that has been written as we cannot replicate the source issue on any of our records.