Size and datatypes for nilDB

[BobbyTable]

Hi,

Im looking to store bytes in nilDB, but am unable to find any documentation or specs about the abilities. This page here about the Blindfold library, says 32-bit signed integer; UTF-8 string (4096 bytes or fewer). And as i understand it, the Secretvault SDK depends on this.

In a blogpost here theres a mention of secretBlob beeing increased in size from 100k to 1mb. This would be interesting, but i cant fint it mentioned in docs or source code.

Im looking to store up to a few megabytes per entry (1mb size limit is ok, I will just implement splitting)

[BobbyTable]

Using Secretvault, the Blindfold library throws further down in the stack, if i pass a larger string - so that's confirmed 😁
errorMessage: 'plaintext must be possible to encode in 4096 bytes or fewer'

[psofiterol]

hey @BobbyTable 👋

Quick note: The blogpost you've linked is from May 2024, and a different product altogether - nilDB and blindfold were not around at that point!

You should be able to store and encrypt bytes in nilDB using blindfold, yes!
As you've already noted, you'll simply have to make sure each object to encrypt is <= 4096 bytes in size.
You'll likely find this example for storing files using the Python SDK useful! It encodes the files as string and splits it into chucks before encrypting them. You can use it for inspiration.

Note: The per-record size limit in nilDB is actually 16MB, so your array of encrypted chunks should fit into a single record.

[BobbyTable]

The Python code linked produces chunks of 1000 bytes each, and creates a record per chunk, if i read correctly Your last statement of record size confuses me, are you saying i can add a single object up to 16MB and get a single _id? Or is the term ”record” something else here?

…

On Mon, 11 Aug 2025 at 14:50, Georgios Pentafragkas < ***@***.***> wrote: hey @BobbyTable <https://github.com/BobbyTable> 👋 *Quick note: The blogpost you've linked is from May 2024, and a different product altogether - nilDB and blindfold were not around at that point!* You should be able to store and encrypt bytes in nilDB using blindfold, yes! As you've already noted, you'll simply have to make sure each object to encrypt is <= 4096 bytes in size. You'll likely find this example for storing files <https://github.com/NillionNetwork/secretvaults-py/blob/main/examples/file_encryption_example.py> using the Python SDK useful! It encodes the files as string and splits it into chucks before encrypting them. You can use it for inspiration. Note: The per-record size limit in nilDB is actually 16MB, so your array of encrypted chunks should fit into a single record. — Reply to this email directly, view it on GitHub <#199 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLTVUGPIZXBI5JGYLCHLBD3NCGSFAVCNFSM6AAAAACDSRE7EKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTIMBXGE2DAMA> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[psofiterol]

The Python code linked produces chunks of 1000 bytes each, and creates a

record per chunk, if i read correctly

Yes, that's correct. In my statement above I suggested it creates one record with the chunks held in an array within it which would also be viable - but as you said in this case it's 1 record per chunk.

Your last statement of record size confuses me, are you saying i can add a

single object up to 16MB and get a single _id?

Or is the term ”record” something else here?

Yes, correct. The size limit for one record (one _id) is 16MB. Note that the 4096B limitation per encrypted object within that record that comes from blindfold still applies, but you can have as many of those as you want in a record.

[BobbyTable]

So the shape of the object would be like:

   {
         "_id": str(uuid.uuid4()),
         "file": [{ "%allot": chunk1 }, { "%allot": chunk2 }, ... ],
    }

Or

   {
         "_id": str(uuid.uuid4()),
         "file": { "%allot": [ chunk1, chunk2, ...]},
    }

[psofiterol]

So the shape of the object would be like:

   {
         "_id": str(uuid.uuid4()),
         "file": [{ "%allot": chunk1 }, { "%allot": chunk2 }, ... ],
    }

This one. Each object to be encrypted is to use the %allot syntax 🙏

[BobbyTable]

Fantastic, i will try this! Now a last question - is there a need to operate on base64 encoded strings of the bytes? Cannot the encryption operate on bytes or ints directly, to efficiently use the data limitations (base64 inflates size by around 33%). Since it's never transported over the wire or anything in this state, and does not have a need to be printable characters

[psofiterol]

No need to use base64 encoding, no. 👍

[BobbyTable]

Diving into the code i have noticed this, but also that the encoding is adding what seems to be a "type byte" first to the encryption payload - , so the actual chunks need to be BELOW 4096 in size and NOT equal to it (in order to accomodate this extra byte)