Compute CRC of whole file

[BenWhite-Price]

Hi,

I'm trying to verify that a file transfer was successful, and was hoping to compute a CRC of the file in LFS. The CRC function in my embedded platform needs the file size (easy to get from LFS) and the start address (hard to get). My question is twofold:

1. Is it possible to get the start address of the file, and if so, is the file stored contiguously?
2. If not, is there a built-in way to compute a CRC for a whole file?
   Thanks!

[geky]

Hi @BenWhite-Price, interesting question, sorry for the late response.

The short answer: Your best option is probably to read the file normally and CRC in chunks. Larger chunks would be faster CPU at a RAM tradeoff.

littlefs is designed to work with storage that needs to go through the bd API, so providing direct addresses to the data wasn't really a consideration. The data is organized into blocks with an invasive CTZ skip-list and likely non-contiguous. You could try to poke around in the internals, but I think it would be more complicated than it's worth, DESIGN.md#ctz-skip-lists might be a good starting point.

1. Is it possible to get the start address of the file, and if so, is the file stored contiguously?

Not via the stable API.

Actually, it's not even available in lfs_file_t, you would need to chase the CTZ pointers, which is implemented in lfs_ctz_find.

2. f not, is there a built-in way to compute a CRC for a whole file?

No.

Maybe it would be an interesting future API to let the user iterate over data addresses (blocks/fragments) in a file, but right now it's only wishful thinking. And may be difficult to add. It's in theory possible though.

One thing that adds to the intrigue of this question is the context of littlefs3 (#1111, unstable), in which we're looking to add more checksumming APIs. This includes lfs3_file_ck which can easily validate the checksums of a file.

But while this lfs3_file_ck API sounds relevant, the Merkle-tree-esque checksum scheme used in littlefs3 and most filesystems is incompatible with the sequential checksum you'd probably calculate outside of the filesystem. One challenge is supporting random-writes, and the Merkle-tree-esque scheme includes filesystem metadata not present in the actual file.

---

There's a related issue to add a sort of "direct access" file, which can be accessed directly for XIP, etc: #692 This conflicts with wear-leveling and other filesystem operations, but may be somewhat reachable by adding filesystem shrinking to allow raw storage allocation for users (outside of wear-leveling, etc).

Though I'm not sure that fits in this case.