[Ask For Help] How generate-car works, more specific how cid generated

[hanbu97]

When I was implementing an App to send user's file directly to Filecoin network. I was obstacled at the first step, to generate the cid of the given file. Which is quite important for real world applications to use Fil as a decentralized storage. However, after some investigating about this, the only way I found to generate cid is the Lotus implemention, even for recent popolar app(boost), use Lotus's implemention directly.

I'm a rust developer. So I use fvm_ipld_car crate, I read user's file as Vec bytes and generate cid with Blake2b256.

let mut file = async_std::fs::File::open(&input).await?;
let mut data: Vec<u8> = vec![];
file.read_to_end(&mut data).await?;

const RAW: u64 = 0x55;
let cid = cid::Cid::new_v1(RAW, Blake2b256.digest(&data));

pub const DAG_CBOR: u64 = 0x71;
let cid = cid::Cid::new_v1(DAG_CBOR, Blake2b256.digest(&data));

However, in this way only when file size is tens of kb, it works and give the same cid as Lotus's(Boost's). For a very small file it gives a shorter hash and for very large file (tens of GB) the hash length is same but with different results. And for file size like 1.4MB it gives a very long hash.

It is obvious, when Lotus is generating a cid, it uses different hash method for different file size and seems do something else than directly send the file bytes to cid generator.

I noticed docs like https://docs.ipfs.tech/concepts/content-addressing/#cid-conversion . But did not explain how detailed cid and car file is generated.

In this way, I really want a clearer explanation about how Lotus generate cid and car file from a vec of bytes.