Releases: storacha/ipfs-car
v3.1.0
v3.0.0
v2.0.0
v1.2.0
v1.1.0
v1.0.0
1.0.0 (2023-03-20)
⚠ BREAKING CHANGES
- The programmatic API has changed significantly, see the README for new streaming API. The CLI "commands" like
--packhave changed topack(i.e. without dashes) but are largely very similar. In the CLI, CAR files written to stdout or piped to another program (i.e. not written to disk using--output) will not have a root CID in the CAR header. Minimum Node.js version for the CLI has changed to 18.
Features
CLI Migration Guide
Packing files into a CAR
Before:
ipfs-car --pack path/to/file/or/dir --output path/to/write/a.carAfter:
ipfs-car pack path/to/file/or/dir --output path/to/write/a.carPacking a file into a CAR, without wrapping in a directory
Before:
ipfs-car --pack path/to/files --wrapWithDirectory false --output path/to/write/a.carAfter:
ipfs-car pack path/to/files --no-wrap --output path/to/write/a.carUnpacking files
Before:
ipfs-car --unpack path/to/my.car --output /path/to/unpack/files/toAfter:
ipfs-car unpack path/to/my.car --output /path/to/unpack/files/toListing root CIDs
Before:
ipfs-car --list-roots path/to/my.carAfter:
ipfs-car roots path/to/my.carListing block CIDs
Before:
ipfs-car --list-cids path/to/my.carAfter:
ipfs-car blocks path/to/my.carListing files
Before:
ipfs-car --list path/to/my.carAfter:
ipfs-car ls path/to/my.carListing files with CIDs
Before:
ipfs-car --list-full path/to/my.carAfter:
ipfs-car ls path/to/my.car --verboseGenerating CAR CID
Before:
ipfs-car --hash path/to/my.carAfter:
ipfs-car hash path/to/my.carAPI Migration Guide
Packing files into a CAR
The ipfs-car module now uses web streams. Note that due to the streaming nature of DAG and CAR generation the programmatic API does not produce CAR files with any roots in the CAR header.
import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car'
const file = new Blob(['Hello ipfs-car!'])
const carStream = createFileEncoderStream(file).pipeThrough(new CAREncoderStream())
// carStream.pipeTo(somewhereWritable)Obtaining the root CID
The root CID is the CID of final block generated by the file/directory encoder stream. Use a transform stream to record the CID of the last block generated:
import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car'
const file = new Blob(['Hello ipfs-car!'])
let rootCID
await createFileEncoderStream(file)
.pipeThrough(new TransformStream({
transform (block, controller) {
rootCID = block.cid
controller.enqueue(block)
}
}))
.pipeThrough(new CAREncoderStream())
.pipeTo(new WritableStream())
console.log(rootCID.toString())Adding root CIDs to the CAR header
If you need root CIDs in the CAR header, there are two approaches you can use:
-
Buffer all the DAG blocks, then encode with known root:
import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car' const file = new Blob(['Hello ipfs-car!']) const blocks = [] // buffer the output await createFileEncoderStream(file) .pipeTo(new WritableStream({ write: b => blocks.push(b) })) const rootCID = blocks.at(-1).cid const blockStream = new ReadableStream({ pull (controller) { if (blocks.length) { controller.enqueue(blocks.shift()) } else { controller.close() } } }) await blockStream .pipeThrough(new CAREncoderStream([rootCID])) // pass root to CAR encoder .pipeTo(new WritableStream())
-
Write to disk with placeholder CID, then update after DAG is completely generated (Note: Node.js only):
import fs from 'fs' import { Writable } from 'stream' import { CarWriter } from '@ipld/car/writer' import { CID } from 'multiformats/cid' import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car' // Root CID written in CAR file header before it is updated with the real root CID. const placeholderCID = CID.parse('bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi') const file = new Blob(['Hello ipfs-car!']) let rootCID await createFileEncoderStream(file) .pipeThrough(new TransformStream({ transform (block, controller) { rootCID = block.cid controller.enqueue(block) } })) .pipeThrough(new CAREncoderStream(placeholderCID)) .pipeTo(Writable.toWeb(fs.createWriteStream('path/to/my.car'))) // update roots in CAR header const fd = await fs.promises.open(opts.output, 'r+') await CarWriter.updateRootsInFile(fd, [rootCID]) await fd.close()
Unpacking files from a CAR
This is no longer provided by this library, but is easy to do with @ipld/car and ipfs-unixfs-exporter modules:
import { CarIndexedReader } from '@ipld/car/indexed-reader'
import { recursive as exporter } from 'ipfs-unixfs-exporter'
const reader = await CarIndexedReader.fromFile('path/to/my.car')
const roots = await reader.getRoots()
const entries = exporter(roots[0], {
async get (cid) {
const block = await reader.get(cid)
return block.bytes
}
})
for await (const entry of entries) {
if (entry.type === 'file' || entry.type === 'raw') {
console.log('file', entry.path, entry.content)
} else if (entry.type === 'directory') {
console.log('directory', entry.path)
}
}