[WIP] add experimental revive option for reproducible CIDs#6509
[WIP] add experimental revive option for reproducible CIDs#6509drbh wants to merge 2 commits intoipfs:masterfrom
Conversation
| // var added int | ||
| addit := special.Entries() | ||
|
|
||
| for addit.Next() { |
There was a problem hiding this comment.
for addit.Next() doesn't initiate a loop on the second default settings. Seems like the entries are removed on each loop. Is there a way we can persist the file as we try different params?
There was a problem hiding this comment.
Technically, you could buffer them all into memory. But that's really not something you want to do.
|
Can we iterate over the flag name? I am not a fan of |
|
agreed ✅I like |
|
Just pushed some minor updates (renamed vars for golint) Also renamed Use |
| } | ||
|
|
||
| if rbset { | ||
| myOpts = append(myOpts, options.Unixfs.RawLeaves(rawblks)) |
There was a problem hiding this comment.
Caveat: RawLeaves are disabled by default when doing ipfs add --cid-version 0 (current default for CIDv0), but are enabled when user opts-in to CIDv1 (ipfs add --cid-version 0.
--reconstruct should see if CID is v0 or v1 and set proper default
| } | ||
|
|
||
| if trickle { | ||
| myOpts = append(myOpts, options.Unixfs.Layout(options.TrickleLayout)) |
There was a problem hiding this comment.
Trickle DAG is created by ipfs files write by default, which complicates things: so we should always try enabling/disabling it as a fallback.
|
If we're offering up a CID to ipfs add --target=QmTfwRggLp9mo7fTF5L5cCxk8djbE6reipLpMMNfYgA6ip /path/to/fileNot a blocker, I can totally live with |
There was a problem hiding this comment.
Unfortunately, this can't be sanely implemented in go-ipfs:
- Files will have to be written to a scratch space (on disk) so they can be re-read when trying new parameters. This would be pretty complicated and invasive addition for a small, infrequently used feature.
- This is a combinatorial problem involving hashing. Users will have to specify the sets of flags that could reasonably have been used.
Instead of implementing this in go-ipfs, it should probably be implemented as an external script (to avoid problem 1) with some kind of DSL for describing possible/likely flag combinations.
An --expect flag could be implemented in go-ipfs but that seems pretty useless given that a user can just check [[ $(ipfs add -q foobar) == "expected" ]].
| // var added int | ||
| addit := special.Entries() | ||
|
|
||
| for addit.Next() { |
There was a problem hiding this comment.
Technically, you could buffer them all into memory. But that's really not something you want to do.
|
This is such an infrequent activity that it should be done manually, or by an external script. It should not be a feature of the command. |
|
If anyone finds this when searching for "reproducible CIDs" relevant work happened in |
6 participants
Draft solution for reviving a CID.
Based on the deep dive specs from IPFS Camp 2019;
This is the case when you have a file - and you know what you expect the CID to be, but you do not know the original params that generated that DAG root.
We implement a
--reviveflag (also called--expected-cidin the spec) on theipfs addcore command.In short
We want to automatically loop through known default settings check the CID and return to the user if we find the correct params.
Example CLI use
Proof of concept work
🐥Add
reviveflag🐥Check if flag is called
🐣Loop over default settings
🐣Return useful information on match
Discussion topics
🥚What is returned to the user?
🥚If its the
fmtstrwhat does that look like?Answerable questions
❓What are the known default settings (when CID versions change)?
❓How to loop over
*files.multipartDirectorymore than once code?