TLDR: this proposes ipfs add --to-files= as atomic replacement for ipfs add --pin=false + ipfs files cp

Problem

There is a UX gap in our MFS API where users importing data to IPFS via ipfs add and ipfs files write get different CIDs due to different way chunks are assembled into a DAG (balanced vs trickle). Details in issues linked below.

While we hoped the problem will get solved by "/api/v1" it did not happen, and we are still confusing users just like we did in 2016:

• 2022:

  • This remains to be a confusing part of IPFS onboarding experience. Example:

• 2021:

  • https://discuss.ipfs.io/t/ipfs-desktop-uses-ipfs-files-write-or-ipfs-add/12412?u=lidel
  • https://discuss.ipfs.io/t/issue-ipfs-add-and-ipfs-files-write-the-difference-between-the-two/5147?u=lidel

• 2018: Upload via WebUI (ipfs.files.write --create) produces different hash than CLI (ipfs.add) ipfs-webui#676

• 2016: "ipfs add" and "ipfs files write" commands returns different hashes ipfs-inactive/support#45

Unpacking UX challenges

If a user wants to add data to MFS directly, they need to glue two (or more) commands together:

ipfs files cp /ipfs/$(ipfs add -Q test.mp4) /test/test.mp4

ipfs add by default creates a low-level pin which keeps data around even when the file is removed from MFS. This behavior is usually NOT what MFS users want, so they use ipfs files write and get different CID of trickle DAG, and can't benefit from common CID.

One could either do ipfs add with --pin=false (risky when GC is enabled) or remove low-level Pin after files cp is successful (still problematic, in case the CID was pinned for different reasons)

Either way, asking users to execute two or more commands is not nice. What users want and asked for is essentially a better API, an atomic ipfs files cp /ipfs/$(ipfs add -Q --pin=false test.mp4) /test/test.mp4 but with GC safety.

Proposed solution: ipfs files add

Add porcelain command that executes ipfs add --pin=false && ipfs files cp in atomic fashion that is safe from GC.

Main characteristics:

• ergonomics similar to ipfs files write but without ability to append data – instead, allow recursive import of entire dirs like ipfs add -r
• produce the same CID as ipfs add
• atomic operation safe from GC, end result should not add any new low-level pins