CODEWITHSUNDEEP
javascriptsha256ipfs
pale bone
pale bone

Reputation: 1836

Node-less way to generate a CID that matches IPFS-Desktop CID

I'd like to generate a CID (Content identifier) for a file in javascript without having access to an IPFS node or the internet. I've tried using js-multihashing-async to first hash the file and js-cid to generate a CID from the hash but I get a different CID than if I just add the file to ipfs-desktop. It looks like the problem is an IPFS node chunks data and the CID is for the DAG that links the files' chunks. I've tried this library but it doesn't produce the same CID as ipfs-desktop does for the same file. This question is essentially the same as mine but none of the answers give a CID that matches the ipfs-desktop-generated CID.

Upvotes: 11

Views: 4957

Answers (2)

pale bone
pale bone

Reputation: 1836

For the sake of posterity, here is how to match an image's CID downloaded via fetch to the CID generated from ipfs-desktop for the same image (added as a file from the local drive). You have to remove the prefix data:*/*;base64, that is prepended to the image's base64string and decode the string into a buffer array. Then you get the matching CID. 

async testHashes() {
  const url = "https://raw.githubusercontent.com/IanPhilips/jst-cids-test/master/src/23196210.jpg";
  fetch(url)
    .then(response => response.blob())
    .then(blob => new Promise((resolve, reject) => {
        const reader = new FileReader();
        reader.onloadend = () => resolve(reader.result);
        reader.readAsDataURL(blob)

    })).then(async dataUrl =>{

        const strData = dataUrl as string;
        // remove "data:*/*;base64," from dataUrl
        const endOfPrefix = strData.indexOf(",");
        const cleanStrData = strData.slice(endOfPrefix+1);
        const data = Buffer.from(cleanStrData, "base64");
        const hash = await Hash.of(data);
        console.log("fetch data CID: " + hash); // QmYHzA8euDgUpNy3fh7JRwpPwt6jCgF35YTutYkyGGyr8f
    });

  console.log("ipfs-desktop CID: QmYHzA8euDgUpNy3fh7JRwpPwt6jCgF35YTutYkyGGyr8f");
}

Upvotes: 3

olizilla
olizilla

Reputation: 446

ipfs-only-hash is the right module to use to create an IPFS CID from a file or a Buffer, without needing to start an IPFS daemon. For the same input file and the same options, it should produce the same CID.

This example is from the ipfs-only-hash tests, where it verifies that it hashes the same buffer to the same CID as a js-ipfs node does.

test('should produce the same hash as IPFS', async t => {
  const data = Buffer.from('TEST' + Date.now())
  const ipfs = new Ipfs({ repo: path.join(os.tmpdir(), `${Date.now()}`) })

  await new Promise((resolve, reject) => {
    ipfs.on('ready', resolve).on('error', reject)
  })

  const files = await ipfs.add(data)
  const hash = await Hash.of(data)

  t.is(files[0].hash, hash)
})

https://github.com/alanshaw/ipfs-only-hash/blob/dbb72ccfff45ffca5fbea6a7b1704222f6aa4354/test.js#L21-L33

I'm one of the maintainers of IPFS Desktop, and under the hood, that app calls ipfs.add on http api for the local IPFS daemon here

When adding or hashing a file manually via the api, there are options to alter how files are chunked into blocks, how those blocks are linked together, and how the blocks are hashed. If any option values differ then the resulting hash and the CID that contains it will be different, even if the input file is the same.

You can experiment with those options and see a visualisation of the resulting DAG (Directed Acyclic Graph) structure here: https://dag.ipfs.io/

For a deep dive on how IPFS chunks and hashes files you can see the author of the ipfs-only-hash and maintainer of js-ipfs explain it here https://www.youtube.com/watch?v=Z5zNPwMDYGg

Upvotes: 12

Related Questions