You Know It - Prove It!

3 min readDec 16, 2019

Erasure is a decentralized data marketplace that allows users to post and sell their data.

The protocol relies on Ethereum timestamps and data hashing to prove with certainty that the specific data was owned by the poster at that specific time. The Erasure intro post has a nice example of how this could be used:

If you submitted two years of daily Google stock price predictions to Erasure and you were 70% accurate, the whole world would be able to see that and check it for themselves (say against Yahoo! Finance data). Everyone would agree that your historical predictions occurred ahead of time, because all of your predictions are timestamped on Erasure. Now people can believe your claims because they don’t need to trust you.

Many traditional web developers may not have experience working with data hashes or encryption or know how to store that data in a decentralised manner using IPFS or 3Box and that’s why I wrote the Numerai Javascript helper library. It speeds of app development by abstracting away these complexities and allows the user to simply provide their raw data and select a storage method (IPFS or 3Box for now). The library handles encryption and storage and provides the various proof hashes and storage information the user requires to work with the Erasure protocol.

This post goes into some of the detail of what the helper library is doing behind the scenes.

Creating A Post

With the helper library creating a post and saving the data to the user 3Box storage is easy:

import NumeraiHelper from "./numerai-helper";

var helper = new NumeraiHelper('numerai', 'creator_eth_address');var postData = NumeraiHelper.savePost('test create post', '3Box');Upload postData.proofhash to Erasure feed.

But there’s quite a lot going on in the background that’s worth exploring:

Generate a symmetric encryption key

Symmetric-key algorithms are algorithms for cryptography that use the same cryptographic keys for both encryption of plaintext and decryption of ciphertext meaning that symmetricKey can be used to encrypt and decrypt the data.

const symmetricKey = ErasureHelper.crypto.symmetric.generateKey()

Hash The Key

A hash function can be used to map data of arbitrary size to fixed-size values. For a given input value it must always generate the same hash value and it is practically one way which means we can’t get the symmetric key from its hash.

The above properties are all useful in this case as it allows us to store a known size of hash on-chain without encurring large costs and without giving the key itself away. It is also used to prove that any symmetric key provided to decrypt that data matches the one provided when the commitment was made.

const symmetricKeyHash = await ErasureHelper.multihash({
        input: symmetricKey,
        inputType: 'raw',
        outputType: 'hex',
      })

Hash The Data

Similar to above this allows us to store the hash on-chain and prove in the future that any data supplied matches the data provided when the commitment was made.

const dataHash = await ErasureHelper.multihash({
        input: RawData,
        inputType: 'raw',
        outputType: 'hex',
      })

Encrypt The Data

Encrypted using the symmetric key. The encrypted data will be uploaded to decentralised storage.

const encryptedFile = ErasureHelper.crypto.symmetric.encryptMessage(symmetricKey, RawData);

Hash The Encrypted Data

Hashing the encrypted data allows us to compute the Content Identifier which gives us a key value to use for both IPFS and 3Box storage. Or to put it simply, the encrypted data will be accessible at its Content Identifier.

const encryptedDataHash = await ErasureHelper.multihash({
        input: JSON.stringify({encryptedData: encryptedFile}),
        inputType: 'raw',
        outputType: 'b58',
      });

Create the Proof JSON

Now we have all the required information needed. A JSON object can be created:

const jsonblob_v1_2_0 = {
    creator: this.ethAddress,
    salt: ErasureHelper.crypto.asymmetric.generateNonce(),
    datahash: dataHash,
    encryptedDatahash: encryptedDataHash,
    keyhash: symmetricKeyHash
  }

Hash The Proof JSON

Now we has the proof JSON. This hash will be commited on-chain. When it is it becomes a time-stamped proof.

const proofHash58 = await ErasureHelper.multihash({
        input: JSON.stringify(jsonblob_v1_2_0),
        inputType: 'raw',
        outputType: 'b58',
      })

Store The Data

Finally we need to store the information. We store the encrypted data and JSON proof. Both are stored using their content identifiers. The helper allows for storage using IPFS or 3Box. This allows anyone to retrieve, decrypt (if they have the key) and validate the timestamped proof — this method will be detailed in the next post.