On Chain Data

Detailed Technical Information

The process for embedding NFT images and metadata on the Chia blockchain was meticulously crafted to ensure their eternal integrity, accessibility, and ease of recovery—enshrining your Chia Gods as everlasting companions in the decentralized universe. This system allows both the images and metadata to be retrieved even if external links become unavailable. Below is the detailed implementation:

Image Chunking and Storage

  1. Chunking the Image Data
    • Each NFT image in the collection was divided into chunks small enough to fit within a Chia Blockchain transaction. The exact size does not matter, as long as the image data + any markers fits within a transaction memo.
    • If the last chunk of an image was smaller than the maximum size, it remained a standalone chunk. No data from the next image was added to the same transaction. This decision was made to simplify parsing during data recovery, ensuring clear boundaries between images.
  2. Filename Encoding
    • After each image chunk was prepared for submission, a check was performed for the presence of the image end chunk marker: []byte{0x00, 0x00, 0x00, 0x00, 0x49, 0x45, 0x4E, 0x44, 0xAE, 0x42, 0x60, 0x82}
    • This sequence of bytes is standard for all png images
    • If this marker was detected, the filename of the image was encoded as bytes and appended to the image bytes. This step ensured that filenames could be recovered for all images, though it wasn’t critical to the NFT's integrity.
  3. Special Markers for Collection Boundaries

    To make it easier to regenerate the entire collection, start and end markers were used

    • The first image in the collection included a start marker prior to any image data: []byte("CHIAGODSSTART")
    • The last image chunk included an end marker after both the image data and the file name encoding (outlined in step 2): []byte("CHIAGODSEND")
  4. Transaction Memo Fields
    • Each transaction memo field was used to store raw byte data, combining any special markers (e.g., start or end markers, filenames) with the corresponding chunk of image data.
    • This structure allowed precise reconstruction of the collection from the transaction data on-chain.

Metadata Storage

  1. Bulk Metadata Storage
    • After all image data for the collection was stored, a single transaction was created to store the metadata in bulk.
    • This transaction contained a json list of base64 encoded metadata objects, where each item in the list corresponds to an individual NFT’s metadata in the collection.
    • Each metadata file was read as raw bytes and stored base64 encoded to ensure the order and formatting of the metadata was unchanged, enabling the restored metadata’s hash to match
    • The metadata is in numerical order of the NFT collection, so a specific NFT’s metadata can be retrieved by index using its series number.
    • Similar to the image data, the metadata is surrounded by start and end markers ([]byte("CHIAGODSMETASTART") and []byte("CHIAGODSMETAEND")) - the data between these markers is gzipped json data which can be decoded, decompressed, and parsed as the json structure described above.
  2. Metadata Transaction Structure
    • The metadata transaction was placed directly after the last image data transaction, maintaining a logical and sequential relationship between the image data and metadata.
    • This structure ensured that the blockchain history could be used to trace both the image data and metadata for the entire collection in an organized manner.

Minting the NFTs

  1. NFT Data Linking

    Once the image data and metadata were stored, the NFTs were minted. Each NFT was linked to its corresponding image data via a coin ID reference stored in the NFT’s metadata. The coin ID references the first chunk of the NFT’s image data, simplifying the recovery if the metadata is available, but the image is not.

  2. Transaction Order

    The minting process strictly followed the sequence: Image data transactions → Metadata transaction → NFT minting transactions. This ensured an unbroken lineage that could be easily followed from the minted NFT back to its associated data, and ultimately the beginning of the collection.

Data Recovery Process

  1. Regenerating the Entire Collection

    The start (CHIAGODSSTART) and end (CHIAGODSEND) markers make it easy to regenerate the entire collection from the blockchain. Using our available open-source recovery tools, anyone with access to a synced full node can query the blockchain for transaction data and reconstruct both the image data and metadata.

  2. Recovering Image Data

    Image data for a given NFT can be reconstructed by querying the relevant transactions and combining the chunks in sequential order, ensuring any special markers are stripped before writing to an image file. Each image’s boundaries are clear, as no chunk contains data from a different image.

  3. Recovering Metadata

    Metadata for all NFTs in the collection can be retrieved from the bulk metadata transaction.

  4. Recovering Filenames

    Filenames can be decoded from the byte representation appended to the memo field of each image’s final chunk. This provides an additional layer of recoverability, though filenames are not essential for NFT functionality.

This detailed implementation ensures that NFT images and metadata are securely stored and recoverable. By leveraging structured chunking, markers, and logical transaction sequencing, the system maintains data integrity and provides a clear and user-friendly mechanism for recovery.

Bonus Fact: Coin Lineage Simplifies Recovery

While not necessary to accomplish our main goals, we intentionally kept the entire coin lineage for embedding the NFT image data very short. It begins with a farmer reward, which is then combined with another coin to increase its value to cover any minting costs. From there, the lineage transitions directly into the image data embedding transactions.

This structure ensures that even if the start collection marker (CHIAGODSSTART) is somehow missed when working backwards through the history, recovery scripts will quickly exhaust the parent coin history. Since the lineage terminates at the root of the coin history (the farmer reward), there are no extraneous or unrelated transactions to parse through. This design not only simplifies the recovery process but also guarantees that the collection’s data remains easy to locate and verify on the blockchain.