Skip to content

Data availability

@johnhenry/raijin-da (0.0.4) abstracts “post these bytes somewhere retrievable” behind one interface, DALayer, with three backends: LocalDA (in-memory, dev/tests), CelestiaDA (a Celestia light node’s REST API), and EthBlobDA (EIP-4844 — a stub that throws). encode()/decode() handle compression framing for the payloads. viem is an optional peer dependency you only need if you implement the ETH path.

Terminal window
npm install @johnhenry/raijin-da

Latest on npm: @johnhenry/[email protected].

What a commitment proves — and doesn’t

Section titled “What a commitment proves — and doesn’t”

submit(data) returns a DACommitment:

Field Meaning
layer Backend name: 'local', 'celestia', 'eth-blobs'
height DA-layer block height at inclusion
index Slot within that height (LocalDA: a counter; CelestiaDA: always 0 — simplified)
hash SHA-256 of the submitted bytes — the part that actually does work

verify(commitment) retrieves what the backend has for that hash and re-hashes it. So it proves exactly one thing: this backend currently serves bytes matching this content hash. It does not prove inclusion at height/index, availability over time, or anything checkable without downloading the full data — there are no namespace proofs, no KZG, no sampling. Treat height/index as retrieval hints, not commitments. If you need real DA guarantees, this package is the interface to build them behind, not the guarantee itself.

Other traps:

  • EthBlobDA throws on all three methods. It pins the interface and its error messages are a step-by-step implementation guide (viem blob sidecars, KZG setup, Beacon API retrieval). Also: Ethereum prunes blobs after ~18 days — the archival problem is yours.
  • encode() is environment-dependent. If the optional fflate package can be imported, shrinkable payloads get deflated with an RJC magic prefix; otherwise everything is RJR raw. decode() of an RJC payload throws when fflate is absent and no inflate was supplied — if any writer might compress, every reader needs fflate installed or an explicit codec (decode(data, { inflate })).
  • decode() caps its output at 16 MiB. DA bytes are untrusted by definition, and DEFLATE reaches roughly 1000:1 on repetitive input, so a few kilobytes can ask for gigabytes. Oversized payloads are refused with DASizeLimitError; raise the cap per call with decode(data, { maxDecompressedSize }) if a payload is genuinely expected to be larger.
  • CelestiaDA refuses to send an auth token over cleartext http: to a non-loopback host. The node token is a node credential and usually carries write access. Use https:, a loopback host (the default http://localhost:26658 is fine), or opt in explicitly with allowInsecureAuth: true.
  • retrieve() throws on a miss; verify() returns false. Same unknown-hash situation, different failure channels — don’t catch around verify.
import { LocalDA, encode, decode } from '@johnhenry/raijin-da'
const da = new LocalDA()
const commitment = await da.submit(await encode(blockBytes))
const bytes = await decode(await da.retrieve(commitment))
await da.verify(commitment) // true
await da.verify({ ...commitment, hash: new Uint8Array(32) }) // false

Content-addressed in-memory store — resubmitting identical bytes lands on the same slot. Test conveniences: nextBlock() advances the reported height and resets the index (heights never move on their own), size counts stored blobs, clear() resets everything. No persistence, no network: its whole job is letting you develop the DA pipeline without infrastructure.

import { CelestiaDA } from '@johnhenry/raijin-da'
const da = new CelestiaDA({
namespace: '00c0ffee00c0ffee', // required — 8-byte hex namespace ID
endpoint: 'http://localhost:26658', // default; a *running light node*, not embedded
authToken: process.env.CELESTIA_NODE_AUTH_TOKEN, // most nodes require one
// allowInsecureAuth: true, // only for a non-loopback http: endpoint
})

submit() posts a PFB (pay-for-blob) to your namespace and returns the inclusion height; retrieve() fetches all blobs in the namespace at that height and returns the one whose hash matches the commitment; verify() is retrieve-and-rehash, mapping any failure — network included — to false. Fees/gas are currently fixed constants in the client. The constructor throws if an authToken is set and endpoint is http: on a non-loopback host, unless allowInsecureAuth: true is passed, and the namespace is percent-encoded into the request path so a caller-supplied namespace cannot rewrite the URL and carry the token to a different route. CelestiaDAOptions (namespace, endpoint, authToken, allowInsecureAuth) is exported for typing.

Constructable (EthBlobDAOptions: rpcUrl, beaconUrl, chainId — defaults localhost:8545, localhost:5052, 1) but every method throws with implementation requirements. The source file (packages/da/src/eth-blobs.ts) contains a complete viem-based sketch: toBlobs()/toBlobSidecars(), KZG via c-kzg-4844, a type-3 transaction, and Beacon API retrieval. If you implement it, keep the DACommitment.hash semantics — content hash of the pre-blob bytes — so verify() stays uniform across backends.

Framing for DA payloads:

  • raw: "RJR" ‖ data
  • compressed: "RJC" ‖ LEB128(decompressed length) ‖ deflate(data)

The compressed frame carries the decompressed length so decode() can refuse an oversized payload before allocating for it. Compression is used only when a deflater is available and the whole frame (length header included) actually shrinks the payload, so tiny or high-entropy payloads pass through raw even with fflate installed. Round-trip is guaranteed: decode(encode(x)) equals x in any single environment.

decode() never returns more than maxDecompressedSize bytes (default MAX_DECOMPRESSED_SIZE, 16 MiB): the declared length is checked against the cap before inflating, the inflater is bounded by that declared length, and the result is checked again in case the inflater ignored its bound. Both frame types are capped.

Errors: every rejection is a DADecodeError (missing or unknown magic, a truncated frame, a failure inside the compression library — wrapped rather than propagated raw), and size refusals are the DASizeLimitError subclass, carrying limit and requested. Options: encode(data, { deflate }) and decode(data, { maxDecompressedSize, inflate }). Supply a codec explicitly in environments where the optional fflate import does not resolve; an Inflate must honor its limit argument as a memory bound, not a post-hoc truncation.

ValidatorNode does not call any of this — DA is not in the block path yet (Architecture explains where it’s headed). The manual wiring is an onBlockFinalized handler: serialize the block, encode(), submit(), store the commitment wherever your app tracks block metadata.

Runnable: npm run example:04 covers the whole surface including the forged-commitment case.