API Reference/
Developer Docs

ZiB Network

Overview

ZiB is decentralised encrypted storage. Files are encrypted client-side before any bytes leave the browser, distributed redundantly across the storage network, and reassembled on read by the CDN. Storage nodes never see plaintext — not the file content, not the filename, not the encryption key.

Your tenant credentials (access_key:secret_key) authenticate API calls. ZiB only knows about tenants — you manage your own users and decide who can access what. ZiB never stores user identities.

Encryption
Client-side
Storage
Distributed
Availability
High — node-failure tolerant
Auth
Bearer token

How It Works

Every file uploaded to ZiB goes through a four-stage pipeline before it is available on the network.

01Encrypt

ZiB generates a fresh per-file encryption key on the backend and returns it to the browser SDK in the upload registration object. The SDK encrypts the file in the browser before any bytes leave the device. The key is stored on the ZiB backend and never sent to storage nodes.

02Distribute

The encrypted payload is broken into redundant fragments and spread across the storage network. The redundancy is high enough that the file remains fully recoverable through multiple simultaneous node failures.

03Isolate

Storage nodes receive ciphertext fragments only — they never receive the encryption key or the plaintext. All node-facing identifiers are opaque UUIDs, so node operators cannot correlate stored fragments with real filenames or object keys.

04Serve

When a client requests a file via cdn.zibnetwork.com/objects/<uuid>, the CDN reassembles the fragments, decrypts with the stored key, and streams the result. The CDN URL is public and auth-free — the UUID itself is the access token.

Quick Start

Integrate ZiB storage into your app in five steps. The key architectural rule: credentials stay on your server, the SDK runs in the browser.

1

Get credentials

Sign in at app.zibnetwork.com → Dashboard → copy your Access Key and Secret Key. Store these as server-side environment variables only — never in browser code or client-side config.

Never expose credentials to the browser. These keys have full access to your account. Put them in .env on your server and access via process.env.ZIB_ACCESS_KEY etc.
2

Load the SDK

Add the SDK script to your HTML. No npm package or build step required.

html
<script src="https://app.zibnetwork.com/sdk/zib-sdk.js"></script>
3

Server route — initiate upload

Create a server route that authenticates your user, then calls ZiB to generate the encryption key and upload URL. The registration object returned has no credentials — it is safe to pass to the browser.

javascript
// Node.js / Next.js App Router
export async function POST(req) {
  // 1. Authenticate with your own system first
  const user = await getAuthenticatedUser(req);
  if (!user) return Response.json({ error: 'Unauthorized' }, { status: 401 });

  const { fileName, fileSize, contentType } = await req.json();

  // 2. Build a scoped object key using your user's ID
  const key = `${user.id}/${Date.now()}-${fileName}`;

  // 3. Call ZiB — credentials stay on the server
  const res = await fetch('https://api.zibnetwork.com/v1/api/upload/initiate', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${process.env.ZIB_ACCESS_KEY}:${process.env.ZIB_SECRET_KEY}`,
    },
    body: JSON.stringify({
      bucket: 'my-bucket',
      key,
      file_size: fileSize,
      // Forward the browser's File.type so the CDN can return the
      // correct Content-Type header. Without this, iOS Safari refuses
      // to render <img> tags and OG/social link previews fall back to
      // text-only because the CDN defaults to application/octet-stream.
      content_type: contentType,
    }),
  });

  if (!res.ok) return Response.json({ error: 'Failed to initiate upload' }, { status: 502 });

  const registration = await res.json();

  // 4. Return the registration object — no credentials in here
  return Response.json(registration);
}
4

Browser — upload the file

In the browser, call your server route to get the registration, then pass it to ZiBStorage.uploadDirect. The SDK handles encryption and upload automatically.

javascript
// Browser — no credentials here
const registration = await fetch('/api/upload-initiate', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    fileName: file.name,
    fileSize: file.size,
    // Pass the browser's MIME type up so the CDN can serve the
    // correct Content-Type — required for <img> tags on iOS Safari
    // and for Open-Graph image unfurling on Slack / Twitter / iMessage.
    contentType: file.type || 'application/octet-stream',
  }),
}).then((r) => r.json());

const result = await ZiBStorage.uploadDirect(file, registration, {
  onProgress: ({ stage, progress }) => {
    console.log(`${stage}: ${progress}%`);
    // stage: 'encrypting' | 'uploading' | 'completing'
  },
});

console.log(result.cdn_url);
// → https://cdn.zibnetwork.com/objects/<uuid>
5

Store the cdn_url

The upload result contains cdn_url and file_id. Save both in your database. cdn_url is the permanent public URL for the file — share it directly with clients or embed in your app.

javascript
// Send back to your server to persist
await fetch('/api/save-file', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    file_id: result.file_id,       // ZiB's internal UUID
    cdn_url: result.cdn_url,       // permanent public URL
    object_key: key,               // the key you provided at initiate
  }),
}).then((r) => r.json());

The Upload Flow

Here is what happens under the hood when you call uploadDirect. The key principle: credentials and keys never cross the server/browser boundary.

Upload Flow
Your Server
ZiB API
Browser
ZiB Node
POST /upload/initiate
with master credentials
→→→
API generates:
• per-file encryption key
• file record
• one-time upload URL
Receives registration
forwards to browser
←←←
Receives registration
no credentials — safe in browser
uploadDirect(file, reg)
Encrypt locally
using key from registration
→→
Stores ciphertext
never sees plaintext or key
Returns:
{ file_id, cdn_url }
Master credentials never leave the server
Your Access Key and Secret Key are only ever used in your server route when calling /v1/api/upload/initiate. They are never included in the registration object or sent to the browser.
The registration is a one-time upload token
The registration object contains a presigned upload URL, the target node address, and a reference to the encryption key stored on the backend. It is not a credential — it authorises one specific upload and nothing else.
Encryption happens before the node sees any bytes
The SDK encrypts the file in the browser before sending data to the node. The node receives only ciphertext.
Nodes store ciphertext only
Storage nodes never receive the encryption key. They store encrypted fragments indexed by UUID — they cannot read, correlate, or expose file content.
Self-enforcing encryption
If someone obtained a registration token and uploaded plaintext directly (bypassing the SDK), the CDN would decrypt those bytes with the stored key and serve garbage. There is no "unencrypted mode" — files uploaded without the SDK cannot be served correctly. The system is self-enforcing.
Multipart is transparent
For large files, ZiB automatically switches to multipart upload. The browser call is identical — uploadDirect handles single vs multipart routing transparently based on file_size returned by the API.
Re-upload to the same key replaces the prior file
If you upload to a (bucket, object_key) that already exists — for any file type, image or video or JSON — the prior file_id is marked cancelled and a new one is issued. The CDN URL cdn.zibnetwork.com/objects/{file_id} always points to the latest upload. Route lifecycle events and stored references by file_id, not object_key. See the "Re-uploads supersede previous uploads" callout in Webhooks for full event semantics.

SDK Reference

The ZiB SDK is a single browser-side JavaScript file with no dependencies. Load it once via script tag.

html
<script src="https://app.zibnetwork.com/sdk/zib-sdk.js"></script>

Primary upload method

staticZiBStorage.uploadDirect(file, registration, options)

The primary upload method. No class instantiation needed — call it as a static method. Encrypts the file client-side using the key generated by the API during initiate, then uploads to the assigned storage node. Automatically uses multipart for large files.

fileFile | Blob | ArrayBufferThe file to upload.
registrationobjectThe object returned by POST /v1/api/upload/initiate on your server.
options.onProgress({ stage, progress }) => voidProgress callback. stage is one of encrypting, uploading, completing. progress is 0–100.
ReturnsPromise<{ file_id: string, cdn_url: string }>

Account-level operations

For listing, deleting, and encoding, instantiate the class with credentials. These calls should be made from your server, not the browser — credentials are required.

new ZiBStorage({ accessKey, secretKey })

Create an instance for account-level operations. Requires your tenant access key and secret key.

accessKeystringYour ZiB tenant access key.
secretKeystringYour ZiB tenant secret key.
storage.listBuckets()

List all S3 buckets in your account.

ReturnsPromise<string> — S3-compatible XML
storage.listObjects(bucket)

List all objects in a bucket.

bucketstringBucket name.
ReturnsPromise<string> — S3-compatible XML
storage.deleteObject(bucket, key)

Delete an object and remove all of its fragments from the storage network.

bucketstringBucket name.
keystringObject key.
ReturnsPromise<void>

Video encoding

storage.startEncoding(fileId)

Trigger HLS encoding (AES-128 encrypted) for an uploaded video. Pass the file_id returned by uploadDirect. Encoding runs asynchronously — poll getEncodingStatus for progress.

fileIdstringThe file_id UUID returned by uploadDirect.
ReturnsPromise<{ encoding_id, status, hls_manifest_url }>
storage.getEncodingStatus(jobId)

Poll the status of an encoding job.

jobIdstringThe encoding_id returned by startEncoding.
ReturnsPromise<{ encoding_id, status, progress, hls_manifest_url }>
storage.getCdnUrl(fileId)

Construct the CDN URL for a file. Equivalent to the cdn_url field returned by uploadDirect.

fileIdstringThe file_id UUID.
Returnsstring — https://cdn.zibnetwork.com/objects/<uuid>

Server-side Integration

The ZiB SDK runs in the browser. For server-side code (Node.js, Python, etc.) call the ZiB API directly with fetch or any HTTP client. All endpoints accept the same Bearer auth format.

Authentication

All authenticated endpoints accept a single header:

http
Authorization: Bearer <access_key>:<secret_key>

Your access key and secret key are displayed once when you create a storage account. Store them as environment variables (ZIB_ACCESS_KEY, ZIB_SECRET_KEY). No SigV4 signing is required.

Node.js example — initiate + upload

javascript
// Server: initiate the upload (credentials stay on your server)
const initRes = await fetch('https://api.zibnetwork.com/v1/api/upload/initiate', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${process.env.ZIB_ACCESS_KEY}:${process.env.ZIB_SECRET_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    bucket: 'my-bucket',
    key: 'videos/intro.mp4',
    file_size: fileBuffer.length,
    content_type: 'video/mp4',  // forwarded to the CDN's Content-Type header
  }),
});
const registration = await initRes.json();
// registration = { type, file_id, upload_url?, upload_id?, ... }
// (Additional opaque fields are returned — pass the entire object straight to the SDK.)

// Return 'registration' to the browser for ZiBStorage.uploadDirect()
// or use the SDK on the server side if you need a non-browser upload path.

Python example — list objects

python
import requests

headers = {"Authorization": f"Bearer {ZIB_ACCESS_KEY}:{ZIB_SECRET_KEY}"}

# List buckets
buckets = requests.get("https://s3.zibnetwork.com/s3/", headers=headers)

# List objects
objects = requests.get("https://s3.zibnetwork.com/s3/my-bucket", headers=headers)

# Start encoding
res = requests.post(
    "https://api.zibnetwork.com/v1/api/encoding/jobs",
    headers={**headers, "Content-Type": "application/json"},
    json={"object_key": file_id, "transcription": "recommended"},
)
job = res.json()  # { encoding_id, status, progress, ... }

Server-side uploads

If you need to upload from a server (not a browser), use the same POST /v1/api/upload/initiate endpoint to get a registration object, then pass it to the ZiB SDK on the server. The SDK encapsulates the encryption and upload format — your code never has to construct the wire format directly.

The encryption wire format is intentionally opaque to integrators. Use the SDK to produce a valid upload — it handles key handling, framing, and chunking transparently. If you have a use case that genuinely cannot use the SDK, contact support.

Upload Pipeline

A pipeline is a tenant-declared chain of stages that should run on a file after upload — encoding (HLS), transcription (SRT/VTT subtitles), and vision analysis (sidecar JSON). Declare the whole chain in a single field on POST /v1/api/upload/initiate and ZiB will (1) route the upload to a storage node that can fulfil every stage and (2) auto-enqueue each downstream stage as soon as the file finishes sharding. You poll a single endpoint — GET /v1/api/pipeline/{file_id} — to track every stage from one place.

The pipeline field replaces the older capabilities array. The legacy array is still accepted for backwards compatibility but new integrations should use pipeline. Pipeline declarations carry tier values (e.g. "recommended" vs "accurate") which the legacy array cannot express.

Pipeline shape

json
{
  "encode": true,
  "transcription": "fastest" | "recommended" | "accurate",
  "vision": "standard" | "hq",
  "webhook_url": "https://yourapp.com/api/zib-pipeline-event",
  "webhook_secret": "whsec_..."
}

Every field is optional. Omit a stage to skip it. The webhook fields are also optional — if provided, ZiB fires HMAC-signed events as each stage completes (in addition to the per-file file.ready webhook configured at the customer level).

Available stages

encode

HLS video transcoding (H.264 multi-bitrate). Required for all video uploads that will be streamed.

transcription

Speech-to-text transcription with three quality tiers. Produces SRT and VTT subtitle files served from the CDN.

vision

AI vision analysis with two quality tiers. Generates a sidecar JSON with scene understanding, ad targeting signals, IAB categories, and more.

Server-side pattern (recommended)

Declare the pipeline on your server when calling POST /v1/api/upload/initiate. The browser only receives the registration object — it never needs to know what jobs will run.

typescript
// server route — e.g. /api/upload-initiate
const res = await fetch('https://api.zibnetwork.com/v1/api/upload/initiate', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: `Bearer ${ACCESS_KEY}:${SECRET_KEY}`,
  },
  body: JSON.stringify({
    bucket: 'video',
    key: `videos/${contentId}/${videoType}-${Date.now()}-${fileName}`,
    file_size: fileSize,
    pipeline: {
      encode: true,
      transcription: 'recommended',
      vision: 'standard',
      // Thumbnails are ON by default — omit the field for defaults.
      // thumbnails: false  → opt out
      // thumbnails: { poster: { at: '10%' }, sprite: { interval: 5, width: 160 } }
      //                     → override individual fields
      // Vertical aspect-variant renditions for short-form / Shorts feeds.
      // OFF by default — omit the field for none. Current v1 accepts only ["9:16"].
      // aspect_variants: ['9:16']  → enable a 9:16 vertical rendition alongside
      //                              the main HLS. See "Aspect variants" below.
      webhook_url: 'https://yourapp.com/api/zib-pipeline-event',
      webhook_secret: process.env.WEBHOOK_SECRET,
    },
  }),
})

const registration = await res.json()
// Return registration to browser — credentials stay on the server
return Response.json(registration)

Polling pipeline status

Once the upload finishes, poll GET /v1/api/pipeline/{file_id} for the rolled-up status of every stage. The top-level status field advances through queued uploading encoding ai_processing complete (or failed). Stages that were not requested in the original declaration are returned as null.

json
{
  "file_id": "a1b2c3d4-...",
  "status": "ai_processing",
  "cdn_url": "https://cdn.zibnetwork.com/objects/a1b2c3d4-...",
  "declared": { "encode": true, "transcription": "recommended", "vision": "standard" },
  "stages": {
    "upload":        { "status": "complete",
                       "phase": "distributed",
                       "replication_safe_at": null },
    "encoding":      { "status": "completed", "encoding_id": "...", "progress": 100,
                       "hls_manifest_url": "https://cdn.zibnetwork.com/...m3u8" },
    "transcription": { "status": "running", "tier": "recommended", "progress": 40,
                       "output_file_id": null },
    "vision":        { "status": "queued", "tier": "standard", "progress": 0,
                       "output_file_id": null }
  },
  "error": null
}

The stages.upload block exposes three fields beyond status:

  • phase — finer-grained lifecycle: pendinguploadingdistributedsafe (≥3 verified non-originator peers per shard) or cancelled (superseded by re-upload, or admin-cancelled). Authoritative for non-video uploads where there's no encoding job to read from.
  • replication_safe_at — ISO timestamp when the file became fully replication-safe. null until the safety sweep flips phase to safe.
  • cancelled_reason — present only when phase === 'cancelled'. Tenant-readable string (e.g. "superseded by re-upload to same bucket+object_key").

If stages.upload.phase === 'cancelled', the rolled-up status is forced to failed regardless of downstream stage state — superseded uploads are terminal even if encoding had partially run. Listen for the file.failed webhook event for an out-of-band signal.

Security note: Always declare pipelines from your server, never from the browser. Master credentials must not reach the browser — call /v1/api/upload/initiate from your backend, then pass the returned registration object to ZiBStorage.uploadDirect() in the browser.

Node selection

ZiB picks a storage node that supports every stage in your declaration — if you ask for vision, only nodes that have registered vision capability in the live capability table are eligible. If no online node can fulfil the full pipeline, the request fails with HTTP 503 immediately so you can fall back gracefully. Capability checks are always against the canonical capability registration, never a stale snapshot.

Upload via Node (PIN)

An alternative to the browser upload path. Instead of having the user's browser encrypt and PUT every byte through the network, you generate a single-use PIN that the user enters in their ZiB Node desktop app. The node app reads the file from local disk, encrypts it natively, shards it, and registers the file — all without the browser ever touching the bytes. After completion, your existing pipeline (encoding, transcription, vision) runs identically to a browser upload — the same file.ready and encoding.* webhooks fire on the same webhook URL.

When to use this. Anyone whose source file already lives on (or near) a machine running a ZiB node. Creators uploading multi-GB videos go from ~30 minutes (slow upstream) to under a minute (local disk read + encrypt). Mobile users on bad cellular pipes can trigger the upload from their phone but the heavy lifting happens on a node at home. Anyone without a node falls back to the normal browser upload path — just keep both buttons.

Flow

  1. Your server calls POST /v1/api/upload/pin/create with the same params as /upload/initiate (bucket, key, pipeline). Returns { pin, session_id, expires_at }.
  2. You display the 3-word PIN to the user (and optionally a QR code).
  3. The user opens the ZiB Node desktop app, clicks Upload → Enter PIN, types the PIN, and chooses a file.
  4. The node app encrypts + shards the file locally and registers it with the network.
  5. Your server polls GET /v1/api/upload/pin/status/{session_id} until status: "complete" (or use the SDK helper below).
  6. The same file.ready webhook fires, the encoding pipeline runs as normal, and you receive encoding.complete on your existing webhook URL.

Server: create a PIN

javascript
// Server-side only — your master Bearer credentials never reach the browser.
const res = await fetch('https://api.zibnetwork.com/v1/api/upload/pin/create', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${process.env.ZIB_ACCESS_KEY}:${process.env.ZIB_SECRET_KEY}`,
  },
  body: JSON.stringify({
    bucket: 'video',
    key: 'videos/abc-123/main.mp4',
    pipeline: {
      encode: true,
      webhook_url: 'https://yourapp.com/api/zib-webhook',
      webhook_secret: process.env.ZIB_WEBHOOK_SECRET,
    },
  }),
});

const { pin, session_id, expires_at } = await res.json();
// pin → "mosaic-trinket-blanket"
// session_id → "5a6b862f-9bb6-4c5a-b5f3-df22bb9da959"
// expires_at → "2026-04-10T15:47:03Z"  (24 hours from creation)

Server: poll for completion

javascript
// Poll until phase is 'ready' or 'failed'. The status endpoint returns
// a full lifecycle rollup so you can render byte-level upload progress,
// then encoding %, then analysis %, all from the same poll.
async function waitForCompletion(sessionId, onProgress) {
  while (true) {
    const res = await fetch(
      `https://api.zibnetwork.com/v1/api/upload/pin/status/${sessionId}`,
      {
        headers: { 'Authorization': `Bearer ${ACCESS_KEY}:${SECRET_KEY}` },
      }
    );
    const s = await res.json();

    // Surface live progress to the UI. `phase` walks through:
    //   pending → redeemed → uploading → encoding → analyzing → ready
    // Each phase populates its own sub-block (defensive read with ?? null
    // so you don't break if a future phase is added).
    onProgress?.({
      phase: s.phase,
      uploadPercent:    s.upload?.progress_percent      ?? null,
      encodingPercent:  s.encoding?.progress_percent    ?? null,
      analysisStage:    s.analysis?.current_stage       ?? null,
      transcriptionPct: s.analysis?.transcription_progress ?? null,
      visionPct:        s.analysis?.vision_progress     ?? null,
    });

    if (s.phase === 'ready')   return { file_id: s.file_id };
    if (s.phase === 'failed' || s.phase === 'expired') {
      throw new Error(`Upload ${s.phase}: ${s.error_message ?? 'no detail'}`);
    }

    await new Promise(r => setTimeout(r, 2000));  // poll every 2s
  }
}

Lifecycle phases

The phase field on /pin/status walks through the full upload-and-pipeline lifecycle so a single button can reflect every stage. The legacy status field is still emitted at the top level for backwards compatibility — new integrations should read phase.

pendingPIN created, waiting for a node to redeem it.
redeemedA node has entered the PIN. The user is now picking a file.
uploadingNode is sending bytes to ZiB storage. Read upload.progress_percent for byte-level progress (updated ≥ every 2s).
encodingUpload landed. Encoder is building HLS variants. Read encoding.progress_percent.
analyzingEncoding done. Whisper / Qwen-VL is running. Read analysis.current_stage and the matching transcription/vision_progress.
readyEvery requested stage has finished. file_id is final. CDN URL is live.
failedUpload or a downstream stage failed. Get a new PIN to retry — the failed PIN is dead.
expiredPIN was not redeemed within 24 hours. Get a new PIN.

Status response shape

json
{
  // Top-level rollup
  "phase": "encoding",                      // pending | redeemed | uploading | encoding | analyzing | ready | failed | expired
  "status": "uploading",                    // legacy field — keep reading 'phase' instead
  "redeemed_by_node_id": "ZB-899233",
  "file_id": "0c8e3765-9b2b-4a29-82cb-2f1911fe750c",
  "file_size_bytes": 1823450000,
  "error_message": null,
  "expires_at": "2026-04-15T15:47:03Z",

  // Populated while phase === "uploading" (omitted if no progress
  // report has landed in the last 30 seconds — node may have stalled)
  "upload": {
    "bytes_uploaded": 412300123,
    "total_bytes": 1823450000,
    "progress_percent": 22.6
  },

  // Populated when the pipeline declares encode: true
  "encoding": {
    "status": "encoding",                   // queued | encoding | complete | failed
    "progress_percent": 47.3
  },

  // Populated when the pipeline declares transcription and/or vision.
  // current_stage tracks the slowest path so a single bar reflects the
  // overall analysis progress.
  "analysis": {
    "current_stage": "transcription",       // "transcription" | "vision"
    "transcription_status": "running",      // null if not declared
    "transcription_progress": 42,           // 0–100, null if not declared
    "vision_status": "queued",              // null if not declared
    "vision_progress": 0                    // 0–100, null if not declared
  }
}
Update cadence. The redeeming node POSTs byte-level progress to the backend every ~2 seconds while uploading. Encoding and analysis progress come from the encoder/AI workers as they run. Polling more often than every 2 s buys you nothing — the underlying numbers don't move faster than that.
Single use, fail forward. Every PIN can only be redeemed once. If anything goes wrong — wrong PIN entered, upload aborted, network blip mid-shard, or the 24-hour timer expires — the PIN is permanently dead and your user has to request a new one. There is no /pin/extend or /pin/retry endpoint. In your UI, when the status becomes failed or expired, show a "Get a new PIN" button that calls /pin/create again. This keeps the security model simple and prevents replay attacks.

Node operator UX (optional context)

When the user enters your PIN in their ZiB Node app, the app shows a capability checklist comparing what your pipeline needs (encode / transcribe / vision) against what their node can do locally. Your tenant_settings.compute_mode setting controls what happens when their node can't do everything:

publicDefault. Any capable node on the network can run downstream jobs. Your user&apos;s node uploads, the network handles the rest.
prioritizePrefer your own user&apos;s nodes for compute, fall back to others if needed. The redeeming node will see a soft warning.
strictOnly your own user&apos;s nodes can run compute. If a third-party node redeems the PIN, the upload still completes but the encoding job will queue until one of your own nodes is online. The node UI shows an explicit warning.
Webhooks are unchanged. Whether the file arrived via browser upload or PIN upload, the same file.ready, encoding.queued, etc. webhooks fire on your existing webhook URL with identical payloads. Your webhook handler doesn't need to know which path was used. Route by file_id as always.

Video Encoding

After uploading a video file, call startEncoding(fileId) to trigger transcoding. ZiB produces an adaptive HLS stream with multiple quality variants, encrypts every segment, and distributes the segments redundantly across the storage network.

Encoding pipeline

1Transcode

Quality variants are produced: 360p, 480p, 720p, 1080p, and 2160p where source resolution allows. Each variant is a separate HLS stream.

2Encrypt segments

Every segment is encrypted before it leaves the encoding node, using a unique key generated per encoding job and held only by the ZiB backend.

3Distribute

Each encrypted segment is broken into redundant fragments and distributed across the storage network — the same redundancy model used for all ZiB files.

4Manifest delivery

The CDN serves standard HLS manifests with the encryption key URL embedded. The player fetches the key once, decrypts segments locally, and streams from the storage network directly.

Playback

The output is a standard HLS stream that plays in any HLS-capable player (hls.js, Video.js, native Safari, AVPlayer, etc.). The CDN serves the master manifest and per-quality variant playlists at predictable URLs — the player handles encryption and adaptive bitrate switching automatically.

Triggering encoding

javascript
// After uploadDirect() completes, pass file_id to startEncoding
const job = await storage.startEncoding(result.file_id);
console.log(job.encoding_id); // use this to poll status

// Poll until complete
const status = await storage.getEncodingStatus(job.encoding_id);
// status.hls_manifest_url — AES-128 encrypted HLS, plays in any HLS player
Listen for the encoding.complete webhook event instead of polling — it fires as soon as the HLS manifest is ready and includes the manifest URL in the payload.

Tracking encoding progress

ZiB fires encoding.progress webhook events throughout an encoding job so you can show a real percentage in your UI. Events are throttled to at most once every 10 seconds or every 5% change, whichever comes first.

The payload carries a progress integer (0–100) and a stage field ("encoding" or "sharding") so you can optionally show which phase the job is in. The percentage is continuous across both phases — you don't need to track them separately.

javascript
// Webhook handler — store the progress value, render in your UI
app.post('/api/zib-webhook', (req, res) => {
  const { event, file_id, progress, stage } = req.body;

  if (event === 'encoding.progress') {
    // progress is an integer 0–100
    // stage is "encoding" or "sharding"
    await db.update('videos', { encoding_progress: progress }, { where: { file_id } });
  }

  if (event === 'encoding.complete') {
    // Final state — manifest URLs are now live
    await db.update('videos', {
      encoding_progress: 100,
      encoding_status: 'complete',
      hls_url: req.body.hls_manifest_url,
    }, { where: { file_id } });
  }

  res.sendStatus(200);
});
Reset on re-upload — and the file_id has changed. A new upload to a (bucket, object_key) you've seen before produces a new file_id. The file.ready webhook arrives carrying the new one. Your stored zib_file_id, zib_encoding_id, CDN URL, and any progress state from the previous upload are all stale and must be replaced — not just progress reset to 0. ZiB cancels the prior encoding job but does not auto-start a new one; your handler must call /v1/api/encoding/request for the new file_id, exactly as it did for the first upload. Pattern A below shows the correct shape.

Edge cases

  • Out-of-order delivery — under retry pressure, a lower percent may arrive after a higher one. Use Math.max(current, incoming) if you want monotonic progress.
  • Fast encodes — small files or hardware-accelerated encodes may jump from 0 to 100 in a single event. The bar fills instantly — this is expected.
  • No events arrive — if all quality variants finish near-simultaneously, you may only see the final encoding.complete. Your UI should still work without progress events.
  • Sharding stage — if you want to show a separate phase label like "Distributing segments…", branch on stage === 'sharding'. Otherwise just use the percentage as-is.

Polling encoding status correctly

If you can't accept webhooks (or want a fallback alongside them), poll GET /v1/api/pipeline/<file_id> or GET /v1/api/encoding/jobs/<encoding_id> with a small interval (2–5 seconds). Four things to get right:

  1. Null-guard the response. Between the moment your /v1/api/encoding/request returns and the moment the encoding job row materialises (typically <1 s), the poll endpoint can return an empty body. Use result?.phase, not result.phase — otherwise your first poll throws a TypeError.
  2. Null-guard the closure inputs, too. If your interval/setTimeout captures variables that may flip to null (React state on rapid navigation, a cleared store), check them on every tick — not just at setup. A common shape: if (!content?.id) return; as the first line of the interval callback. A closure that was alive when the fetch was scheduled can fire after the component unmounts or refetches, with stale references that have since been cleared. If you have more than one poller in the same file, make sure every one has the guard — it's easy to add it to the obvious one and miss a sibling that takes the same code path.
  3. Stop on terminal phase. Three phases are terminal and you must break out of the loop when you see one:
    • phase === 'safe' — success. HLS manifest is live and replication-safe (≥3 peers). Surface the playback URL.
    • phase === 'failed_user_action' — terminal failure after retry budget exhausted. Show a Retry button; transient phase === 'failed' values are retry-in-progress, not terminal.
    • phase === 'cancelled' — superseded by a re-upload to the same (bucket, object_key). Switch your UI to the new file_id.
  4. Use phase, not status. phase is the canonical lifecycle state. status is a back-compat string the backend derives from phase via a trigger; reading it works but ties your code to legacy semantics that may evolve.
  5. Backoff sensibly. 2–5 second poll intervals are plenty. Encoding takes seconds to minutes; tighter polling just burns your rate-limit budget without changing the outcome.
javascript
async function waitForEncode(fileId, { timeoutMs = 10 * 60_000, intervalMs = 3_000 } = {}) {
  const deadline = Date.now() + timeoutMs;

  while (Date.now() < deadline) {
    const res = await fetch(`https://api.zibnetwork.com/v1/api/pipeline/${fileId}`, {
      headers: { Authorization: `Bearer ${process.env.ZIB_ACCESS_KEY}:${process.env.ZIB_SECRET_KEY}` },
    });
    if (!res.ok) {
      // Transient network / backend error — log and retry next tick.
      await sleep(intervalMs);
      continue;
    }
    const data = await res.json();

    // 1. Null-guard: data or stages.encoding might be missing during the
    //    brief window between request and job-row creation.
    const phase = data?.stages?.encoding?.phase ?? data?.phase;
    if (!phase) {
      await sleep(intervalMs);
      continue;
    }

    // 2. Terminal: success.
    if (phase === 'safe') {
      return { ok: true, manifestUrl: data.cdn_url };
    }

    // 3. Terminal: hard failure (retry budget exhausted).
    if (phase === 'failed_user_action') {
      return { ok: false, error: data.error ?? 'Encoding failed after retries' };
    }

    // 4. Terminal: superseded by a re-upload.
    if (phase === 'cancelled') {
      return { ok: false, error: 'Upload superseded; check for the new file_id' };
    }

    // Anything else (queued, encoding, sharding, failed-transient,
    // local_complete, replicating) — keep polling.
    await sleep(intervalMs);
  }
  return { ok: false, error: 'Timed out waiting for encode' };
}

const sleep = ms => new Promise(r => setTimeout(r, ms));

Webhooks are still the preferred path — your server doesn't pay for the polling round-trips and you react instantly when the state changes. Polling is the right tool when your environment can't accept inbound HTTPS (a background worker, a CLI script, a deployment behind a firewall without an ingress).

Playing the stream

html
<!-- hls.js (browser) -->
<script src="https://cdn.jsdelivr.net/npm/hls.js@latest"></script>
<video id="video" controls></video>
<script>
  const hls = new Hls();
  hls.loadSource('https://cdn.zibnetwork.com/stream/<file_id>/master.m3u8');
  hls.attachMedia(document.getElementById('video'));
</script>

<!-- Native HLS (Safari / iOS) -->
<video controls src="https://cdn.zibnetwork.com/stream/<file_id>/master.m3u8"></video>

Re-encoding the same file

POST /v1/api/encoding/request is idempotent. If you request encoding for a file_id that already has a completed encoding job whose segments are live on the CDN, ZiB will not start a second encode — you get back the existing job's ID and CDN URL so your code can carry on as if it had just queued one:

json
// Response when the file is already encoded
{
  "encoding_id": "771a1e5e-5e29-470a-9f1f-5ad8b999efc4",
  "status": "already_encoded",
  "file_id": "01846192-315c-4f0a-b766-12b18a26b209",
  "cdn_url": "https://cdn.zibnetwork.com/stream/01846192-315c-4f0a-b766-12b18a26b209/master.m3u8",
  "message": "File is already encoded by an existing job. Pass { \"force\": true } in the request body to deliberately re-encode."
}

The same shape applies while a job is mid-flight: you get status: "already_queued" with the in-flight job's ID. Branch on status to decide whether to poll progress, fire your own "already done" UI, or just record the encoding_id and move on.

Forcing a re-encode

When you genuinely want a fresh encode of the same source bytes (changed encoding presets, repaired segments, etc.) pass force: true in the request body:

javascript
await fetch('https://api.zibnetwork.com/v1/api/encoding/request', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: `Bearer ${ZIB_ACCESS_KEY}:${ZIB_SECRET_KEY}`,
  },
  body: JSON.stringify({
    object_key: fileId,        // existing file_id
    force: true,               // override the idempotency gate
    webhook_url: WEBHOOK_URL,  // optional
  }),
});
The CDN URL never changes. A re-encode produces new segments and a new HLS key under the hood, but cdn.zibnetwork.com/stream/{file_id}/master.m3u8 continues to serve playable content throughout — the old encode keeps serving until the new one replaces it. You do not need to swap URLs in your app. Listen for encoding.complete on the new encoding_id to know the new encode is live.

Why this matters (the bug we closed)

Before 2026-05-12 the endpoint would happily create a new encoding job every time it was called, even when the file was already encoded. Every job has its own random AES-128 key for HLS segment encryption, so a file with N completed jobs ended up with N different keys floating in the database. The CDN's key-lookup picked one of them arbitrarily and the wrong key was sometimes served — which lenient players (hls.js, Bitmovin, Video.js) silently retried or skipped past, but Safari native HLS / AVPlayer correctly rejected with a black screen and no error.

Two fixes shipped together: (1) the CDN now always picks the key whose job has live segments, so the bad state cannot produce a wrong-key symptom even when it accidentally happens, and (2) this endpoint refuses to create the bad state in the first place unless you pass force: true.

Tenant-side hygiene we recommend

  • Treat /v1/api/encoding/request as idempotent — calling it twice for the same file_id is safe and cheap.
  • Don't add your own retry loop on top. If the response is already_queued or already_encoded, store the returned encoding_id and rely on webhooks / status polling.
  • Guard your tenant-side "Re-encode" UI behind auth and a server-side dedup check on the in-flight job. Without it, a fast double-click or a curl loop will fire many requests; ZiB will collapse them server-side but you'll waste roundtrips and confuse your own UI.
  • The file.ready webhook fires exactly once per upload today. If we ever add at-least-once retry semantics, your handler should already be idempotent: check whether you've already kicked off encoding for that file_id before calling startEncoding.

Thumbnails

Every encode produces a poster frame and a WebVTT-driven sprite sheet for scrub-bar previews. Default poster is grabbed at 10% of duration (clamped to 1–10s) so cold-opens and fade-ins don't become the cover image. Default sprite cadence is one tile every 5 seconds at 160px wide, packed into 10×10 grid JPEGs — the WebVTT references each tile with #xywh=x,y,w,h so Bitmovin, hls.js, Video.js and shaka-player pick it up natively. Defaults are ON; pass thumbnails: false to opt out.

javascript
// All four shapes are valid on POST /v1/api/encoding/request:

// 1. Omit the field — thumbnails generated with defaults.
{ object_key: fileId }

// 2. Explicit defaults.
{ object_key: fileId, thumbnails: true }

// 3. Opt out entirely.
{ object_key: fileId, thumbnails: false }

// 4. Override individual fields (any subset; missing keys fall back to defaults).
{
  object_key: fileId,
  thumbnails: {
    poster: { at: "10%" },            // or "5s" / "Ns" for an absolute timestamp
    sprite: { interval: 5, width: 160 }
  }
}

When thumbnails are produced, the thumbnails object is surfaced in three places — identical shape on each, so tenants who can't receive webhooks (localhost dev, replayed deliveries, missed events) can still pick the URLs up by polling:

  • The encoding.complete webhook payload (top-level field).
  • GET /v1/api/pipeline/{file_id} stages.encoding.thumbnails.
  • GET /v1/api/encoding/jobs/{encoding_id} thumbnails (top-level).

The field is omitted (or null) until the encoder has registered the artifacts — typically a few seconds after phase reaches local_complete. Tenants who opt out (thumbnails: false) will see it stay null forever; that's the contract.

json
// thumbnails block — same shape on webhook + both polling endpoints.
// Omitted (or null) when the tenant opted out or before registration completes.
{
  "thumbnails": {
    "poster_url":     "https://cdn.zibnetwork.com/objects/<poster_file_id>",
    "sprite_url":     "https://cdn.zibnetwork.com/objects/<first_sprite_sheet_file_id>",
    "sprite_vtt_url": "https://cdn.zibnetwork.com/objects/<sprite_vtt_file_id>"
  }
}

Use poster_url directly as the <video poster="..."> attribute or as the cover image in any thumbnail grid. The sprite-sheet VTT plugs into the major HLS players verbatim:

javascript
// Bitmovin
player.load({
  hls: hlsManifestUrl,
  poster: thumbnails.poster_url,
  tracks: {
    thumbnailTrack: {
      file: thumbnails.sprite_vtt_url
    }
  }
});

// hls.js — load the VTT yourself and feed it to your seekbar component:
const vtt = await fetch(thumbnails.sprite_vtt_url).then(r => r.text());
// Each cue body is "<sprite_sheet_url>#xywh=x,y,w,h" — render that tile into your scrubber.

// Video.js (vtt-thumbnails plugin)
player.vttThumbnails({ src: thumbnails.sprite_vtt_url });
Idempotency and re-encodes. Like the rest of the encoding pipeline, thumbnails are bound to the live encoding job. Calling POST /v1/api/encoding/request a second time for a file that already has a complete encode returns already_encoded and does not regenerate thumbnails — the existing CDN URLs keep serving. To regenerate (e.g. after changing the poster timestamp), pass force: true alongside your new thumbnails config.

When to opt out

Defaults are tuned for long-form, scrubbable video — the kind of content where a viewer hovers the timeline to preview frames. If your pipeline doesn't render a seek bar or doesn't need a cover image, opt out and save the ~1-2 s of encoder wall time plus the storage:

  • Ad creatives, pre-rolls, short transactional clips — no scrub UI, host system usually already has its own preview thumbnail. Opt out with thumbnails: false.
  • Audio-only or visualizer content — sprite-sheet previews of a static waveform aren't useful. Opt out.
  • Long-form video / VOD, courses, tutorials — leave the defaults on. The sprite VTT feeds Bitmovin / hls.js / Video.js scrub-bar previews with zero extra integration code.
  • UGC where you display a cover image but don't scrub — current API is all-or-nothing per encode; opt out with thumbnails: false and reuse whatever cover image your upload flow already produces. (We can add a poster-only mode if enough integrations need it — open a ticket.)

Aspect variants

Generate a vertical 9:16 HLS rendition alongside the main horizontal encode — for Shorts shelves, TikTok-style vertical feeds, iPhone portrait playback. Each rendition is a full multi-bitrate HLS ladder with its own poster, content-encrypted with a per-rendition AES-128 key, and served from its own CDN stream URL. Defaults are OFF; opt in by passing aspect_variants: ["9:16"]. v1 accepts only 9:16; the field is an array so adding 4:5 / 1:1 / 4:3 later is a non-breaking extension.

The rendition is produced with a center crop (no smart-crop in v1) and an adaptive ladder derived from your source — rungs whose bitrate would exceed the source's are dropped automatically, so a 720p source produces a 720p-and-below vertical ladder, not an upscaled 1080p one.

javascript
// 1. Inline mode — declared at upload time, runs alongside the main encode.
//    Add to the same `pipeline` block you pass to /v1/api/upload/initiate.
{
  bucket: 'video',
  key: 'videos/abc.mp4',
  file_size: 1234567,
  pipeline: {
    encode: true,
    aspect_variants: ['9:16'],          // ← new
    webhook_url: 'https://...',
    webhook_secret: process.env.WEBHOOK_SECRET,
  }
}

// 2. Inline mode via /v1/api/encoding/request (same root-level field).
{ object_key: fileId, aspect_variants: ['9:16'] }

// 3. Backfill mode — produce variants for a file that already has a complete
//    horizontal encode, WITHOUT re-running the main pass. Cheaper than force:true.
{ object_key: fileId, aspect_variants_only: ['9:16'] }
//
// Response: { status: 'queued', mode: 'aspect_variants_only',
//             parent_encoding_job_id: '...', child_encoding_job_ids: ['...'] }
//
// Mutually exclusive with force:true and with aspect_variants on the same call.

Once each variant is registered, the aspect_variants object is surfaced in the same three places as thumbnails — keyed by ratio string:

  • The encoding.complete webhook payload (top-level field).
  • GET /v1/api/pipeline/{file_id} stages.encoding.aspect_variants.
  • GET /v1/api/encoding/jobs/{encoding_id} aspect_variants (top-level).

Each ratio entry is omitted from the object entirely until the encoder has registered it (typically a few seconds after the main encode reaches local_complete). If a variant encode fails the ratio is simply absent from the response; the parent encode still completes. The entire aspect_variants field is omitted when the tenant didn't opt in.

json
// aspect_variants block — same shape on webhook + both polling endpoints.
{
  "aspect_variants": {
    "9:16": {
      "hls_manifest_url": "https://cdn.zibnetwork.com/stream/<vertical_file_id>/master.m3u8",
      "poster_url":       "https://cdn.zibnetwork.com/objects/<vertical_poster_id>"
    }
  }
}

The hls_manifest_url is a standard HLS stream — point any player at it the same way you would the horizontal manifest. The vertical rendition has its own file_id and its own AES-128 key (served from /stream/<vertical_file_id>/key), so it's independently routable, cacheable, and replaceable without touching the main encode.

javascript
// Bitmovin — switch between horizontal and vertical based on viewport
const horizontal = encodingResult.hls_manifest_url;
const vertical   = encodingResult.aspect_variants?.['9:16']?.hls_manifest_url;
const verticalPoster = encodingResult.aspect_variants?.['9:16']?.poster_url;

player.load({
  hls: window.innerHeight > window.innerWidth ? (vertical || horizontal) : horizontal,
  poster: window.innerHeight > window.innerWidth
    ? (verticalPoster || encodingResult.thumbnails?.poster_url)
    : encodingResult.thumbnails?.poster_url,
});

// React Native / iOS-native — pick the vertical rendition directly for a portrait feed.
const shortsUrl = videoRow.aspect_variants?.['9:16']?.hls_manifest_url;
if (shortsUrl) {
  // Render a vertical scrolling feed using AVPlayer / ExoPlayer.
}
Backfill an existing library. Calling POST /v1/api/encoding/request with aspect_variants_only: ["9:16"] against a file that's already encoded queues the variant on top of the existing parent — no force:true, no main re-encode. Ideal for adding a Shorts shelf to a library of horizontal videos. The response gives you the parent_encoding_job_id + a list of child_encoding_job_ids, one per requested ratio; webhook / polling surfaces the URLs the moment each child finishes.

When to opt in

  • Shorts shelf / vertical feed on mobile — opt in. Native portrait playback without forcing the player to letterbox / pillarbox horizontal content.
  • Trailers, hero videos, social-share clips — opt in if you embed those on a vertical scrolling surface.
  • Long-form VOD where the viewer is always landscape — leave it off. Vertical rendition costs encoder time and storage; horizontal already covers the use case.
  • Ad creatives, pre-rolls, square / portrait sources — usually unnecessary. If the source is already vertical or square, the horizontal HLS is the only rendition you need.
v1 limits. Center-crop only (smart-crop deferred). The crop is taken from the source center on every frame — if the focal subject is hard-pinned to the left or right edge, parts of it can be lost on a vertical view. For most trailer / hero / social content the center is the right answer. If smart-crop becomes a hard requirement, open a ticket so we can prioritise.

Pattern A — webhook-triggered encode

For the standard tenant upload flow, you typically kick off encoding the moment ZiB delivers file.ready. The shape below is what we recommend for any handler that receives file.ready and decides to start an encode. The idempotency check at step 3 is the most important part — duplicate webhook deliveries should be cheap no-ops, not double-fires.

typescript
// app/api/zib-file-ready/route.ts (Next.js App Router style)

export async function POST(req: Request) {
  const event = await req.json();

  // 1. Verify the webhook signature (mandatory — see Webhooks).
  if (!verifyZibSignature(req.headers.get('x-zib-signature'), event)) {
    return new Response('invalid signature', { status: 401 });
  }

  if (event.event !== 'file.ready') {
    return Response.json({ received: true, ignored: true });
  }
  const { file_id, bucket, object_key } = event;

  // 2. Look up your local row by the STABLE identifier — (bucket, object_key).
  //
  //    Do NOT look up by file_id. A re-upload to the same key produces a
  //    NEW file_id, so the row you stored against the prior file_id will
  //    not match and your handler will silently no-op while the user's
  //    UI sits at "Queued — waiting for an encoder" forever.
  const creative = await db.creatives.findUnique({
    where: { bucket_object_key: { bucket, object_key } },
  });
  if (!creative) return Response.json({ received: true, matched: false });

  // 3. Idempotency keyed on file_id — NOT on encoding_status.
  //
  //    - If creative.zib_file_id === event.file_id and we already stored an
  //      encoding_id for it, this is a duplicate webhook delivery → skip.
  //    - If they differ, the user re-uploaded. The stored encoding belongs
  //      to the OLD file and is now cancelled by ZiB; we must fire a fresh
  //      encoding request for the NEW file_id.
  //
  //    A common bug — and the one that causes "Queued forever" deadlocks —
  //    is keying this check on `encoding_status !== 'ERROR'`. That returns
  //    "already handled" for every re-upload whose previous encode succeeded,
  //    so the new upload is never encoded. The user sees their re-upload
  //    "queued" indefinitely and ZiB never receives an encoding request.
  if (creative.zib_file_id === file_id && creative.zib_encoding_id) {
    return Response.json({ received: true, matched: true, deduped: true });
  }

  // 4. Fire encoding request. ZiB is also idempotent server-side — if
  //    another code path already fired for this file_id you'll receive
  //    status: 'already_queued' (in-flight job exists) or
  //    status: 'already_encoded' (completed job exists). Treat both the
  //    same — record the encoding_id and trust webhooks for the rest.
  const res = await fetch('https://api.zibnetwork.com/v1/api/encoding/request', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${process.env.ZIB_ACCESS_KEY}:${process.env.ZIB_SECRET_KEY}`,
    },
    body: JSON.stringify({
      object_key,                 // stable key — NOT file_id
      bucket,
      // webhook_url and webhook_secret are optional here — if you've
      // configured them once on your customer profile, ZiB uses those.
    }),
  });
  if (!res.ok) {
    return new Response(`ZiB encode request failed: ${res.status}`, { status: 502 });
  }
  const job = await res.json();

  // 5. Persist the new file_id AND the new encoding_id together. Writing
  //    file_id is the critical step that lets future webhook deliveries
  //    recognise this row as up to date — without it, the next file.ready
  //    for this key will look like another re-upload and you'll loop.
  await db.creatives.update({
    where: { id: creative.id },
    data: {
      zib_file_id:     file_id,                  // overwrite prior file_id
      zib_encoding_id: job.encoding_id,
      cdn_url:         job.cdn_url ?? job.hls_manifest_url ?? null,
      encoding_status:
        job.status === 'already_encoded' ? 'COMPLETE' : 'ENCODING',
    },
  });

  return Response.json({
    received: true,
    matched: true,
    encoding_id: job.encoding_id,
    status: job.status,
  });
}

Pattern B — user-triggered re-encode

For a "Re-encode" button in your own UI (encoder settings changed, source corrupted, etc.), the call shape is the same plus force: true. The three things you MUST add on your side — none of which ZiB can enforce for you — are: auth on the endpoint, authorization (the caller owns the content), and a server-side dedup check against your own DB. The UI's in-flight lock is not enough; a page reload or a second tab bypasses it.

typescript
// app/api/re-encode/route.ts

import { getServerSession } from '@/lib/auth';

export async function POST(req: Request) {
  // 1. Authentication. Refuse anonymous callers.
  const session = await getServerSession(req);
  if (!session?.user) {
    return new Response('unauthenticated', { status: 401 });
  }

  const { content_id, video_type } = await req.json();
  //    video_type: 'main' | 'trailer'

  // 2. Authorization. Caller must own this content (or be admin).
  const content = await db.content.findUnique({ where: { id: content_id } });
  if (!content) return new Response('not found', { status: 404 });
  if (content.owner_id !== session.user.id && !session.user.isAdmin) {
    return new Response('forbidden', { status: 403 });
  }

  const statusKey = `${video_type}_encoding_status`;
  const idKey     = `${video_type}_zib_encoding_id`;
  const fileKey   = `${video_type}_source_file_id`;

  // 3. Server-side dedup. Refuse if an encode is already in flight for
  //    this content+video_type. The UI lock isn't enough — page reload
  //    clears it, a second tab bypasses it, a scripted POST loop wouldn't
  //    even render the UI.
  if (['QUEUED', 'ENCODING'].includes(content[statusKey])) {
    return Response.json(
      { error: 'already_in_progress', encoding_id: content[idKey] },
      { status: 409 },
    );
  }

  // 4. Fire ZiB with force=true. The old encoded segments keep serving
  //    the CDN URL until the new encode completes, so playback isn't
  //    interrupted. Do NOT delete the prior encoded data first — that
  //    creates a window where the file has no playable output if the
  //    new encode never starts (e.g. ZiB returns an error). Let ZiB
  //    do the cutover atomically once segments land.
  const res = await fetch('https://api.zibnetwork.com/v1/api/encoding/request', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${process.env.ZIB_ACCESS_KEY}:${process.env.ZIB_SECRET_KEY}`,
    },
    body: JSON.stringify({
      object_key: content[fileKey],
      force: true,
    }),
  });
  if (!res.ok) {
    return new Response(`ZiB encode request failed: ${res.status}`, { status: 502 });
  }
  const job = await res.json();

  // 5. Record new encoding_id. The cdn_url field on your record does
  //    NOT change — the CDN URL is keyed by file_id, not encoding_id.
  await db.content.update({
    where: { id: content_id },
    data: {
      [idKey]: job.encoding_id,
      [statusKey]: 'ENCODING',
    },
  });

  return Response.json({
    encoding_id: job.encoding_id,
    status: job.status,
  });
}
What NOT to do. Don't delete the existing encoded record on your side before firing the re-encode (purge-then-encode). If the encode POST fails after the purge, you've wiped playable content with no replacement. The ZiB-side guarantee is that cdn.zibnetwork.com/stream/{file_id}/master.m3u8 stays playable across re-encodes — let the cutover happen there.

Quick self-check before shipping

  • Your re-encode endpoint requires auth — anonymous curl against it returns 401.
  • Your re-encode endpoint refuses with 409 if your own DB says encoding is in progress for that content.
  • Your file.ready handler looks up rows by (bucket, object_key), not by file_id. file_id changes on every re-upload; the key does not.
  • Your file.ready handler treats a webhook whose file_id differs from your stored zib_file_id as a re-upload, fires a fresh encoding request, and writes the new file_id back to your row. Idempotency is keyed on file_id, never on encoding_status.
  • Test this end-to-end: upload v1 → encoded → re-upload v2 to the same key → your row points at v2's file_id, v2's encoding_id, v2's CDN URL, and the user can play v2 — without you touching anything by hand.
  • You never delete the prior encoded output on your side before requesting a new encode.
  • You handle { status: 'already_queued' } and { status: 'already_encoded' } the same as a fresh queue: record the encoding_id and move on.

Why Pattern A is so emphatic about file_id comparison

2026-05-13 production deadlock: a tenant's zib-creative-ready handler keyed idempotency on encoding_status !== 'ERROR'. The first upload of a creative encoded fine and the row settled at encoding_status = FINISHED. The user then re-uploaded the same creative via the dashboard. ZiB cancelled the prior encoding row, accepted the new file, and fired file.ready with the new file_id.

The handler ACK'd 200, saw encoding_status === 'FINISHED', decided "already handled", and returned. No /v1/api/encoding/requestwas ever fired for the new file_id. The dashboard UI optimistically set its own local status to Queued — waiting for an encoder. ZiB had no encoding job for the new file and no signal to start one. Both sides waited on the other indefinitely; the user's screen stayed at "Queued" with no error.

Fix shape: idempotency check compares creative.zib_file_id to event.file_id; mismatch means re-upload; re-upload means fire encoding for the new file_id and persist it. Status fields are derived state and unsafe as idempotency keys whenever the entity they describe can be replaced.

Viewer Telemetry

Opt-in browser feature for video tenants. The SDK observes HLS segment fetches via the PerformanceObserver API, batches per-segment timings (TTFB, total duration, transfer size), and posts them to ZiB. The backend fuses them with synthetic and peer-node probes into a per-(node, viewer-region) fitness score that quarantines slow nodes from the segment-holder picker — so a CF colo that's silently routing slowly to one part of the world stops being handed to viewers there.

Optional but recommended for any tenant streaming video. Without RUM, ZiB can only see the network from its own backend vantage — that misses the entire class of geo-specific edge degradation that motivated this system. One line of integration, no UI surface, no PII collected.

What gets sent

  • node_id — the ZB-XXXXXX ID parsed from each segment URL hostname (zb-*.zibnetwork.com)
  • ttfb_ms, total_ms, size_bytes from the browser's resource timing entry
  • An ephemeral random viewer_session_id (UUID) for throttling — never tied to a wallet, account, or persistent identifier
  • The viewer's CF colo (e.g. SYD, LHR) inferred server-side from CF-Ray

No URLs, no IP, no user agent, no cookies. Data is pruned after 24 hours and never leaves ZiB's own infrastructure.

SDK 1.4.0 — three observation channels. The SDK uses whichever channels your stack makes available: PerformanceObserver (passive, free, catches main-thread fetches), recordFetch() (explicit per-segment hook for players that fetch in a Web Worker — bitmovin, modern hls.js, shaka, dash.js — where the observer is structurally blind), and attachToVideo() (universal element-level fallback that works even on Safari native HLS). For commercial players, recordFetch() is the primary integration.

Quickest integration — one line

If your player is a vanilla <video src=".m3u8"> or simple hls.js setup with workers disabled, just call the static helper once on page load. Telemetry observes Resource Timing in the background and flushes on pagehide.

html
<script src="https://app.zibnetwork.com/sdk/zib-sdk.js"></script>
<script>
  // Anywhere on a watch page, after the SDK script loads.
  // Idempotent — calling twice is harmless.
  ZiBStorage.startTelemetry();
</script>
Most modern players need an explicit hook. Bitmovin, hls.js with enableWorker: true (the default in recent versions), shaka, and dash.js all fetch HLS segments inside a Web Worker. Worker fetches do not appear in main-thread PerformanceObserver, so the quick-start above produces zero data with these players. Add ZiBStorage.recordFetch() on your player's segment-loaded event (next subsection) — it's a 5-line addition.

recordFetch() — the explicit per-segment API

Hook your player's per-segment download event and pass the URL + timing to ZiBStorage.recordFetch(). The SDK parses the node id from the URL, batches the sample, and ships it through the same flush pipeline. No prior startTelemetry() call required — the first recordFetch() lazily initialises the singleton.

javascript
// === bitmovin player (8.x — verified against 8.241.0) ===
// IMPORTANT: use SegmentRequestFinished, not SegmentPlayback.
//   - SegmentPlayback fires when a segment starts playing (no timing data)
//   - SegmentRequestFinished fires after download with downloadTime,
//     timeToFirstByte, size, success, httpStatus, url
//
// SegmentRequestFailed does NOT exist in 8.x — failures arrive on
// Finished with event.success === false. httpStatus === 0 means network
// error / CORS reject / abort.
//
// Time units: bitmovin's TypeScript type doc says seconds for downloadTime
// and timeToFirstByte. Multiply by 1000 to get ms. Verify against the DB
// once samples flow — if avg ttfb_ms ends up in the 50,000+ range, the
// runtime actually emitted ms and the multiplier should be removed.
player.on(bitmovin.player.PlayerEvent.SegmentRequestFinished, (event) => {
  // Clamp the -1 "cancelled before headers" sentinel.
  const ttfbSec = Math.max(0, event.timeToFirstByte || 0);
  ZiBStorage.recordFetch({
    url: event.url,
    ttfb_ms: Math.round(ttfbSec * 1000),
    total_ms: Math.round((event.downloadTime || 0) * 1000),
    size_bytes: event.size || 0,
    status: event.success === false
      ? (event.httpStatus === 0 ? 'network_error' : String(event.httpStatus))
      : (event.httpStatus ? String(event.httpStatus) : '200'),
  });
});

// === hls.js (any version) ===
hls.on(Hls.Events.FRAG_LOADED, (_, data) => {
  const stats = data.frag.stats || data.stats;  // hls.js v1 vs v0 paths
  ZiBStorage.recordFetch({
    url: data.frag.url,
    ttfb_ms: Math.round(stats.tfirst - stats.trequest),
    total_ms: Math.round(stats.tload - stats.trequest),
    size_bytes: stats.total,
    status: '200',
  });
});

// === shaka player ===
player.addEventListener('downloadcompleted', (event) => {
  ZiBStorage.recordFetch({
    url: event.request.uris[0],
    ttfb_ms: Math.round(event.response.headers['x-firstbyte-ms'] || event.response.timeMs),
    total_ms: Math.round(event.response.timeMs),
    size_bytes: event.response.data.byteLength,
    status: '200',
  });
});

Samples whose URL doesn't match zb-XXXXXX.zibnetwork.com are silently dropped — without a node identifier in the URL there is nothing to attribute. Pass node_id directly if you have it from another source. Errors are reported with status: 'timeout' or 'network_error'.

attachToVideo() — universal element-level fallback

For coverage that works regardless of player or fetching architecture — including Safari native HLS where neither PerformanceObserver nor a Worker hook is available — attach to the <video> element directly. The SDK reports stalls (waiting → playing transitions), frame drops (via getVideoPlaybackQuality()), and playback errors. File-correlated, not per-segment; the backend resolves to nodes via segment_holders at sample time.

javascript
// On player init, after the <video> element is in the DOM
const detach = ZiBStorage.attachToVideo(videoEl, { fileId: '<file-uuid>' });

// On player teardown
detach();
Wire both channels — they're not interchangeable. recordFetch() gives ZiB per-segment-per-node TTFB attribution (the signal Phase 2 routing acts on). attachToVideo() reports file-correlated stalls and frame drops (file-level signal that backstops Safari iOS whererecordFetch can't fire). Shipping only one leaves a structural gap:attachToVideo alone has no node attribution; recordFetch alone misses Safari iOS viewers and any playback issue that isn't a segment-fetch problem (audio glitches, decoder stalls, etc.). Both share the same singleton and flush in the same batch — total cost is two function calls on player init.

Transport & CORS — what the SDK sends, what the API accepts

The SDK posts samples to POST https://api.zibnetwork.com/v1/api/telemetry/segment-fetch on a 5-second flush timer and one final pagehide flush via navigator.sendBeacon(). The endpoint accepts both transports — the responses are equivalent. You don't need to configure anything for this to work; the notes below exist for tenants who are debugging from DevTools and want to know what to expect.

CORS shape

  • Access-Control-Allow-Origin echoes the request Origin (not *) so that navigator.sendBeacon() works — sendBeacon is credentialed by spec, and browsers reject wildcard origins on credentialed responses.
  • Access-Control-Allow-Credentials: true is set for the same reason. The endpoint itself is anonymous (no cookie or auth read); the header exists purely so the browser doesn't drop the response.
  • Vary: Origin is set on every response — CDNs caching this endpoint should respect it.
  • If you build a custom telemetry helper (instead of using the SDK) and see "Cannot use wildcard in Access-Control-Allow-Origin when credentials flag is true", you're hitting a non-telemetry endpoint by mistake — the global API CORS is wildcard. The telemetry endpoint has the per-route override.

Inside an existing SDK instance

If you already construct ZiBStorage on the player page, opt in via the constructor option. Same effect; no second include needed.

javascript
const zib = new ZiBStorage({
  accessKey: '...',
  secretKey: '...',
  telemetry: { enabled: true },   // <-- one extra option
});

// Optional — bring your own session ID if you already have one
// (e.g. tied to a playback session in your analytics).
new ZiBStorage({
  accessKey: '...',
  secretKey: '...',
  telemetry: { enabled: true, viewerSessionId: '...' },
});

React / Next.js — mount in the root layout

Mount it once at the root layout, not on the watch page. Tenants frequently play video on more surfaces than just /watch/<slug> — content-detail pages with trailers, search-result hover previews, embed routes, series episode views. If telemetry only mounts on a single route, every other player surface goes uninstrumented and the dashboard looks emptier than reality. The SDK is ~30 KB gzipped and startTelemetry() is a no-op on pages without zb-*.zibnetwork.com Resource Timing entries, so the cost on non-player routes is one cached <script> tag and a dormant PerformanceObserver. Mount high, forget about it.

Drop the SDK <Script> and a small client wrapper into your root app/layout.tsx — not a route-group layout like app/(app)/layout.tsx. Route-group layouts only cover routes inside that group; the root layout covers everything.

tsx
// app/layout.tsx — Server Component is fine
import Script from 'next/script';
import { ZiBTelemetry } from '@/components/ZiBTelemetry';

export default function RootLayout({ children }: { children: React.ReactNode }) {
  return (
    <html lang="en">
      <body>
        {/* Loads on every page; next/script dedupes by src so it's safe to
            keep any per-page <Script> tags in upload flows that need a hard
            guarantee the SDK is present. */}
        <Script src="https://app.zibnetwork.com/sdk/zib-sdk.js" strategy="afterInteractive" />
        <ZiBTelemetry />
        {children}
      </body>
    </html>
  );
}
tsx
// components/ZiBTelemetry.tsx — small client wrapper
'use client';
import { useEffect } from 'react';

export function ZiBTelemetry() {
  useEffect(() => {
    let tele: { stop: () => void } | undefined;
    let cancelled = false;

    // Poll for the SDK global — handles the race where <Script
    // strategy="afterInteractive"> hasn't finished loading yet.
    const tick = () => {
      if (cancelled) return;
      // @ts-ignore — global injected by the SDK <script>
      const ZiBStorage = (window as any).ZiBStorage;
      if (ZiBStorage?.startTelemetry) {
        tele = ZiBStorage.startTelemetry();
        return;
      }
      setTimeout(tick, 100);
    };
    tick();

    return () => {
      cancelled = true;
      tele?.stop();   // fires final navigator.sendBeacon flush
    };
  }, []);

  return null;
}

That's the entire integration for any Next.js App Router app. Pages Router is the same idea — drop the <Script> and <ZiBTelemetry /> into pages/_app.tsx instead.

Common placement traps

  • Route-group layout (app/(app)/layout.tsx) — covers only that group. Marketing/landing pages, content-detail pages, public preview surfaces typically live in a different group and would miss telemetry. Mount in the root layout instead.
  • Single watch route (app/watch/[slug]/page.tsx) — covers exactly one route. Trailers, embed players, hover previews on other routes go uninstrumented.
  • Edit-mode-only mount — gating the SDK <Script> behind an isEditMode flag (common in CMS-like rite3 setups) means viewers never load it. Either ungate, or mount in the root layout where the gate doesn't apply.

How to verify it's working

  1. Test on at least two different player surfaces — e.g. a watch page and a content-detail page that plays a trailer. If telemetry only fires on one, your mount is too narrow (route-group layout instead of root layout) — see the placement traps above.
  2. Open DevTools → Network and filter for zib-sdk.js. The script should load on every page where you expect telemetry, including non-player pages that share the same root layout.
  3. Switch the filter to telemetry and play a video for ~30 s. You should see a POST to /v1/api/telemetry/segment-fetch every 30 s or every 10 segments, returning { accepted: <n>, rejected: 0 }. Confirm the segments themselves are coming from zb-*.zibnetwork.com hostnames in the Domain column — that's the URL pattern the SDK matches.
  4. Close the tab. The SDK fires a final navigator.sendBeacon on pagehide; that flush still shows up in DevTools' Network panel under Show navigation requests.
  5. If you have admin access, your samples land in /superadmin/network within ~60 s, tagged with the appropriate CF colo region (SYD, LHR, etc.) rather than the synthetic-probe sentinel _backend.

Diagnose by response shape

The /v1/api/telemetry/segment-fetch response shape tells you exactly which channels are wired and which are missing. Click any telemetry POST in DevTools → Response and read both fields:

ResponseDiagnosisFix
{ accepted: N, ... }
{ playback_events_accepted: M }
both > 0
Both channels wired. Full coverage.Done. ✅
{ accepted: N }
(no playback_events)
recordFetch wired; attachToVideo not called.Add ZiBStorage.attachToVideo(videoEl, { fileId }) on player init. Backstops Safari iOS where worker fetches don't fire.
{ accepted: 0,
playback_events_accepted: M }
(M > 0)
attachToVideo wired; recordFetch not called. Most common gotcha for commercial-player tenants.Add the player's segment-loaded event hook (bitmovin SegmentPlayback, hls.js FRAG_LOADED, etc.) calling ZiBStorage.recordFetch(). Without this you have no per-node TTFB attribution.
{ accepted: 0, rejected: N }
(N > 0)
recordFetch wired but the URL pattern doesn't match zb-XXXXXX.zibnetwork.com.Log event.segment.uri from your hook. If you see cdn.* instead of zb-* hosts, the player is using CDN-cached URLs — pass node_id directly from another source (e.g. holder lookup), or contact ZiB to widen the regex.
No POST at allSDK didn't load, or the hook isn't on the active route, or the player isn't actually playing.Console: typeof ZiBStorage should be 'function'. If 'undefined', the <Script> tag isn't on this route — see "Common placement traps" above.

The third row — accepted: 0, playback_events_accepted: N — is by far the most common partial-integration state. attachToVideo is a 2-line addition while the player-event hook needs the integrator to know their player's event API; tenants tend to ship the easy one first and forget the other. Both are needed for full coverage.

SDK matches direct-to-node URLs only. Modern HLS AES-128 content streams segments direct from zb-XXXXXX.zibnetwork.com/<hash> — that's what the SDK observes. Legacy content that pre-dates HLS AES-128 still streams via the CDN proxy (cdn.zibnetwork.com/stream/<file_id>/segments/...); those fetches don't carry a node identifier in the URL and aren't observed. If all your video is legacy, you'll see telemetry POSTs reporting { accepted: 0 } — re-encode a piece of content via the modern pipeline to verify, or check the Domaincolumn in DevTools to confirm at least some segments are zb-*.zibnetwork.com.
Phase 1 is detection only. Right now telemetry feeds the dashboard only — segment-holder selection in the CDN layer hasn't been changed yet. Phase 2 (next release) wires the fitness score into routing so a degraded node is automatically deprioritised for the affected viewer regions.

Webhooks

ZiB sends signed POST requests to your configured webhook URL when files, encoding jobs, and AI jobs change state.

Configuration patterns

There are three valid ways to consume ZiB lifecycle events. Pick the one that matches how your integration is operated:

Pattern 1 — Per-requestrecommended for engineering teams

Send webhook_url and webhook_secret in the body of every POST /v1/api/encoding/request (or inside the pipeline field of /v1/api/upload/initiate). The URL and secret travel with the request in code — survives any tenant-level config drift, no ops dependency, easy to rotate per environment.

Pattern 2 — Tenant-level defaultfor operator-managed products

Configure once via the Webhooks panel on the Storage page, or programmatically via PATCH /v1/api/customers/<customer_id>/webhook + POST /v1/api/customers/<customer_id>/webhook/secret. Every request without a per-request override inherits these defaults. Simpler when one URL serves the whole tenant; vulnerable to silent breakage if an operator clears the URL or rotates the secret without notifying the receiving service.

Pattern 3 — Polling onlywhen you can't host inbound HTTPS

Skip webhooks entirely; poll GET /v1/api/pipeline/<file_id> until you see a terminal phase. See Video Encoding → Polling encoding status correctly for the correct loop shape. Valid for CLI scripts, background workers, and behind-firewall deployments without ingress; the trade-off is round-trip latency and rate-limit pressure.

Pattern precedence when more than one is configured: per-request body fields always win over tenant-level defaults. Polling can be combined with either of the other two as belt-and-braces.

Common pitfall: if your code assumes tenant-level config is set but the operator clears it (or never set it), encoding requests still succeed but no webhooks fire — silent breakage. Pattern 1 avoids this entire class by carrying config in every request.

Headers

Every webhook request carries:

text
Content-Type:    application/json
X-ZiB-Event:     <event name>            e.g. encoding.complete
X-ZiB-Job-ID:    <uuid>                  on encoding.* and {transcription,vision}.complete
X-ZiB-Signature: sha256=<hex>            HMAC-SHA256 over the raw request body

Events

file.readyFile is fully sharded across the network. From here on the file_id is permanent.
file.replicatedFile reached replication-safe state — ≥3 verified non-originator peers per shard. Backups can rely on it now.
file.failedFile became terminal-cancelled (currently fires on re-upload supersede; cancelled_reason in payload distinguishes scenarios). Tenants relying on this file_id should switch to the new one.
encoding.queuedEncoding job has been accepted and is waiting for a node.
encoding.startedA node has claimed the job and begun encoding.
encoding.progressThrottled progress update (every ~10s or every 5% delta) carrying a 0–100 integer percentage. Use this to drive a progress bar in your UI.
encoding.shardingTranscoding finished; encrypted segments are being distributed.
encoding.completeHLS manifest is live on the CDN — the file is playable now. Fires as soon as the originating node finishes sharding; ZiB replicates to peers in the background and the CDN seamlessly hands off as that completes (no tenant action needed).
encoding.failedEncoding failed — check error_message in the payload.
transcription.completeWebVTT subtitle track is ready (srt_file_id in extras).
vision.completeZibSidecar JSON is ready (sidecar_file_id in extras).

Payload shapes

Each event has its own envelope. All payloads include a top-level event string and timestamp (ISO 8601 UTC).

json
// file.ready — fired the moment shard count reaches the redundancy target
{
  "event":       "file.ready",
  "file_id":     "a1b2c3d4-...",
  "bucket":      "my-bucket",
  "object_key":  "videos/2026-04/film.mp4",
  "size_bytes":  524288000,
  "customer_id": "c0ffee...",
  "timestamp":   "2026-04-08T17:30:00.000Z"
}
json
// encoding.queued | encoding.started | encoding.sharding | encoding.complete | encoding.failed
// All five share the same shape — only the "event" and "status" fields change.
{
  "event":               "encoding.complete",
  "job_id":              "<encoding_jobs.id>",
  "status":              "complete",          // queued | encoding | sharding | complete | failed
  "file_id":             "a1b2c3d4-...",
  "customer_id":         "c0ffee...",
  "hls_manifest_url":    "https://cdn.zibnetwork.com/<file_id>/master.m3u8",  // null until "complete"
  "error_message":       null,                // populated on "failed"
  "transcription_model": "recommended",       // null if transcription not requested
  "vision_model":        "standard",          // null if vision not requested
  "thumbnails": {                              // null if tenant opted out; see "Thumbnails" above
    "poster_url":     "https://cdn.zibnetwork.com/objects/<poster_file_id>",
    "sprite_url":     "https://cdn.zibnetwork.com/objects/<first_sprite_sheet_file_id>",
    "sprite_vtt_url": "https://cdn.zibnetwork.com/objects/<sprite_vtt_file_id>"
  },
  "aspect_variants": {                         // omitted if tenant didn't opt in; see "Aspect variants" above
    "9:16": {
      "hls_manifest_url": "https://cdn.zibnetwork.com/stream/<vertical_file_id>/master.m3u8",
      "poster_url":       "https://cdn.zibnetwork.com/objects/<vertical_poster_id>"
    }
  },
  "timestamp":           "2026-04-08T17:32:14.000Z"
}
json
// encoding.progress — throttled mid-encode update
// Fires periodically while encoding is running so your UI can render a real
// progress bar instead of sitting at 0% from encoding.started until
// encoding.complete. Throttle policy: at most every 10s OR every 5% delta,
// whichever comes first. Idempotent — replays of the same percent are safe.
//
// You don't need this event to know the job will eventually finish — the
// existing encoding.complete event still fires. encoding.progress is purely
// for UI feedback.
{
  "event":     "encoding.progress",
  "job_id":    "<encoding_jobs.id>",
  "file_id":   "a1b2c3d4-...",
  "progress":  47,                 // integer 0–100
  "stage":     "encoding",         // "encoding" | "sharding"
  "timestamp": "2026-04-08T17:31:08.000Z"
}
json
// transcription.complete — fires when the SRT/WebVTT artefact has been
// produced and stored. Fetch via https://cdn.zibnetwork.com/objects/<srt_file_id>
{
  "event":        "transcription.complete",
  "job_id":       "<encoding_jobs.id OR compute_jobs.id>",
  "file_id":      "a1b2c3d4-...",
  "srt_file_id":  "f00d...",
  "timestamp":    "2026-04-08T17:34:02.000Z"
}
json
// vision.complete — fires when the ZibSidecar JSON has been produced and
// stored. Fetch via https://cdn.zibnetwork.com/objects/<sidecar_file_id>
// and parse as JSON for the full sidecar (scenes, IAB categories, etc.).
{
  "event":            "vision.complete",
  "job_id":           "<encoding_jobs.id OR compute_jobs.id>",
  "file_id":          "a1b2c3d4-...",
  "sidecar_file_id":  "ba11...",
  "timestamp":        "2026-04-08T17:36:48.000Z"
}
job_id semantics. The job_id field carries an encoding_jobs.id for encoding.* events. For transcription.complete and vision.complete it carries either a compute_jobs.id (when the AI was submitted as a standalone compute call) OR an encoding_jobs.id (when the AI ran as part of a combined encode+AI request). If you need a single dispatcher across both pipelines, route by file_id instead — it is unique per file regardless of which pipeline produced the event.
Re-uploads supersede previous uploads. When you upload a new file to a (bucket, object_key) that already has an object, ZiB replaces the prior file_id with a fresh one, marks the old row cancelled, and evicts the prior bytes from every node that held them (source shards + encoded segments). This applies to every upload type — images, JSON, generic blobs, and encoded video alike — not just files that have an encoding job attached. If the prior upload had an in-flight encoding job, that job is cancelled and no further encoding.* webhooks fire for it. You will only ever receive lifecycle events for the most recent upload to a given key. This means:
  • Route lifecycle events by file_id — it is the canonical discriminator and is guaranteed unique per upload.
  • When you receive file.ready for a key you've seen before, treat it as a hard reset: clear any state stored against the previous file_id for that key, including any prior job_id, CDN URLs, manifests, and progress.
  • Do not maintain a fallback that matches webhooks by job_id alone — a stale job_id from a prior upload should not be allowed to overwrite state for the current one.
  • The CDN URL cdn.zibnetwork.com/objects/{file_id} always reflects the latest upload, because the new file_id is what the SDK returns and what S3 GET redirects to.
  • You do not need to call DELETE /v1/api/file/{prior_file_id} as a cleanup step on re-upload — the bytes are reclaimed automatically. The cancelled s3_objects row stays for audit and is harvested by orphan cleanup later. Explicit DELETE is still the right call when you want to remove a file without replacing it.

Verifying signatures

Every webhook request includes an X-ZiB-Signature header containing an HMAC-SHA256 signature of the raw request body. Always verify this before processing the payload.

javascript
import crypto from 'crypto';

export async function POST(req) {
  // Read raw body before JSON parsing — signature is over raw bytes
  const body = await req.text();
  const sig = req.headers.get('x-zib-signature'); // "sha256=<hex>"

  const expected =
    'sha256=' +
    crypto
      .createHmac('sha256', process.env.ZIB_WEBHOOK_SECRET)
      .update(body)
      .digest('hex');

  if (sig !== expected) {
    return new Response('Unauthorized', { status: 401 });
  }

  const payload = JSON.parse(body);

  if (payload.event === 'file.ready') {
    // update your DB with cdn_url, mark upload complete, etc.
  }

  if (payload.event === 'encoding.complete') {
    // store hls_manifest_url, notify your users, etc.
  }

  return new Response('OK');
}
Use a constant-time comparison (like crypto.timingSafeEqual) to avoid timing attacks when comparing HMAC signatures.

Deleting Files

When you remove an asset from your application — a creator deletes a piece of content, a poster gets replaced, a blog post comes down — call the delete endpoint to remove the underlying bytes from ZiB. A single call cleans up everything: the source shards on every node holding them, any encoded HLS segments, the database row, the bucket counters, and the Cloudflare CDN cache.

Auth: server-side Authorization: Bearer access_key:secret_key — the same credentials you use for /v1/api/upload/initiate. Never call this from a browser; the secret must not leave your server.

Two ways to delete

Pick whichever you have on hand. Most tenants store the file_id returned by uploadDirect() alongside their content row, so the by-file_id form is usually the cheaper path.

bash
# Form A — by file_id (preferred)
curl -X DELETE \
  -H "Authorization: Bearer $ZIB_ACCESS_KEY:$ZIB_SECRET_KEY" \
  https://api.zibnetwork.com/v1/api/file/01846192-315c-4f0a-b766-12b18a26b209

# Form B — by bucket + object_key (S3-shaped)
curl -X DELETE \
  -H "Authorization: Bearer $ZIB_ACCESS_KEY:$ZIB_SECRET_KEY" \
  https://api.zibnetwork.com/v1/api/storage/posters/abc123/poster.card.webp

Response shape

json
{
  "success": true,
  "file_id": "01846192-315c-4f0a-b766-12b18a26b209",
  "bucket": "posters",
  "key": "abc123/poster.card.webp",
  "deleted": {
    "video_segments": 0,
    "encoding_jobs": 0,
    "shards": 12,
    "s3_objects": 1,
    "files": 1
  },
  "node_cleanup": {
    "ok": 12,
    "failed": 0,
    "queued_for_offline": 4
  },
  "encoding_jobs_cancelled": 0
}

The deleted map shows row counts removed from each table. The node_cleanup map distinguishes nodes that confirmed the shard delete synchronously (ok) from those that didn't respond in time (failed) — the latter still get a queued pending_deletions row that the node consumes on its next heartbeat, so eventual cleanup is guaranteed even when a node is offline at the moment you call.

Idempotency

Deleting a file_id that doesn't exist (or that belongs to another customer) returns 200 with { success: true, already_deleted: true } rather than 404. This deliberately doesn't leak ownership info, and it makes "clean up everything in this list" loops trivially safe to retry.

json
// Repeat call after the file is gone
{
  "success": true,
  "already_deleted": true,
  "file_id": "01846192-315c-4f0a-b766-12b18a26b209"
}

Side effects

  • In-flight encoding cancelled. If you delete a video while it's still being encoded, the encoding job is marked cancelled before the bytes go away. The encoder won't deliver any further encoding.* webhooks for that file_id.
  • CDN cache purged. ZiB calls Cloudflare's purge API for both cdn.zibnetwork.com/objects/{file_id} and cdn.zibnetwork.com/stream/{file_id}/. Worldwide eviction typically completes within ~30 seconds. Until then, edges with a warm cache may continue to serve the asset.
  • Source bytes removed. Every node holding a Reed-Solomon shard of this file is told to delete it. Encrypted shards never leave the network as plaintext, but they do consume disk — this reclaims that space.
  • Bucket + storage counters decremented. Reflected immediately in /v1/api/me/buckets and your storage usage dashboard.
Hard delete, no undo. There is no soft-delete or restore path. Once a delete returns success, the bytes are unrecoverable. Confirm at the application layer before you call.

The file.deleted webhook

If you have an account-level webhook URL configured, ZiB fires file.deleted after a successful delete. The shape mirrors file.ready so the same handler can route by event name. Headers: X-ZiB-Event: file.deleted, X-ZiB-File-ID, and the standard X-ZiB-Signature: sha256={hex}.

json
{
  "event": "file.deleted",
  "file_id": "01846192-315c-4f0a-b766-12b18a26b209",
  "bucket": "posters",
  "key": "abc123/poster.card.webp",
  "timestamp": "2026-05-14T03:21:09.482Z"
}

The webhook is fire-and-forget — the API response returns as soon as the bytes are gone, and webhook delivery happens asynchronously with the same 3-attempt retry + exponential backoff as every other ZiB webhook. Don't treat it as your trigger to remove the row from your DB — the API response already told you the delete succeeded. The webhook is for fan-out (audit logs, search index invalidation, etc.) when multiple systems want to know.

Recommended patterns

Pattern 1 — content row delete cascades to ZiB. When a creator removes a piece of content, fan out to ZiB for every asset stored on the row.

typescript
// app/api/content/[id]/delete/route.ts
import { NextResponse } from 'next/server';

export async function DELETE(req: Request, { params }: { params: { id: string } }) {
  const content = await db.content.findUnique({ where: { id: params.id } });
  if (!content) return NextResponse.json({ ok: true }, { status: 200 });

  // 1. Collect every ZiB file_id this row references.
  const fileIds = [
    content.video_file_id,
    content.poster_thumb_id,
    content.poster_card_id,
    content.poster_hero_id,
  ].filter(Boolean);

  // 2. Delete from ZiB in parallel — the endpoint is idempotent so retries are safe.
  await Promise.all(
    fileIds.map((id) =>
      fetch(`https://api.zibnetwork.com/v1/api/file/${id}`, {
        method: 'DELETE',
        headers: { Authorization: `Bearer ${process.env.ZIB_ACCESS_KEY}:${process.env.ZIB_SECRET_KEY}` },
      }).catch((e) => console.warn(`ZiB delete failed for ${id}`, e))
    )
  );

  // 3. Delete the local row last — if step 2 partially fails, your DB still
  //    holds the file_ids so a cleanup job can retry.
  await db.content.delete({ where: { id: params.id } });
  return NextResponse.json({ ok: true });
}

Pattern 2 — replace an asset. Uploading a new file to the same (bucket, object_key) as an existing asset automatically evicts the prior bytes from every node and decrements your storage counters — no explicit DELETE call is needed. Just upload the new asset and write the new file_id to your local row.

typescript
async function replacePoster(contentId: string, newFile: File) {
  // 1. Upload the new poster to the same object_key. ZiB detects the
  //    existing row at (bucket, object_key), cancels any in-flight
  //    encoding on the prior file_id, fires file.failed for it, and
  //    evicts the prior bytes from every holding node in the background.
  const init = await fetch('/api/posters/upload-initiate', { method: 'POST', body: JSON.stringify({ contentId }) });
  const registration = await init.json();
  const { file_id: newFileId } = await ZiBStorage.uploadDirect(newFile, registration);

  // 2. Write the new file_id to your row. The supersede flow has already
  //    taken care of evicting the prior bytes — no follow-up DELETE needed.
  await db.content.update({ where: { id: contentId }, data: { poster_card_id: newFileId } });
}

Two corollaries:

  • If you receive file.failed with cancelled_reason: "superseded by re-upload to same bucket+object_key", it's informational only — you don't need to take any action; the eviction is already underway.
  • Explicit DELETE /v1/api/file/{file_id} is still the right call when you want to remove a file without replacing it (Pattern 1 above) — that path additionally drops the audit row + fires file.deleted.
GDPR / right-to-be-forgotten. A successful delete response is your audit trail that the bytes have been removed from the storage network. Combine with the file.deleted webhook receipt log if your compliance posture requires a second-system attestation.

Security Model

ZiB guarantees
  • Every file encrypted with a unique key before leaving the browser
  • Keys stored only in ZiB backend — never on storage nodes
  • Node operators see only encrypted fragments — no filenames, no keys, no plaintext
  • Node-facing identifiers are opaque UUIDs — real object keys never exposed
  • High redundancy — files survive multiple simultaneous node failures
Your responsibility
  • Keep access_key and secret_key on your server only
  • Authenticate your own users before calling /v1/api/upload/initiate
  • Store file_id and cdn_url in your own database
  • Verify webhook signatures with X-ZiB-Signature before processing
  • Access control for CDN URLs (UUID-gated, but not secret — treat as public)

Why credentials must stay on the server

The upload/initiate endpoint uses your master credentials, which have full access to your ZiB account — all buckets, all objects, all quota. Exposing these in browser JavaScript would let anyone who reads your source code upload to your buckets and exhaust your storage quota.

The registration object returned by initiate is a one-time upload token. It authorises a single upload to a single pre-determined location. It contains no credentials and cannot be used to list buckets, delete objects, or do anything other than complete the specific upload it was created for. This is why it is safe to pass from your server to the browser.

Self-enforcing encryption

ZiB's encryption model is self-enforcing. If an attacker obtained a registration token and uploaded plaintext bytes directly to the node URL (bypassing the SDK), the CDN would decrypt those bytes using the key generated at initiate time and return garbage — because the bytes stored on the node would not be valid ciphertext for that key.

There is no configuration flag, no "plaintext mode", and no bypass. The CDN always decrypts. Files that were not encrypted correctly before upload cannot be served correctly — ever. This makes the security property durable even if an integration makes a mistake.

CDN URLs are public by design. The UUID in cdn.zibnetwork.com/objects/<uuid> is effectively a capability — possessing the URL grants read access to the file. Treat CDN URLs like signed S3 URLs: don't expose them in public listings if the files are meant to be private. Store them in your database and serve them only to authenticated users.

Job Status API

Webhooks are the fastest way to know a file finished encoding, but if you need to render progress to an end user (an upload dashboard, a "your video is processing" screen, a retry button), poll GET /v1/api/my-jobs. Every encoding job in your account is returned with its current lifecycle phase — from queued all the way through safe (replicated to enough peers that the file is durable).

Auth: same as every other tenant endpoint — Authorization: Bearer access_key:secret_key. Jobs returned are scoped to your customer automatically.

Lifecycle phases

Each job is in exactly one phase:

  • queued — accepted, waiting for a node to claim it
  • encoding — FFmpeg is running; phase_progress_pct ticks 0–100
  • sharding — encoded segments are being hashed + written to storage
  • local_complete — all bytes on the originating node, awaiting backend greenlight
  • replicating — segments are being pulled to peer nodes
  • safe — ✅ 3+ peers confirmed; the file is durably stored and will survive any single-node failure. replication_safe_at is set.
  • failed — transient failure, auto-retrying (retry_after tells when)
  • failed_user_action — retries exhausted; surface a Retry button to the user
  • cancelled — user-cancelled; derivatives being cleaned up

List jobs

bash
curl -s https://api.zibnetwork.com/v1/api/my-jobs \
  -H "Authorization: Bearer ${ACCESS_KEY}:${SECRET_KEY}"

# Filter:
curl -s "https://api.zibnetwork.com/v1/api/my-jobs?status=all" \
  -H "Authorization: Bearer ${ACCESS_KEY}:${SECRET_KEY}"

?status=active (default) returns jobs whose phase is not safe or cancelled. ?status=all returns everything.

Response

json
{
  "jobs": [
    {
      "file_id": "0c8e3765-9b2b-4a29-82cb-2f1911fe750c",
      "title": "trailer-final.mp4",
      "phase": "replicating",
      "phase_progress_pct": 66.7,
      "phase_detail": "peer 2 of 3 verified",
      "phase_updated_at": "2026-04-24T05:12:04Z",
      "replication_peer_count": 2,
      "replication_target_count": 3,
      "replication_safe_at": null,
      "retry_count": 0,
      "last_error": null,
      "cancellable": true,
      "retriable": false,
      "created_at": "2026-04-24T04:58:12Z",
      "node_id": "ZB-899233"
    }
  ]
}

Cancel a job

bash
curl -X DELETE https://api.zibnetwork.com/v1/api/my-jobs/${FILE_ID} \
  -H "Authorization: Bearer ${ACCESS_KEY}:${SECRET_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"reason":"user_abandoned_upload"}'

Sets phase=cancelled and queues cleanup commands to every node that holds any segment of the file. Not reversible. Only valid while cancellable: true.

Retry a failed job

bash
curl -X POST https://api.zibnetwork.com/v1/api/my-jobs/${FILE_ID}/retry \
  -H "Authorization: Bearer ${ACCESS_KEY}:${SECRET_KEY}"

Resets a failed_user_action job back to queued. Use this behind a Retry button in your UI. Only valid while retriable: true.

Suggested polling cadence

  • While your UI is visible and you have any job in a non-terminal phase: every 5 s
  • Background tab or no active jobs: every 30 s
  • Webhooks remain the preferred signal for terminal transitions — use /my-jobs for in-flight progress UX

The durability guarantee

Once a job reaches phase: "safe", the file has ≥3 verified non-originator peers holding every segment. From that point:

  • Any one peer can go offline permanently and the file is still served
  • The originating node can freely reclaim its local copy
  • Playback is served through cdn.zibnetwork.com with automatic failover between peers

Before safe is reached, the originator's bytes are unconditionally protected — the file cannot be lost to a transient failure on the originating node while replication is in progress.

Compute / AI

ZiB Compute is an optional AI inference layer built on top of ZiB storage. Compute-capable nodes run transcription and vision analysis on videos stored in the network. Output is an encrypted WebVTT subtitle file (auto-injected into the HLS manifest) and an encrypted ZibSidecar JSON — a full metadata envelope covering transcription, chapters, scene understanding, IAB content taxonomy, GARM brand safety, clip suggestions, and thumbnail candidates.

Compute jobs run on the same encrypted infrastructure as storage. The node decrypts the source video in memory to run inference, then re-encrypts and shards the output before reporting file IDs to the backend. Node operators never see plaintext video or AI output.

Trigger AI from an encoding request

Pass transcription and/or vision options to startEncoding() — the encoding node runs AI immediately after encoding while the decrypted source is already in memory.

javascript
const { encoding_id } = await zib.startEncoding(fileId, {
  transcription: 'recommended',  // 'fastest' | 'recommended' | 'accurate'
  vision: 'standard'             // 'standard' | 'hq'
});

// Poll until complete
const status = await zib.getEncodingStatus(encoding_id);
// status.srt_file_id    → WebVTT file (auto-injected into HLS manifest)
// status.sidecar_file_id → ZibSidecar JSON

Standalone compute jobs

Run transcription or vision on any already-stored video — no re-encoding needed.

javascript
// Standalone transcription
const { job_id } = await zib.submitTranscription(fileId, 'recommended');

// Standalone vision AI
const { job_id } = await zib.submitVisionAI(fileId, 'standard');

// Poll job status
const result = await zib.getComputeStatus(job_id);
// result.status          → 'queued' | 'processing' | 'complete' | 'failed'
// result.srt_file_id     → WebVTT file ID (when complete)
// result.sidecar_file_id → ZibSidecar file ID (when complete)

Reading the ZibSidecar

The CDN decrypts the sidecar on the fly — fetch the CDN URL directly and parse as JSON.

javascript
// Get the sidecar CDN URL from job status
const status = await zib.getComputeStatus(job_id);
const sidecarUrl = zib.getCdnUrl(status.sidecar_file_id);

// Fetch and parse
const sidecar = await fetch(sidecarUrl).then(r => r.json());

// Key fields:
sidecar.scene_understanding.executive_summary        // 150-200 word summary
sidecar.content_classification.garm.brand_safety_score  // 0.0–1.0
sidecar.clip_suggestions[0].virality_score           // 0.0–1.0
sidecar.chapters_youtube_format                      // paste into YouTube description
sidecar.transcript.full_text                         // full transcript string
sidecar.transcript.segments[0].intent               // hook / payoff / cta / ...
sidecar.content_classification.iab_categories[0].name  // IAB 3.0 taxonomy

Quality tiers

Transcription
fastest
Highest throughput
Good for short clips, lower accuracy
recommended
Balanced — default
Best accuracy/speed balance
accurate
Highest quality
Best accuracy, slower processing
Vision AI
standard
Default
Fast, good quality scene analysis
hq
Higher quality
Deeper analysis, slower processing

Deep-dive docs

Errors & Troubleshooting

Real status codes you'll see, what they mean, and what (if anything) to do about them. Every ZiB error response is JSON with an error field describing the cause.

HTTP status reference

400Bad Request

When you see it: Malformed body, missing required field, invalid bucket name, file_size of 0, or pipeline stage requested for a non-existent tier (e.g. transcription "ultra"). The error message names the specific field.

What to do: Fix the request body. Not retryable as-is.

401Unauthorized

When you see it: Bearer token missing, malformed, or wrong format. Master credentials use the format `<accessKey>:<secretKey>` in the Authorization header, not OAuth-style.

What to do: Verify ZIB_ACCESS_KEY / ZIB_SECRET_KEY in your environment. Regenerate via the Storage UI if you suspect rotation.

403Forbidden

When you see it: Credentials are valid but you don't own the resource (customer_id mismatch, bucket belongs to a different tenant).

What to do: Check the customer_id / bucket scoping. Cross-tenant operations need separate credentials.

404Not Found

When you see it: file_id, encoding_id, customer_id, or bucket does not exist (or was deleted). On the CDN, a 404 on `/master.m3u8` means no segments exist yet for that file_id — either the encode hasn't finished or no encode was ever requested.

What to do: Verify the ID. For CDN 404s during an active encode, poll `/v1/api/pipeline/<file_id>` to see actual phase.

409Conflict

When you see it: Direct S3-path uploads where the (bucket, key) already exists with different contents. The SDK's uploadDirect path handles supersede transparently; you only see 409 if you bypass the SDK.

What to do: Use `ZiBStorage.uploadDirect()` from the SDK. If you really need direct S3 PUT, delete the existing key first or use a different key.

500Internal Server Error

When you see it: Unhandled exception on the backend. Always a ZiB-side bug. Should not happen in normal operation.

What to do: Retry once after a short delay. If reproducible, file a bug with the request_id from the response (when present) and the timestamp.

502Bad Gateway

When you see it: Almost never returned by ZiB directly. If you see this in your tenant proxy, it usually means YOUR proxy received a non-JSON response from ZiB and translated it to 502. The original error is in your proxy's stdout — don't treat the 502 as the bug, it's a relay.

What to do: Check your proxy logs (e.g. your Next.js API route) for the upstream response. The real status code + body tell you what ZiB actually returned.

503Service Unavailable

When you see it: "No storage nodes available" — the upload selector couldn't find a node satisfying all gates (storage_enabled, s3_enabled, mesh_reachable, public_url, storage_allocated_gb > 0). Fleet capacity is genuinely tight or all eligible nodes recently went offline.

What to do: Retry with exponential backoff (10s, 30s, 90s). The fleet recovers on its own as nodes come back online. Persistent 503s for &gt; 5 min indicate a real outage — contact ZiB ops.

Tenant-proxy 502s — the relay trap

If your tenant has a Next.js / Express / Lambda proxy in front of ZiB (typical for browser clients that don't hold master credentials), be aware: that proxy almost certainly translates any non-JSON ZiB response into a generic 502. The browser then shows "Failed to initiate upload (502)" or similar.

The 502 is rarely the actual bug. It's a relay. The real cause is in the proxy's server-side log:

  • If the proxy logs ZiB returned 503 — fleet capacity, see 503 above.
  • If ZiB returned 401 — master credentials wrong on the proxy.
  • If ZiB returned 5xx HTML — temporary backend issue, retry.
  • If connection refused / timeout — proxy can't reach ZiB; check egress firewall.

Surface the upstream status code in your proxy's 502 response body. Future-you debugging at 3am will thank you.

Common silent failures

These don't throw an error, but produce wrong behaviour. All have bitten real tenants:

  • Missing webhook_url in the request body. If you use per-request webhook config but forget to include the URL on a particular call, ZiB stores null, no events fire, and there's no error. Verify with SELECT webhook_url FROM encoding_jobs WHERE id = '...' if you have DB access, or check your request-builder for conditional logic that drops the field.
  • Polling that never stops on terminal phase. If your code treats any non-error response as "keep polling", you'll hammer the endpoint forever after the encode is done. Always break on phase === 'safe' / 'failed_user_action' / 'cancelled'. See Video Encoding section.
  • Routing logic that keys on encoding_id. Re-encode requests can return a different encoding_id if forced or after supersede. Always route by file_id in your storage and references; treat encoding_id as job-ephemeral.
  • Reading status when you should read phase. Both fields are present in responses; they mostly agree but phase is canonical. Code keyed on status may miss new phase values added in the future.
  • Webhook handler that 200s before persisting. ZiB retries on non-2xx but stops on 200. If your handler returns 200 then crashes before writing the event to your DB, you've dropped it. Persist first, ack second.

Getting help

When opening a support thread, include:

  • customer_id and the specific file_id / encoding_id involved.
  • UTC timestamp of the failing request (within ±30 s is plenty).
  • The full request body (redact sensitive fields like webhook_secret) and the response status + body.
  • If a proxy is involved, the upstream response your proxy received from ZiB — not just the 502 your code saw.