Forge - Resumable Encryption v1

Overview

Forge is SparkVault's encryption and transfer service, the data plane that every Ingot (an encrypted file stored in a Vault) passes through on its way into and out of storage. It is a high-performance encryption proxy that handles AES-256-GCM encryption and decryption server-side, so your client never touches key material or implements cryptography.

Uploads use TUS v1.0 resumable sessions with bounded PATCH chunks, so every file size (from a one-page PDF to a 5 TB archive) follows the same protocol and cryptographic format.

Base URL

Forge runs on a dedicated endpoint optimized for resumable encrypted transfer:

https://forge.sparkvault.com

Authentication

Forge uses Ingot Secure Transaction Keys (ISTKs) for authentication. An ISTK is a temporary, scoped credential generated when you access an Ingot in a Vault. It authorizes a specific cryptographic operation (encrypt or decrypt) for a limited time.

An ISTK is an opaque, base64url-encoded 32-byte random value. It carries no embedded claims. All context (account, vault, ingot, operation, key material) lives in a server-side record that Forge looks up when the token is presented.

How to Obtain an ISTK

The Ingot endpoints issue ISTKs. Creating an ingot (POST /v1/vaults/{vault_id}/ingots) returns a forge_url for encryption; initiating a download (POST /v1/vaults/{vault_id}/ingots/{ingot_id}/download) returns a download_url for decryption. Both URLs carry the ISTK as the istk query parameter:

curl -X POST https://api.sparkvault.com/v1/vaults/{vault_id}/ingots \
  -H "X-API-Key: sv_live_xxx" \
  -H "X-Vault-Access-Token: YOUR_VAT" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "large-backup.tar.gz",
    "size_bytes": 1073741824,
    "content_type": "application/gzip"
  }'

{
  "data": {
    "ingot_id": "ing_abc123...",
    "name": "large-backup.tar.gz",
    "size_bytes": 1073741824,
    "content_type": "application/gzip",
    "status": "uploading",
    "forge_url": "https://forge.sparkvault.com/encrypt?istk=3q2-7wX9kLmN0pQrStUvWxYzAb1Cd2Ef3Gh4Ij5Kl6M",
    "expires_at": 1782172800
  }
}

Single-Use vs Streaming ISTKs

Encrypt ISTKs and default decrypt ISTKs are single-use: Forge atomically claims the token the first time it is presented, and any reuse fails with 401 AUTHENTICATION_ERROR (details.code: ISTK_USED). In practice this means a download_url works exactly once.

An upload's ISTK stops working the moment a later upload of the same file is created (a retried create whose first answer was lost, or a second device uploading the same name): the later upload is the one kept. From then on the earlier upload URL, and every request on an upload session it opened, is answered 410 UPLOAD_GONE naming UPLOAD_SUPERSEDED in details.coreCode and in the X-Upload-Refusal header, no further byte of it is stored, and what it had stored is discarded. A new upload is no remedy: the later one already holds the file.

It stops working the same way the moment the ingot it uploads to is deleted: the upload URL, and every request on an upload session it opened, is answered 410 UPLOAD_GONE ("The file this upload was for no longer exists") with X-Upload-Refusal: NOT_FOUND, and what it had stored is discarded. A new upload can store the file again.

An ISTK is the 43-character base64url string SparkVault issues; any other value is answered as an ISTK Forge does not know (401, details.code: INVALID_ISTK), with nothing read.

For media playback (where a player issues many Range requests against one URL), pass { "stream": true } in the body of the Ingot download request (POST /v1/vaults/{vault_id}/ingots/{ingot_id}/download). This mints a reusable streaming-decrypt URL that is not consumed on use. Its blast radius is bounded by a cumulative byte cap and a sliding idle TTL (see the expiration note below).

Using the ISTK with Forge

Extract the ISTK from the returned forge_url and pass it in the X-ISTK header when creating the upload session (POST /encrypt). That is where the token is validated and atomically consumed, unless Forge answers 503: a service it depends on failed for a moment before anything was spent, the token is handed back unconsumed, and the same request may be sent again after its Retry-After. Subsequent PATCH/HEAD/DELETE requests are keyed by the upload ID from the Location header and do not read X-ISTK (sending it is harmless but unnecessary). For decryption, the ISTK rides along as the istk query parameter already embedded in the download_url.

curl -i -X POST https://forge.sparkvault.com/encrypt \
  -H "Tus-Resumable: 1.0.0" \
  -H "Upload-Length: $(wc -c < plaintext.file)" \
  -H "X-ISTK: 3q2-7wX9kLmN0pQrStUvWxYzAb1Cd2Ef3Gh4Ij5Kl6M"

API Reference

Health Check

curl https://forge.sparkvault.com/health

{
  "status": "healthy",
  "version": "1.0.0",
  "timestamp": "2026-07-02T12:00:00.000Z"
}

Capability Discovery

Tus-Resumable: 1.0.0
Tus-Version: 1.0.0
Tus-Extension: creation,termination,expiration
Tus-Max-Size: 5497558138880

Tus-Max-Size is 5,497,558,138,880 bytes (5 TB), the maximum size of a single ingot.

Create Upload Session

curl -i -X POST https://forge.sparkvault.com/encrypt \
  -H "Tus-Resumable: 1.0.0" \
  -H "Upload-Length: $(wc -c < plaintext.pdf)" \
  -H "X-ISTK: 3q2-7wX9kLmN0pQrStUvWxYzAb1Cd2Ef3Gh4Ij5Kl6M"

Upload Chunk

curl -X PATCH https://forge.sparkvault.com/encrypt/{upload_id} \
  -H "Tus-Resumable: 1.0.0" \
  -H "Upload-Offset: 0" \
  -H "Content-Type: application/offset+octet-stream" \
  --upload-file plaintext.pdf

Forge may answer a PATCH with an error before its body is fully sent, so a client must stop sending once a response arrives: listen for the response, not only for write back-pressure.

Check Upload Offset

A HEAD whose new attempt to publish is refused for good answers 410 (or 412) with no body: its X-Upload-Refusal header names the reason, as the final PATCH's details.coreCode does (UPLOAD_POOL_FULL, UPLOAD_SUPERSEDED, UPLOAD_PROTOCOL_RETIRED, PRECONDITION_FAILED, or NOT_FOUND when the file it was for no longer exists). A HEAD or PATCH of an upload a later upload of the same file replaced answers 410 with X-Upload-Refusal: UPLOAD_SUPERSEDED at any offset, the PATCH before any of its body is stored, and one whose file was deleted answers 410 with X-Upload-Refusal: NOT_FOUND the same way.

Cancel Upload

Decrypt Stream

Byte ranges are the basis for resumable and partial downloads and for media seeking with reusable streaming URLs. To resume a download, send the bytes you still need as Range and the ETag you were served as If-Range (from a fresh download URL when yours was single-use): a 206 continues the same content, while a 200 means the ingot was replaced since, and its whole new content follows from the first byte.

Every refusal below carries the same JSON error envelope as Forge's other errors ({ "success": false, "error": { "code", "message" } }, Content-Type: application/json) beside the headers it names.

# The download_url from the Ingot download endpoint includes the ISTK
curl "https://forge.sparkvault.com/decrypt?istk=3q2-7wX9kLmN0pQrStUvWxYzAb1Cd2Ef3Gh4Ij5Kl6M" \
  --output decrypted.pdf

curl "https://forge.sparkvault.com/decrypt?istk=3q2-7wX9kLmN0pQrStUvWxYzAb1Cd2Ef3Gh4Ij5Kl6M" \
  -H "Range: bytes=1048576-2097151" \
  --output part.bin

# The status line is printed: 206 is the rest of the same content; 200 means
# the ingot was replaced, and rest.bin holds its whole new content.
curl -D - "https://forge.sparkvault.com/decrypt?istk=3q2-7wX9kLmN0pQrStUvWxYzAb1Cd2Ef3Gh4Ij5Kl6M" \
  -H "Range: bytes=1048576-" \
  -H 'If-Range: "tus_9f2c41d07be3a856"' \
  --output rest.bin

Check Progress

{
  "success": true,
  "progress": {
    "bytes_processed": 52428800,
    "total_bytes": 1073741824,
    "status": "forging"
  }
}

Examples

Encrypt and Store a File

# 1. Create the ingot - the response's forge_url carries the ISTK
RESPONSE=$(curl -s -X POST https://api.sparkvault.com/v1/vaults/vlt_xxx/ingots \
  -H "X-API-Key: $API_KEY" \
  -H "X-Vault-Access-Token: $VAT" \
  -H "Content-Type: application/json" \
  -d "{
    \"name\": \"backup.tar.gz\",
    \"size_bytes\": $(wc -c < backup.tar.gz),
    \"content_type\": \"application/gzip\"
  }")

FORGE_URL=$(echo $RESPONSE | jq -r '.data.forge_url')
ISTK="${FORGE_URL#*istk=}"

# 2. Create a TUS upload session (the only request that needs X-ISTK)
HEADER_FILE=$(mktemp)
curl -s -D "$HEADER_FILE" -o /dev/null -X POST https://forge.sparkvault.com/encrypt \
  -H "Tus-Resumable: 1.0.0" \
  -H "Upload-Length: $(wc -c < backup.tar.gz)" \
  -H "X-ISTK: $ISTK"

# Location is a path ("/encrypt/{upload_id}") - resolve it against the Forge origin
UPLOAD_URL="https://forge.sparkvault.com$(awk 'tolower($1) == "location:" { print $2 }' "$HEADER_FILE" | tr -d '\r')"

# 3. Upload plaintext through the TUS session. A file up to 50 MB fits in a
#    single PATCH; larger files send one PATCH per 50 MB chunk (see below).
curl -X PATCH "$UPLOAD_URL" \
  -H "Tus-Resumable: 1.0.0" \
  -H "Upload-Offset: 0" \
  -H "Content-Type: application/offset+octet-stream" \
  --upload-file backup.tar.gz

Resumable Uploads

# Ask Forge where to resume (keyed by the upload URL - no ISTK needed)
OFFSET=$(curl -sI "$UPLOAD_URL" \
  -H "Tus-Resumable: 1.0.0" | awk 'tolower($1) == "upload-offset:" { print $2 }' | tr -d '\r')

# Continue at that offset with the next plaintext chunk (non-final chunks
# must be exactly 50 MB = 52428800 bytes)
tail -c +$((OFFSET + 1)) backup.tar.gz | head -c 52428800 > chunk.bin
curl -X PATCH "$UPLOAD_URL" \
  -H "Tus-Resumable: 1.0.0" \
  -H "Upload-Offset: $OFFSET" \
  -H "Content-Type: application/offset+octet-stream" \
  --upload-file chunk.bin

Retrying an Upload

A request whose answer was lost may still have landed, so an upload never resends a chunk from an offset the server has not just named. What to do with each answer:

• No answer (a dropped connection, a timeout), or 408, 425, 429 or 5xx: wait an exponential backoff with jitter, never shorter than Retry-After, then HEAD the upload and continue from its Upload-Offset.
• 409 OFFSET_MISMATCH: continue at once from the response's own Upload-Offset; no HEAD is needed.
• 409 CONFLICT (no Upload-Offset): another request holds the upload's write lease. Wait Retry-After, then HEAD and continue.
• 404 or 410: the upload no longer exists (cancelled, expired, its file removed, or refused for good at publication). Only a new upload from byte 0, on a fresh Forge URL, can store the file, unless the answer names UPLOAD_POOL_FULL or UPLOAD_SUPERSEDED (details.coreCode, or the X-Upload-Refusal header): no new upload remedies either.
• Any other 4xx is the server's verdict on the request: sending it again earns the same answer.

Bound the retries by failures in a row and by time without progress, and start both over whenever the offset moves, so a slow upload that keeps landing bytes is never cut off while a dead one still ends. An Upload-Offset equal to Upload-Length proves every byte arrived, not that the upload is published: finish with a zero-length PATCH at that offset, which answers 204 once the upload is published; on a 409 or 503, send it again after Retry-After.

Every answer about an upload that still stands carries Upload-Expires (the tus expiration extension), but for one given before the upload is read (a malformed Upload-Offset) and a server error other than a deferred publication's 503 (a shutting-down Forge's 503, a cancel's 503 for chunks it has yet to delete, a failure of the session store): the time the upload lapses if nothing more happens. While it receives, that is 72 hours from its creation or its last chunk, and never later than the time its size allows (its size at 1 MiB per second, and at least 7 days from its creation). Once every byte has arrived, it is 72 hours from the last chunk or from the latest attempt to publish it, past that allowance if need be. Past it an upload that never finished answers 410; one SparkVault published answers its full offset instead.

JavaScript / Node.js

import * as tus from 'tus-js-client';

// Forge fixes every session's chunk geometry: each non-final PATCH must be
// exactly X-Chunk-Size (52,428,800 bytes). tus-js-client cannot size its
// PATCHes from a response header, so pin the value here.
const FORGE_CHUNK_SIZE = 52428800;

function uploadWithForge(file, forgeUrl) {
  const url = new URL(forgeUrl);
  const istk = url.searchParams.get('istk');

  return new Promise((resolve, reject) => {
    const upload = new tus.Upload(file, {
      endpoint: `${url.origin}/encrypt`,
      chunkSize: FORGE_CHUNK_SIZE,
      // tus-js-client sends headers on every request; Forge only requires
      // X-ISTK on session creation and ignores it elsewhere.
      headers: { 'X-ISTK': istk },
      // tus-js-client never reads Retry-After: these delays outlast Forge's.
      retryDelays: [0, 2000, 5000, 10000, 30000],
      metadata: {
        filename: file.name,
        filetype: file.type || 'application/octet-stream'
      },
      // onSuccess also follows a HEAD that shows every byte received, which
      // is not yet publication: confirm it before resolving.
      onSuccess: () => confirmPublished(upload.url, file.size).then(resolve, reject),
      onError: reject
    });

    upload.start();
  });
}

// The zero-length PATCH at the final offset answers 204 once the upload is
// published, and 409 or 503 with Retry-After while its publication is in
// progress or deferred.
async function confirmPublished(uploadUrl, size) {
  for (let attempt = 0; attempt < 60; attempt += 1) {
    const response = await fetch(uploadUrl, {
      method: 'PATCH',
      headers: {
        'Tus-Resumable': '1.0.0',
        'Upload-Offset': String(size),
        'Content-Type': 'application/offset+octet-stream'
      }
    });
    if (response.ok) return;
    if (response.status !== 409 && response.status !== 503) {
      throw new Error(`Upload not published (status ${response.status})`);
    }
    const seconds = Number(response.headers.get('Retry-After')) || 5;
    await new Promise((wait) => setTimeout(wait, seconds * 1000));
  }
  throw new Error('Upload not published yet: check the ingot later');
}

Python

import os
from urllib.parse import parse_qs, urljoin, urlparse

import requests

def upload_file(forge_url: str, input_path: str):
    """Upload a file through Forge with TUS."""
    parsed = urlparse(forge_url)
    istk = parse_qs(parsed.query)['istk'][0]
    endpoint = f'{parsed.scheme}://{parsed.netloc}/encrypt'
    size = os.path.getsize(input_path)

    # X-ISTK is only needed here, on session creation
    create = requests.post(
        endpoint,
        headers={
            'Tus-Resumable': '1.0.0',
            'Upload-Length': str(size),
            'X-ISTK': istk,
        },
    )
    create.raise_for_status()
    upload_url = urljoin(endpoint, create.headers['Location'])
    # Forge dictates the chunk geometry: every non-final PATCH must be
    # exactly this many bytes, so never substitute a value of your own.
    chunk_size = int(create.headers['X-Chunk-Size'])

    offset = 0
    with open(input_path, 'rb') as f:
        while offset < size:
            chunk = f.read(chunk_size)
            patch = requests.patch(
                upload_url,
                headers={
                    'Tus-Resumable': '1.0.0',
                    'Upload-Offset': str(offset),
                    'Content-Type': 'application/offset+octet-stream',
                },
                data=chunk,
            )
            patch.raise_for_status()
            offset = int(patch.headers['Upload-Offset'])


def decrypt_file(download_url: str, output_path: str):
    """Decrypt a file through Forge.

    The download_url is provided by the Ingot download endpoint
    and includes the ISTK as a query parameter. Default download
    URLs are single-use.
    """
    response = requests.get(download_url, stream=True)
    response.raise_for_status()

    with open(output_path, 'wb') as out:
        for chunk in response.iter_content(chunk_size=8192):
            out.write(chunk)

    print(f'Decrypted: {output_path}')


# Usage
upload_file(forge_url, 'report.xlsx')
decrypt_file(download_url, 'report-restored.xlsx')

Common Use Cases

Security Details

Encryption Algorithm

Forge uses AES-256-GCM (Galois/Counter Mode) for authenticated chunk encryption:

• Key Size: 256 bits (32 bytes), the vault-level Vault Access Key (VAK)
• IV/Nonce: a random 96-bit base nonce per upload; each chunk's nonce is derived deterministically as base nonce XOR chunk index
• Authentication Tag: 128 bits (16 bytes) per chunk for integrity verification
• Additional Authenticated Data: every chunk's AAD binds the account, vault, ingot, and chunk index, so ciphertext cannot be replayed across ingots or reordered within one
• Mode: Galois/Counter Mode provides both confidentiality and authenticity

Key Handling

Forge never derives or stores long-lived keys of its own. The key arrives with the ISTK:

1. The Core API envelope-encrypts the vault's VAK into the ISTK record: an HSM-derived key-encryption key wraps a random data key, which wraps the VAK
2. Forge validates the ISTK, derives the same KEK from the HSM, and unwraps the data key, then the VAK
3. The VAK is used directly as the AES-256-GCM key for the operation
4. Isolation between ingots and chunks comes from the per-upload random base nonce and the per-chunk AAD binding, not from separate keys
5. The VAK is zeroized from memory as soon as the operation completes or fails

Transport Security

All traffic to and from Forge is carried over HTTPS. TLS terminates at the platform edge before requests reach the service, so plaintext never crosses the public internet unencrypted.

Error Responses

{
  "success": false,
  "error": {
    "code": "AUTHENTICATION_ERROR",
    "message": "ISTK expired",
    "details": {
      "code": "ISTK_EXPIRED"
    }
  }
}

Best Practices

• Check ISTK expiration before starting: Verify that expires_at gives enough time to complete your operation. Obtain a fresh Forge URL if needed.
• Use tus clients: Keep upload logic on the resumable tus path and send bounded PATCH chunks with Upload-Offset.
• Store metadata separately: Forge only encrypts ingot bytes. Store filename, content type, and other metadata in SparkVault ingot metadata.
• Implement retry logic: Retry what can succeed when sent again (a request that got no answer, a 408, 425, 429 or 5xx) after an exponential backoff with jitter that never undercuts Retry-After. An upload's 409 names the offset to continue from or asks you to wait, and its 404 or 410 calls for a new upload (Retrying an Upload, above); every other 4xx is final. Continue a download from the bytes you hold (Resuming a Download).
• Verify with checksums: Calculate checksums of plaintext before encryption and after decryption to verify data integrity end-to-end.
• Use separate Vaults for different data types: Create distinct Vaults for different data categories (backups, documents, media). Each Vault has its own VAK, giving you key isolation and independent access control.

Usage

Forge operations consume pooled bandwidth included with your seat subscription. Both encrypt and decrypt operations count the processed data size against your pooled bandwidth. There is no per-GB charge. When pooled bandwidth is exhausted, extend it with a capacity block.

For subscription tiers and capacity blocks, see the Pricing page.