The asset & upload pipeline

Stateless edges, stateful core

Benchy's asset pipeline runs on a small set of Cloudflare Workers (the benchy-edge repo) built around one principle: the edge is deliberately dumb. Workers verify signatures and move bytes; they hold no user state and make no authorization decisions. The application database remains the single authority on who owns what, who bought what, and which policy decisions were recorded. Every security review gets shorter when the edge can't make decisions.

The mesh pipeline — grant, PUT, promote, queue, validate, apply — lives in two Workers, and the platform underneath (Cloudflare Queues and R2) carries the operational weight rather than bespoke infrastructure.

The workers

Deployed Worker names carry a benchy- prefix (benchy-upload-gateway and so on); the names above are the directories in the repo. A shared edge-gateway package supplies the common primitives: HMAC-SHA256 signed-token verification, R2 key safety, and HTTP Range parsing.

Where files live

The workers move bytes; Cloudflare R2 buckets hold them. Clear storage boundaries make it easier to understand which files are public, which are entitlement-gated, and which are still being checked.

Data location

Benchy production buckets are configured with Cloudflare R2's location hint Eastern North America (ENAM)— benchy-models-prod, benchy-products-prod, benchy-media-prod, benchy-ingest-prod, and their matching -dev buckets. Application metadata runs on Convex; edge delivery and ingest run on Cloudflare Workers.

Stage A — upload

Clients never write to public storage. An upload starts with an HMAC-signed grant — audience-scoped, short-lived, bound to one pending/v1/… key in the ingest bucket with an explicit byte ceiling. The gateway accepts a single PUT up to 64 MiB, or multipart in 32 MiB parts (create → upload parts → complete, with abort support) up to 512 parts. When the application finalizes the upload, a separate promote credential — a different audience entirely — copies the object from ingest to its destination bucket (models, products, or media), computing the SHA-256 in-stream.

Stage B — validate

model created / new version published
  → fast content preflight (initial feedback only)
  → enqueue { modelId, modelKey, sha256, policyVersion,
              validationProfile, executionMode, unitAssumption, idempotencyKey }
  → Cloudflare Queue "benchy-mesh-validation"
  → worker: fetch bytes from R2
      ≤ 4 MiB    → benchy-mesh compiled to WASM, in-Worker
      > 4 MiB    → FILE_TOO_LARGE
  → report → R2: derived/v2/mesh-reports/<sha256>/<policy>/<profile>/<mode>/<unit>.json
  → POST /mesh/apply → model.meshValidation updated

Details that matter:

• Preflight uses client_advisory mode for fast feedback. It cannot grant eligibility or overwrite a completed server policy decision.
• Jobs are idempotent by key — validation:<modelId>:<sha256>:<policy>:<profile>:<mode>:<unit> — so retries and duplicate enqueues are no-ops without conflating a changed unit context.
• Files up to 4 MiB validate in-Worker via benchy-mesh-wasm. Above that the job returns FILE_TOO_LARGE and the object stays stored and owner-accessible.
• Internal hops (enqueue, apply) are authenticated with shared secrets, and the apply callback is allowlisted to a fixed set of Convex deployment hosts over HTTPS — the worker can't be pointed at arbitrary URLs.

Stage C — apply & gate

The worker writes the full report to R2 under derived/v2/mesh-reports/<sha256>/<policy>/marketplace_publish/server_enforced/<unit>.json, then posts a mapped summary — lifecycle, validation profile, execution mode, decision, check coverage, issue codes, observations, and print-readiness assessment — to the application. The application applies it to the model and enforces the gates: a rejection blocks publishing publicly and blocks commerce, and nothing else. File checks passed appears only when the complete current v2 evidence is bound to the model's exact bytes, format, and unit context. An engine name alone never grants eligibility.

The downloads gateway

World checks the buyer's acquisition against the exact immutable release, package manifest, and license rights. It signs a short-lived capability for each authorized file; the gateway at models.benchy.world verifies the bearer header and streams protected bytes with Range support. A recurring grant's capability expires no later than its paid access period. Unlisting stops discovery or sales while preserving acquired files. The worker verifies capabilities without querying the application database.

The validation engine these workers embed is fully open — benchy-mesh — and the reasoning behind the architecture is on the blog in Building Benchy.