Video hosting: resumable ingest, idempotent media jobs, and cacheable playback

The design separates four lifecycles: upload bytes, publication policy, media derivation, and playback. Completing one does not silently complete the others. Sources remain quarantined until checksum, moderation/rights policy, and the required rendition set all pass.

Resumable upload semantics

The upload path follows tus 1.0 core semantics. HEAD reads the server's Upload-Offset; PATCH uses Content-Type: application/offset+octet-stream plus that exact Upload-Offset. A mismatched offset is rejected with 409 Conflict. It is not a Content-Range protocol.

Offset advancement and chunk identity are concurrency-controlled. The service reserves the exact current offset, persists bytes provisionally, verifies the chunk boundary/checksum policy, then atomically advances the committed offset. A crash before that commit leaves the old public offset; retrying the same range is safe because provisional chunk identity and checksum are deterministic. Two clients cannot both claim the same offset. Finalization verifies the declared length and source checksum, but it leaves the object quarantined until the publish gate succeeds.

Media-pipeline delivery semantics

Kafka consumer-group assignment is not called a “lease.” A worker receives a job, derives deterministic output keys from video id, source version, profile set, and rendition, and commits the consumer offset only after every required output and readiness record is durable. A crash after output write but before offset commit can redeliver the job, so writes and catalog transitions are idempotent.

Partial outputs are never made playable. Exhausted retries preserve source version, job identity, attempt history, and safe diagnostics in a dead-letter queue. Redrive is reviewed and reuses deterministic identity. Error details visible to users do not expose private source URLs, tokens, or unsafe moderation evidence.