Python Batch Worker

Worker Flow

1. Pull batch items from queue.
2. Create and start jobs with concurrency limit.
3. Poll statuses in async loop.
4. Write outputs and usage metrics to DB.

Operational Advice

• Use dead-letter queue for repeated failures.
• Expose worker metrics for observability.
• Gracefully shutdown workers to avoid job loss.

Operational Patterns

• Use queue-based orchestration for sustained throughput.
• Separate latency-sensitive traffic from bulk processing workloads.
• Centralize webhook/event consumption through one idempotent handler.
• Use dead-letter queue for repeated failures and manual replay.

Go-Live Checklist

1. Load test with realistic clip duration distribution.
2. Validate alerting for error rate, queue lag, and failed jobs.
3. Confirm credit accounting and billing UX alignment.
4. Prepare escalation template containing job_id + payload summary.

Python Worker Scaling Strategy

1. Start with bounded concurrency and tune by model profile.
2. Separate polling coroutine pool from create/start tasks.
3. Track queue lag and auto-scale workers on demand.