Reference
Production checklist
Validate security, durability, observability, and operations before exposing a Flary agent to users.
Use this checklist before the first production release.
Runtime
- Pin Flary, Flue, Pi, and Cloudflare adapter versions.
- Run typecheck, unit tests, Worker integration tests, and the clean npm installation test.
- Apply the D1 run migration and verify the Flue Durable Object migration.
- Set an explicit default mode, model, cache policy, and run limits.
- Confirm the production R2 binding exists when large files or history need it.
Security
- Authenticate every route before resolving TrustedRunContext.
- Derive tenant, application, project, agent, and identity from trusted server data.
- Keep provider credentials and MCP credentials out of prompts, events, logs, and API responses.
- Use encrypted secret storage and a single refresh lock per connection.
- Authorize every R2 operation. Do not treat a tenant prefix as a security boundary.
- Block private-network MCP destinations and unapproved outbound access.
- Require approval for writes, deletes, secret access, external messages, commits, merges, deploys, and Sandbox imports.
Durability
- Submit an idempotency key for every run and state-changing tool call.
- Persist the run before starting model or tool work.
- Reconnect with the last event sequence after browser or network loss.
- Test Durable Object eviction during a model stream and during a tool write.
- Test waiting for user input and approval across a restart.
- Test credential rotation and revoked credentials during recovery.
- Reconcile outcome_unknown instead of repeating an external write.
Operations
- Record provider, model, billing mode, cache policy, token usage, tool calls, latency, retries, and trace IDs.
- Keep subscription cost unknown when the provider reports no price.
- Project the Flue stream to immutable history without creating a second transcript authority.
- Index Recall records with organization and project filters.
- Define retention and deletion for source history, R2 blobs, and search index records.
- Set quotas for runs, subagents, tools, tokens, duration, and storage.
Failure drills
Run these scenarios in a staging environment:
- Close the browser during a long run. Reconnect and verify no duplicate events or tool calls.
- Rotate a BYOK credential after admission. Verify recovery uses the admitted generation or returns a typed recovery failure.
- Approve a write, stop the Worker, and verify the journal prevents a second write.
- Use the same public run ID under two tenants. Verify both bindings remain isolated.
- Delete a project and verify history, blobs, and derived Recall records are removed or tombstoned.