Work
Multi-Turn Agent Orchestration
Durable workflows for running multi-turn conversations with a RAG agent in production, with graceful fallback on model failure.
Brief
The team needed to run multi-turn conversations with a RAG agent in production, end to end, without losing state when a model response was slow or a downstream call failed.
Approach
- Conversation Lifecycle: Integrated the RAG agent into durable Temporal workflows spanning follow-up messages, model output with citations, and translation.
- Infrastructure: Ran the service behind mTLS with dedicated ingress routes, backed by caching and pub-sub.
- Fallback Logic: Built circuit breakers that routed to a human agent or surfaced a service request based on the model’s output and defined rules.
- Resiliency: Handled retries and timeouts so a slow or failed model call degraded gracefully instead of losing the conversation.
Result
Conversations stayed durable and auditable through model failures, timeouts, and retries, with a defined fallback path whenever the model couldn’t complete a turn.