Stacks/Customer support

Intercom Fin

Intercom's Fin is a production support agent whose published reliability work focuses on running on top of imperfect model vendors. A sophisticated routing layer fails over across Anthropic, Bedrock, Vertex, Azure, and OpenAI, isolates Fin's capacity, and can move traffic between Sonnet 4 and GPT models. Observability is built around per-call token and latency data, with Datadog shown as a monitoring surface, and answers go through a custom RAG and validation pipeline.

Use case

A customer-support AI agent (chat, and in later work email and voice) that Intercom says resolves up to 70% of customers' conversations, with a 99.8% monthly uptime SLA

Category

Customer support

Team size

not stated

Orchestrator / framework

A custom LLM routing layer sits in front of every request. Routes are defined per model, region, and use case, with latency-based routing, load balancing, sequenced failover, cross-vendor failover, and cross-model failover (for example, sending traffic to a GPT model if Sonnet 4 is unreachable on all vendors). Capacity is isolated so Fin's pool cannot be exhausted by lower-priority jobs such as translation. Each Fin request uses multiple models. Intercom says it is exploring off-the-shelf proxies such as LiteLLM and Bifrost but still runs its own router.

Memory

A bespoke retrieval-augmented generation pipeline grounded in approved sources: past Intercom conversations, help center articles, PDFs, and HTML/URLs.

Evals

The Fin AI Engine validates whether each LLM output meets accuracy, safety, confidence, and groundedness standards. Intercom says it continuously tests both LLM accuracy and individual RAG steps.

Observability

Every LLM call is instrumented for token usage, response times, and system load across vendors. A Datadog chart is shown in the reliability post. Single-point-of-failure reporting raises high-priority alerts when a model's redundancy or capacity isolation falls short.

Monthly cost

not stated

Models

Claude Sonnet 4GPT models (cross-model failover)

Observability tools

Datadog

Other tools

AWS BedrockGCP Vertex AIAnthropic APIAzure OpenAIOpenAI APILiteLLM (being explored)Bifrost (being explored)

Source

https://fin.ai/research/fin-running-a-reliable-service-over-unreliable-parts

Ketan Bhatt · Aug 7, 2025