Stacks/Internal engineering platform

Duolingo's production agent platform

Duolingo built a shared agent platform so teams would stop rebuilding MCP setup, credentials, repo workspaces, orchestration, and evals. Developers define an agent in a registry; Temporal's AgentWorkflow runs it on interchangeable SDKs (Claude Agents SDK, Codex CLI, OpenAI Agents SDK) and routes model calls through an internal LLM Gateway. The same Temporal backbone runs production agents and the eval suite.

Use case

An internal platform so any Duolingo team can define a production agent once and invoke it from Slack, internal sites, a CLI, or another Temporal workflow. In use for fixing CI failures, addressing code-review comments, and a Slack bot for release managers that investigates crashes and summarizes findings

Category

Internal engineering platform

Team size

not stated

Orchestrator / framework

Agents are declared in a registry (prompt, model, MCP servers, output type, owner). A Temporal workflow called AgentWorkflow loads the definition, prepares the environment, runs the agent on a pluggable runtime, and returns structured output. Runtimes already include the Claude Agents SDK, Codex CLI, and the OpenAI Agents SDK. With Temporal's plugin, MCP tool calls become Temporal activities.

Memory

The workflow prepares a workspace, including cloning configured repositories and loading project context. Further long-term memory is not described.

Evals

Agent evals also run on Temporal. Real agents are run against authored scenarios; graders include structured_output checks, diff_assertions on the repo, no_op_consistency, and an optional LLM-as-judge. A suite workflow fans out child workflows, aggregates a report, and can save runs to a dashboard. Deterministic graders are described as the foundation.

Observability

Tool-call inputs, outputs, failures, and retries are visible in the Temporal UI when MCP calls run as activities. The internal LLM Gateway provides cost tracking, usage tracking, and provider abstraction.

Monthly cost

not stated

Models

gpt-5.5 (shown in an example agent definition)Models reached through an internal LLM Gateway

Observability tools

TemporalInternal LLM Gateway

Other tools

MCP (GitHub and Sentry appear in the example definition)SlackCLIInternal site for creating agents

Source

https://blog.duolingo.com/production-ready-ai-agent-platform/

Guadalupe Aliseda-Canton · Aug 3, 2026