Stacks/Knowledge work
Long-running document agents rebuilt on LangGraph (X thread)
Abhijeet Tiwari's thread on production agents that grind through messy documents for half an hour: start right (frontier model, full harness, expert playbook as skills, golden datasets), then cheap (small models first, read-once index, compaction), then both (route models by role, evidence packs, background review, escalate only on failure), and finally own the loop by rebuilding it on LangGraph with byte-for-byte parity on hundreds of recorded requests before the evals judged it. Lesson: cheapest per token is not cheapest per task.
- Use case
- Production agents that work through a stack of messy documents for about half an hour and decide on their own when the job is done, following an expert playbook.
- Category
- Knowledge work
- Team size
- not stated
- Orchestrator / framework
- Loop rebuilt on LangGraph after proving request-level parity on hundreds of recorded requests (same prompts, tools and context, byte for byte). Models are routed by role, workers get evidence packs, review runs in the background, and the model reasons while code enforces.
- Memory
- A read-once index of documents plus context compaction, replacing fresh sub-agent contexts and an ever-growing history. A footprint check verifies evidence before an agent may say a value is not in the documents.
- Evals
- Golden datasets from day one; evals judged the outputs of the rebuilt LangGraph loop against recorded requests.
- Observability
- not stated
- Cost (stated in writeup)
- Not stated
- Cost per output (stated)
- Not stated in the writeup
- WhatWeRun estimate
WhatWeRun estimate
Not enough info to estimate
Models aren't named and no token counts or volumes are given; only that runs take about half an hour.
Reviewed Nov 14, 2026
- Models
- Observability tools
- not stated
- Other tools
- Expert playbook encoded as skillsRecorded-request replay for parity testing
- Source
- https://x.com/khwabeedaAbhi/status/2120809415167414369
Abhijeet Tiwari (@khwabeedaAbhi) · Published Nov 12, 2026 · Last verified Nov 14, 2026
Related
More like this
langchain.comWorkday's internal Agent Platform
An internal developer platform for building, deploying, and running AI agents for HR, finance, and IT on Workday's infrastructure, with auth, security, compliance, and audit built in
Dropbox's file-acting agent
An agent inside Dropbox that answers questions about a user's files and carries out tasks such as organizing folders, renaming files, and preparing materials to share
notion.comNotion Agent (sixth-generation harness)
Knowledge-work agents and Custom Agents inside Notion, including tasks that can now run for more than 24 hours