![\"\"](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2026\/04\/AI-Pre-Compute-Engine.png\")
<\/p>\nAI coding assistants are powerful but only as good as their understanding of your codebase. When we pointed AI agents at one of Meta\u2019s large-scale data processing pipelines \u2013 spanning four repositories, three languages, and over 4,100 files \u2013 we quickly found that they weren\u2019t making useful edits quickly enough.\u00a0<\/p>\n
We fixed this by building a pre-compute engine: a swarm of 50+ specialized AI agents that systematically read every file and produced 59 concise context files encoding tribal knowledge that previously lived only in engineers\u2019 heads. The result: AI agents now have structured navigation guides for 100% of our code modules (up from 5%, covering all 4,100+ files across three repositories). We also documented 50+ \u201cnon-obvious patterns,\u201d or underlying design choices and relationships not immediately apparent from the code, and preliminary tests show 40% fewer AI agent tool calls per task. The system works with most leading models because the knowledge layer is model-agnostic.<\/p>\n
The system also maintains itself. Every few weeks, automated jobs periodically validate file paths, detect coverage gaps, re-run quality critics, and auto-fix stale references. The AI isn\u2019t a consumer of this infrastructure, it\u2019s the engine that runs it.\u00a0<\/p>\n
The Problem: AI Tools Without a Map<\/p>\n
Our pipeline is config-as-code: Python configurations, C++ services, and Hack automation scripts working together across multiple repositories. A single data field onboarding touches configuration registries, routing logic, DAG composition, validation rules, C++ code generation, and automation scripts \u2013\u00a0 six subsystems that must stay in sync.<\/p>\n
We had already built AI-powered systems for operational tasks, scanning dashboards, pattern-matching against historical incidents, and suggesting mitigations. But when we tried to extend it to development tasks, it fell apart. The AI had no map. It didn\u2019t know that two configuration modes use different field names for the same operation (swap them and you get silent wrong output), or that dozens of \u201cdeprecated\u201d enum values must never be removed because serialization compatibility depends on them.<\/p>\n
Without this context, agents would guess, explore, guess again and often produce code that compiled but was subtly wrong.<\/p>\n
The Approach: Teach the Agents Before They Explore<\/p>\n
We used a large-context-window model and task orchestration to structure the work in phases:\u00a0<\/p>\n
Two explorer agents mapped the codebase,\u00a0
\n11 module analysts read every file and answered five key questions,\u00a0
\nTwo writers generated context files, and\u00a0
\n10+ critic passes ran three rounds of independent quality review,\u00a0
\nFour fixers applied corrections,\u00a0
\nEight upgraders refined the routing layer,\u00a0
\nThree prompt testers validated 55+ queries across five personas,\u00a0
\nFour gap-fillers covered remaining directories, and\u00a0
\nThree\u00a0 final critics ran integration tests \u2013 50+ specialized tasks orchestrated in a single session.<\/p>\n
The five questions each analyst answered per module:<\/p>\n
What does this module configure?
\nWhat are the common modification patterns?
\nWhat are the non-obvious patterns that cause build failures?
\nWhat are the cross-module dependencies?
\nWhat tribal knowledge is buried in code comments?<\/p>\n
Question five was where the deepest learnings emerged. We found 50+ non-obvious patterns like hidden intermediate naming conventions where one pipeline stage outputs a temporary field name that a downstream stage renames (reference the wrong one and code generation silently fails), or append-only identifier rules where removing a \u201cdeprecated\u201d value breaks backward compatibility. None of this had been written down before.<\/p>\n
What We Built: A Compass, Not An Encyclopedia<\/p>\n
<\/p>\n
Each context file follows what we call \u201ccompass, not encyclopedia\u201d principle \u2013\u00a0 25\u201335 lines (~1,000 tokens) with four sections:<\/p>\n
Quick Commands (copy-paste operations).\u00a0
\nKey Files (the 3\u20135 files you actually need).\u00a0
\nNon-Obvious patterns.\u00a0
\nSee Also (cross-references).\u00a0<\/p>\n
No fluff, every line earns its place. All 59 files together consume less than 0.1% of a modern model\u2019s context window.<\/p>\n
On top of this, we built an orchestration layer that auto-routes engineers to the right tool based on natural language. Type, \u201cIs the pipeline healthy?\u201d and it scans dashboards and matches against 85+ historical incident patterns. Type, \u201cAdd a new data field\u201d and it generates the configuration with multi-phase validation. Engineers describe their problem; the system figures out the rest.<\/p>\n
The system self-refreshes every few weeks, validating file paths, identifying coverage gaps, re-running critic agents, and auto-fixing issues. Context that decays is worse than no context at all.<\/p>\n
Beyond individual contextual files, we generated a cross-repo dependency index and data flow maps showing how changes propagate across repositories. This turns \u201cWhat depends on X?\u201d from a multi-file exploration (~6000 tokens) into a single graph lookup (~200 tokens) \u2013 in config-as-code where one field change ripples across six-subsystems.<\/p>\n
Results<\/p>\n
Metric
\nBefore
\nAfter<\/p>\n
AI context coverage
\n~5% (5 files)
\n100% (59 files)<\/p>\n
Codebase files with AI navigation
\n~50
\n4,100+<\/p>\n
Tribal knowledge documented
\n0
\n50+ non-obvious patterns<\/p>\n
Tested prompts (core pass rate)
\n0
\n55+ (100%)<\/p>\n
In preliminary tests on six tasks against our pipeline, agents with pre-computed context used roughly 40% fewer tool calls and tokens per task. Complex workflow guidance that previously required ~two days of research and consulting with engineers now completes in ~30 minutes.<\/p>\n
Quality was non-negotiable: three rounds of independent critic agents improved scores from 3.65 to 4.20 out of 5.0, and all referenced file paths were verified with zero hallucinations.<\/p>\n
Challenging the Conventional Wisdom on AI Context Files<\/p>\n