What is the biggest difference between context engineering and prompt engineering?

Prompt engineering asks 'how do I phrase this to get the right output?' and focuses on instruction wording. Context engineering asks 'what does the model need to know to do this right?' and focuses on information architecture. When tasks shift from single interactions to multi-step agent workflows, instruction wording matters far less than context design.

My project only has a few hundred users. Do I need context engineering?

Scale is not the deciding factor — task complexity is. If your AI feature only does single-turn knowledge retrieval (Q&A, search), RAG is usually sufficient. Once you need cross-session memory, multi-step reasoning, or long-running agent tasks, you need context engineering thinking, regardless of user volume.

What is context rot and how do you diagnose it quickly?

Context rot is the degradation of context quality in long conversations. Symptoms include responses repeating earlier patterns, increasing contradictions, and growing tool selection errors. Quick diagnosis: at conversation turn 20-30, ask the model about an early decision and check for inconsistencies. Fix with Simon Willison's context pruning (removing old irrelevant history) or summarization compression.

What is the relationship between MCP and context engineering?

MCP (Model Context Protocol) is a standard protocol for connecting AI models to external tools — it's infrastructure. Context engineering is the skill of managing those connections wisely, including tool count control (accuracy drops above 10 tools), tool description quality optimization, and avoiding stale cache silent failures. MCP makes connections easy; context engineering tells you which connections to make and how to manage them.

What is the fastest way to start learning context engineering?

Start with davidkimai/Context-Engineering on GitHub (9.1k+ stars) for a complete handbook and examples. Practically, begin by validating Write + Select strategies with LangGraph. Add Compress when context approaches limits (achieves 75-90% cost savings). Only consider Isolate when task complexity demands multi-agent architecture. Don't try to implement all four strategies at once.

Context Engineering Guide 2026: Beyond Prompting

Your AI agent starts contradicting itself at turn 5, picks the wrong tool, or simply "forgets" what was discussed earlier. You rewrite the prompt repeatedly, but the problem persists. This is not a precision problem with your instructions. According to Cognition AI, the root cause of most AI agent failures is context architecture, not instruction wording.

In 2026, the core skill AI engineers need is undergoing a fundamental shift: from "writing better prompts" to "designing the information architecture of AI systems." This guide provides a complete practical framework starting from Karpathy's precise definition, breaking down four failure modes, four strategies, tool selection, and a three-tier implementation path you can start today.

TL;DR

Core definition (Karpathy): context engineering is "the delicate art and science of filling the context window with just the right information for the next step." LLM as CPU, context window as RAM, the engineer's job is OS management — loading the right data into working memory at each step.
Four failure modes: Context Poisoning (hallucination compounds), Context Distraction (history overload), Context Confusion (irrelevant noise degrades tool selection), Context Clash (contradictory cross-turn information)
Four strategies: Write (externalize information), Select (retrieve relevant information), Compress (reduce token usage), Isolate (partition agent environments)
Implementation path: RAG + scratchpad (tier 1) → Summarization compression (tier 2) → Multi-agent isolation (tier 3, as needed)

What Is Context Engineering and Why Isn't Prompt Engineering Enough?

In June 2025, Andrej Karpathy published a defining post on X, explicitly endorsing the term "context engineering":

"+1 for 'context engineering' over 'prompt engineering'. People associate prompts with short task descriptions you'd give an LLM in your day-to-day use. When in every industrial-strength LLM app, context engineering is the delicate art and science of filling the context window with just the right information for the next step."

This is not just a terminology swap. Karpathy is pointing to a fundamental shift in perspective.

Prompt engineering asks: "How do I phrase this to get the model to do it right?" Context engineering asks: "What does the model need to know to do it right?"

The former focuses on instruction wording; the latter on information architecture. For single-turn interactions (one question, one translation), prompt influence is sufficient. But when tasks become multi-step agent workflows requiring cross-session memory, dynamic tool use, and conditional reasoning, the prompt's wording is no longer the bottleneck — what's in the context window is.

Karpathy's mental model is worth keeping: LLM as CPU, context window as RAM, the engineer's work as OS management. Before each inference, you decide what code and data to load into working memory. Load the wrong things, and even the fastest CPU produces wrong results.

This is why Cognition AI directly positions context engineering as "the #1 job of engineers building AI agents." When building complex agent systems, Cognition's engineers found that model capability was already sufficient — the limiting factor was always information management: what's in the context, when it goes in, and how much of it.

Four Context Failure Modes: Why Your Agent Crashes

Based on LangChain's framework, agent failures fall into four distinct patterns. Understanding these modes is prerequisite to designing defensive strategies.

1. Context Poisoning

Hallucinated information doesn't self-correct once it enters the context. Every subsequent turn of reasoning builds on that error, compounding it. Example: the agent forms an incorrect user preference judgment at turn 3. By turn 7, that judgment has been referenced as fact three times.

Defenses: Context quarantine (isolate suspect information segments), verification mechanisms (cross-source validation before writing to long-term memory).

2. Context Distraction

Cumulative conversation history grows until the model over-relies on outdated behavioral patterns — even with a large context window. SwirlAI's 2026 report documents a concrete quantified threshold: accuracy starts dropping noticeably above 10 tools; 90 tools equals 50K+ tokens of overhead, consuming a massive portion of context budget.

This phenomenon is also widely discussed in the Hacker News community (915 upvotes): engineers report that despite vendors claiming million-token support, accuracy measurably degrades after roughly 10k tokens in practice.

Defenses: Periodic summarization compression of history; keep tool count under 10.

3. Context Confusion

Irrelevant information filling the context window causes errors in tool selection and task execution. This is the "lost-in-the-middle" phenomenon in concrete form: LLMs pay significantly less attention to information placed in the middle of context versus at the start or end. This characteristic does not improve as context windows grow larger.

Defenses: RAG retrieves only the 20-30 most relevant chunks; place critical information at the beginning or end, not buried in the middle.

4. Context Clash

When contradictory information appears across conversation turns, the model's reasoning ability rapidly degrades. Example: the agent receives "user prefers English output" at turn 1, then "all outputs in Japanese please" at turn 8. Without an explicit conflict resolution mechanism, the model's behavior becomes unpredictable.

Defenses: Context pruning (remove conflicting older information), explicit "latest instruction takes precedence" rules.

Four Strategies: Write / Select / Compress / Isolate

LangChain's Lance Martin, after studying production agent systems, organized four core strategy categories. This framework has become the foundational consensus for context engineering.

Write: Externalize Information for Retention

The Write strategy persists important information outside the context window, ensuring durability across turns or sessions. Two levels:

Short-term scratchpad: Lets agents write intermediate reasoning results and completed steps during a task, preventing "amnesia"
Long-term memory: Writes user preferences, task history, and key decisions to a vector DB or structured store for future sessions

When to use: Any scenario requiring cross-session memory; multi-step tasks needing execution state tracking.

Select: Retrieve the Most Relevant Information

The Select strategy dynamically pulls the most relevant information from external storage into the context window, rather than stuffing everything in at once.

Embedding search + RAG: Retrieve the most relevant chunks from a knowledge base
Tool description filtering: Don't put all tool descriptions in context — only the subset the current task likely needs

Pinecone defines five key context elements: conversation history, user input, long-term memory, background knowledge, and tool definitions. The core of the Select strategy is precisely retrieving what the current step needs from these five buckets.

When to use: Systems with large knowledge bases; agents with more than 10 tools.

Compress: Reduce Token Usage

The Compress strategy reduces token consumption while preserving critical information. This is the highest ROI priority when working within budget constraints.

Summarization: Compress long conversation history into summaries
Trimming: Remove history segments that are no longer relevant
Prompt caching: Cache commonly used system prompts or knowledge base segments to avoid recomputation

DEV Community's Gabriel Henrique documents that prompt caching in production systems can achieve 75-90% cost savings. A concrete production example: Claude Code automatically triggers auto-compact summarization when context usage reaches 95% — the Compress strategy in action in a real production environment.

When to use: Long-conversation systems; when cost control is a priority; when context window approaches its limit.

Isolate: Partition Context Environments

The Isolate strategy decomposes complex tasks across multiple agents or sandboxes, each holding only the tool subset and information segments it needs.

Multi-agent architecture: Different subtasks go to different specialized agents, preventing single-agent context overload
Tool subset assignment: Each agent only sees the tools it needs, preventing Context Confusion
Context sandboxing: Sensitive information processed in isolated environments, preventing Context Poisoning from spreading across agents

The Hacker News engineering community reinforces this strategy: "Complex tasks should be split across multiple agents, each with a dedicated tool subset."

When to use: Complex multi-step tasks; security isolation requirements; when a single agent's tool count is out of control.

When Is RAG Enough? When Do You Need Full Context Engineering?

This is the most practical judgment question engineers face. The answer has clear criteria.

When RAG is sufficient:

Single-turn or few-step knowledge retrieval (document Q&A, semantic search)
No cross-session memory requirements
Linear task logic without conditional branching or tool switching

Triggers for upgrading to full context engineering:

Context poisoning appears (hallucinations starting to compound)
Tool count exceeds 10
Tasks require cross-session memory
Multi-step reasoning where the agent must decide next actions based on prior results

Towards Data Science provides a clear warning: naive RAG automatically fails under 800-token budget constraints across multiple turns, and the failure is silent — no obvious error messages. The recommended complete architecture is: Hybrid Retriever (~85ms retrieval latency) + Memory layer + Compression engine + Token Budget Enforcer.

It's worth noting that large context windows cannot replace precise information selection. Research shows 100k+ token contexts still exhibit "lost-in-the-middle" degradation — attention to middle-positioned information is significantly lower than content at the ends. The extreme case: 90 tools generating 50K+ tokens overhead (SwirlAI 2026), making even a 200k context window suddenly cramped.

Tool Selection: LangChain, LlamaIndex, and DIY

For most developers, "which framework to use" is the most practical question. Each framework has different levels of support across the four context engineering strategies.

LangGraph (LangChain ecosystem)

LangGraph has mature support for Compress and Isolate strategies, especially suited for scenarios needing rapid validation of multi-agent patterns. LangChain's official context engineering research itself is built on LangGraph. For most developers, this is the best entry point: richest documentation and community resources, lowest friction for rapid prototyping.

LlamaIndex

Deeper customization depth for the Select strategy (RAG pipeline) than LangChain, particularly for hybrid search integration. If your core requirement is high-quality knowledge base retrieval, LlamaIndex's pipeline design is more flexible. Downside: agent orchestration support is relatively weaker, and Isolate strategy implementation requires more manual assembly.

DIY systems

Appropriate for teams that already have stable context management requirements and need deep optimization for specific bottlenecks. Start with frameworks to validate patterns, identify which strategy is the bottleneck, then consider building that specific part yourself. Don't start from scratch.

Context caching vs. context engineering

The prompt caching feature in Claude and Gemini (API-level caching) is one cloud implementation of the Compress strategy, not an independent technical concept. Understanding the four-strategy framework helps clarify when to use context caching — it addresses "repeated computation costs for frequently used content" within the Compress strategy.

Vector DB vs. Graph DB

For most scenarios, semantic similarity search from vector DBs (Pinecone, Weaviate, Qdrant) is sufficient to support the Select strategy. Neo4j's GraphRAG fits specific scenarios: when the knowledge base contains complex relationship structures (enterprise knowledge graphs, multi-hop reasoning), and flat vector search accuracy is clearly insufficient.

Production Environment Pitfalls

Diagnosing and Fixing Context Rot

Context rot is context quality degradation in long conversations. Common symptoms: responses start repeating early behavioral patterns, tool selection accuracy drops, and contradictory statements increase.

Simon Willison shared three practical techniques in the Hacker News discussion:

Context quarantine: Isolate newly entered context information first, confirm reliability before allowing subsequent reasoning to use it
Context pruning: Periodically remove irrelevant conversation history segments rather than letting all history accumulate indefinitely
Context offloading: Move information that doesn't need immediate access to external storage (long-term memory), retrieving it with the Select strategy when needed

MCP Tool Management

After Anthropic donated MCP (Model Context Protocol) to the Agentic AI Foundation in late 2025, it reached 97M+ monthly downloads (SwirlAI 2026), becoming the industry standard. But managing MCP servers is itself a context engineering application, with common pitfalls including:

Tool overload: Audit tool count before going live; above 10 requires grouping or dynamic filtering strategies
Poor description quality: Unclear tool descriptions directly hurt Select strategy accuracy
Stale cache silent failures: After MCP server version updates, old cached descriptions can cause the model to pick wrong tools without obvious error messages

A Three-Tier Implementation Path

Based on SwirlAI's recommendations and Lance Martin's research, the best onboarding approach is layered implementation — don't try to build all four strategies at once.

Tier 1: Write + Select (Start Today)

Build a RAG pipeline + agent scratchpad. This is the lowest-friction starting point: use LangGraph to build an agent that can query a knowledge base while writing intermediate steps to a scratchpad during tasks.

This combination covers the Write and Select strategies, solving the fundamental "amnesia" and "knowledge limitations" problems. The davidkimai/Context-Engineering GitHub repository (9.1k+ stars, backed by IBM Zurich and Princeton research) provides forkable examples.

Tier 2: Compress (Prioritize When Budget-Constrained)

When the Tier 1 system starts hitting context window limits or API costs are growing rapidly, add the Compress strategy.

Implementation sequence: try prompt caching first (Claude/Gemini API level, near-zero implementation cost) → add conversation history summarization → only then consider custom trimming logic.

According to documented research, prompt caching can achieve 75-90% cost savings — a number with direct decision impact for startups and freelance developers.

Tier 3: Isolate (Only When Tasks Demand It)

When the system needs to handle complex tasks across multiple domains, or tool count has grown out of control, introduce multi-agent isolation architecture.

Confirm trigger conditions before implementing: tool count exceeds 10 and can't be reduced, task branching is too complex for single-agent management, explicit security isolation requirements exist. If none of these conditions apply, Tier 1 and 2 architecture is likely sufficient.

Conclusion

The shift from "writing better prompts" to "designing context architecture" is the most practical skill upgrade direction for AI engineers in 2026. Karpathy's definition reminds us: context engineering is both art and science — the core is not finding a magic prompt, but loading the right information into the model's working memory before each inference.

Four failure modes (Poisoning, Distraction, Confusion, Clash) give you diagnostic language. Four strategies (Write, Select, Compress, Isolate) give you design tools. The three-tier progressive path lets you start today, without waiting until your system is too complex to fix.

The starting point is davidkimai/Context-Engineering (9.1k+ stars) and LangGraph. You don't need to build everything from scratch — validate the pattern first, then optimize for bottlenecks.

For deeper exploration of the AI agent tooling layer, see LangGraph Production Agent Guide and Best MCP Servers Guide.