Context Engineering Beyond CLAUDE.md: The 5-Layer Hierarchy

The Skill That Replaced Prompt Engineering

In February 2025, Andrej Karpathy coined "vibe coding" and the world embraced it. By the end of 2025, MIT Technology Review traced a clear arc: the industry had pivoted from "just accept all AI suggestions" to a discipline called context engineering.

The shift makes sense. Prompt engineering focuses on what you say to an AI model. Context engineering focuses on everything the model sees: project rules, session history, tool outputs, memory systems, and multi-agent coordination. As Anthropic's engineering team puts it: "Building effective AI agents is less about finding the right words and more about answering a critical question: What configuration of context is most likely to generate our model's desired behavior?"

This post goes beyond the CLAUDE.md deep-dive in Part 3 of our AI Technical Debt series. Where Part 3 covered CLAUDE.md optimization as a single lever, this guide covers the full context engineering discipline: the hierarchy, the memory patterns, the failure modes, and the strategies that compound across every line of AI-generated code.

What Context Engineering Actually Means

Birgitta Bockeler, Distinguished Engineer at Thoughtworks, defines it simply: "Context engineering is curating what the model sees so that you get a better result." Her analysis on martinfowler.com uses Claude Code as the primary example of how context configuration options have exploded.

Andrej Karpathy frames it with a systems analogy: the LLM is a CPU and the context window is RAM. "The engineer's job is akin to an operating system: loading that working memory with just the right code and data for the task."

This includes:

• Task descriptions and explanations (your prompt)
• Few-shot examples (patterns the model should follow)
• RAG and retrieved data (project-specific knowledge)
• Tool definitions and outputs (what the agent can do and has done)
• State and history (conversation context)
• Compaction (summarizing to stay within limits)

Tobi Lutke, CEO of Shopify, endorsed the shift: "I really like the term 'context engineering' over prompt engineering. It describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM."

Why the Distinction Matters for Coding

Simon Willison captured why the naming matters: "Context engineering captures the fact that previous responses from the model are a key part of the process, while 'prompt engineering' suggests only user prompts matter."

For coding agents specifically, this distinction is critical. When Claude Code generates your authentication module, the quality depends on:

1. What CLAUDE.md says about your security requirements
2. What the agent explored in your codebase before writing
3. What context survived compaction from earlier in the session
4. What subagents found when analyzing related files
5. What rules in .claude/rules/ constrain behavior for auth-related files

A perfectly crafted prompt cannot compensate for poor context architecture. That is the core insight driving the industry shift.

The Context Hierarchy: Four Layers of Agent Memory

Claude Code's official documentation describes a 4-level memory architecture with clear priority ordering:

Layer 1: Enterprise Policy

The highest-priority context. Organization-wide constraints that cannot be overridden by project or session context. This is where compliance requirements (HIPAA, SOC 2, GDPR) live as non-negotiable rules.

Layer 2: Project Memory (CLAUDE.md)

This is the layer our Part 3 guide covered in depth. CLAUDE.md files act as persistent project rules that Claude Code reads automatically. Arize AI's research proved this layer's power: optimizing CLAUDE.md alone achieved +10% improvement on SWE Bench Lite without changing architecture, tools, or fine-tuning.

The methodology (Prompt Learning) uses RL-inspired optimization:

1. Run Claude Code on training tasks
2. Evaluate with unit tests
3. Get LLM feedback on failures
4. Meta-prompt suggests CLAUDE.md modifications
5. Iterate until accuracy stabilizes

The full methodology and code is open-source.

Layer 3: Project Rules (.claude/rules/)

More granular than CLAUDE.md. These are Markdown files in .claude/rules/ that target specific file patterns using globs. For example:

• auth-rules.md applies to src/auth/**/*.ts
• migration-rules.md applies to supabase/migrations/*.sql
• api-rules.md applies to src/app/api/**/*.ts

This prevents your authentication security rules from cluttering the context when Claude is editing a blog component. Context relevance matters.

Layer 4: Conversation History

The most volatile layer. Conversation history includes your prompts, Claude's responses, tool outputs, and everything generated during the session. This is where context rot becomes a real problem.

Context Rot: The Silent Quality Killer

Chroma Research tested 18 state-of-the-art models (GPT-4.1, Claude 4, Gemini 2.5, Qwen3) and found a counterintuitive result: adding more context often makes AI output worse.

Key findings:

• Adding full conversation history (~113k tokens) can drop accuracy by 30% compared to a focused 300-token version
• Models performed better on shuffled haystacks than logically structured ones
• Performance degradation is highly task-dependent
• Model reliability decreases significantly with longer inputs, even on simple tasks

A separate study by Norman Paulsen (published January 2026) found that the Maximum Effective Context Window (MECW) differs drastically from advertised limits. Some top-performing models failed with as few as 100 tokens, and most showed severe accuracy degradation by 1,000 tokens.

What This Means for Coding Sessions

If you have been working with Claude Code for two hours on a complex feature, the accumulated context (file reads, edits, test outputs, error messages) may be actively degrading output quality. The model is not getting tired; it is getting buried in irrelevant information.

This is why Claude Code implements auto-compaction at 98% of the effective context window. The process:

1. Clears older tool outputs first (least valuable)
2. Summarizes remaining conversation
3. Reinitiates with the compressed context

You can also trigger this manually with the /compact command at any time. If you notice quality dropping mid-session, compacting is often the fastest fix.

Memory Patterns That Actually Work

Anthropic's context engineering guide describes three core memory strategies for coding agents:

1. Compaction (Short-Term Memory Management)

Compaction is "taking a conversation nearing the context window limit, summarizing its contents, and reinitiating a new context window with the summary." It serves as the first lever in context engineering for better long-term coherence.

Claude Code now maintains continuous session memory in the background, making compaction "instant" rather than requiring a pause while the model summarizes.

2. Structured Note-Taking (Agentic Memory)

"A technique where the agent regularly writes notes persisted to memory outside of the context window. These notes get pulled back into the context window at later times."

In practice, this looks like Claude Code creating a to-do list, or your custom agent maintaining a NOTES.md file. The key insight: memory stored in files outlasts any single context window.

Letta's benchmarking research validated this approach with hard numbers: agents using simple filesystem storage achieved 74.0% accuracy on the LoCoMo benchmark, outperforming Mem0's graph-based approach at 68.5%. Their conclusion: "Memory is more about how agents manage context than the exact retrieval mechanism used."

3. Just-in-Time Context (Dynamic Loading)

Agents maintain lightweight identifiers (file paths, stored queries, web links) and use these references to dynamically load data into context at runtime. Instead of keeping everything in memory, the agent knows where to look and loads data on demand.

This is why Claude Code's Explore subagent exists. Rather than loading your entire codebase into context, Claude spawns a read-only Explore agent to search and analyze files, returning only the relevant findings to the main conversation.

Multi-Agent Context: Isolation as a Strategy

The LangChain State of Agent Engineering survey (1,340 respondents, late 2025) found that 57% of teams have agents in production, with one-third citing quality as their primary blocker. Context management is a significant part of that quality challenge.

Claude Code addresses this through subagents that operate in isolated context windows:

• Explore agent: Read-only codebase analysis with adjustable thoroughness (quick, medium, very thorough)
• Plan agent: Architectural planning without cluttering the main context
• Custom subagents: User-defined agents with specific tool constraints and system prompts

The VentureBeat coverage of Claude Code's Tasks feature describes how these subagents coordinate through directed acyclic graphs (DAGs): Task 3 (Run Tests) cannot start until Task 1 (Build API) and Task 2 (Configure Auth) complete.

Why Isolation Beats Accumulation

When Claude explores 15 files to understand your authentication flow, those file contents do not need to persist in the main conversation. The Explore agent reads them in its own context window, synthesizes the findings, and returns a concise summary. Your main context stays clean.

This directly combats the context rot problem. Instead of loading everything into one increasingly degraded context, you distribute work across focused contexts that each maintain high accuracy.

The Research-Backed Case: Bain, Arize, and Beyond

Bain & Company: The Lifecycle Gap

Bain's 2025 Technology Report found that writing and testing code accounts for only 25-35% of time from idea to launch. Teams seeing just 10-15% productivity boosts are optimizing only the coding step. Organizations achieving 25-30% gains address the entire lifecycle.

Context engineering is how you address the full lifecycle. Your CLAUDE.md encodes not just coding standards but deployment procedures, testing expectations, documentation requirements, and review processes. The developer collaboration framework covers the human side of this lifecycle: which roles own which gates, and why those roles still need juniors who would have caught this before AI could ship it.

Arize AI: The Prompt Learning Breakthrough

The Arize research bears repeating because of its implications. Optimizing a single file (the system prompt) achieved:

• +5.19% improvement (by-repo test split)
• +10.87% improvement (in-repo test split)
• Previous work on Cline showed 15% boosts, bringing GPT-4.1 up to Sonnet 4.5-level accuracy

As Arize CEO Aparna Dhinakaran noted: "We optimized Claude Code's system prompt, just its prompt, and achieved +10% boost on SWE Bench."

The implication: context engineering may be the highest-leverage investment in AI code quality, outperforming tool upgrades, model switching, or architectural changes.

Technology.org: 220K Lines of Clean Code

A 15-week research program with 2 part-time developers produced roughly 220k lines of clean TypeScript and 78 features using an AI-native architecture built on structured context. Their system used two interconnected context layers: a declarative rulebook encoding repository structure and security patterns, plus a native Repo MCP server exposing live project knowledge as tools and resources.

Practical Context Engineering Playbook

Step 1: Audit Your Context Architecture

Map your current context layers:

• Do you have a CLAUDE.md? If not, start with the production guide in Part 3.
• Are you using .claude/rules/? File-pattern rules keep context relevant. An auth rule should not load when editing a blog post.
• How often do you compact? If sessions run over an hour, you should be compacting proactively.
• Are you using subagents? Exploration tasks should not accumulate in your main context.

Step 2: Optimize CLAUDE.md for Your Codebase

The Arize Prompt Learning approach is reproducible:

1. Extract 20-30 representative tasks from your actual backlog
2. Run Claude Code with your current CLAUDE.md on all tasks
3. Track failures: wrong API usage, missed edge cases, security flaws
4. Use LLM analysis to generate CLAUDE.md improvements
5. Test improvements on a held-out set of 10 tasks
6. Iterate until accuracy stabilizes

The open-source implementation is available for replication.

Step 3: Implement File-Pattern Rules

Create .claude/rules/ files for your critical domains:

• Security rules targeting auth, payment, and data export files
• Database rules targeting migration files with RLS requirements
• API rules targeting route handlers with rate limiting and validation requirements
• Test rules targeting test files with coverage and assertion requirements

Step 4: Build Session Hygiene Habits

• Start complex tasks with a clear task description (not "fix the bug")
• Use /compact before pivoting to a different feature
• Delegate exploration to subagents instead of reading files in the main context
• After long sessions, consider starting a fresh session with a summary of decisions made

Step 5: Connect to Governance

Context engineering provides the input quality for AI-generated code. Thread-based engineering provides the output verification. Together, they form the complete governance loop:

1. Context engineering ensures the AI has the right information, constraints, and patterns
2. Thread-based engineering ensures humans verify the output at critical checkpoints
3. CI/CD quality gates (covered in Part 3) automate what can be automated

Without context engineering, your governance catches problems too late. Without governance, great context engineering still produces unchecked output. You need both.

What Comes Next: The Context Engineering Frontier

The academic research is accelerating. A paper titled "Everything is Context" (UNSW, December 2025) proposes file-system abstractions for context engineering inspired by Unix's "everything is a file" philosophy. Whether a resource is a knowledge graph, memory store, or human-curated note, it can be represented through a standardized file interface.

"Memory in the Age of AI Agents" proposes three evolutionary stages of agent memory: Storage, Reflection, and Experience. Systems can prevent context drift by "summarizing or rewriting old entries when new evidence appears," maintaining bounded memory with lower hallucination rates.

The trajectory is clear: context engineering is becoming a proper engineering discipline with research foundations, benchmarks, and best practices. Teams investing in it now are building institutional knowledge that compounds with every project.

Conclusion: Context Is the New Code

The vibe coding technical debt crisis is fundamentally a context engineering failure. Teams that accept AI output without structuring what the AI sees get unpredictable, unmaintainable code. Teams that invest in context architecture get compounding quality improvements across every line.

The research backs this up: +10% from CLAUDE.md optimization alone, 30% accuracy drop from unmanaged context accumulation, 74% accuracy from simple filesystem memory patterns. Context engineering is not theoretical. It is measurable, reproducible, and the highest-leverage investment you can make in AI-assisted development.

Ready to implement context engineering in your development workflow?

• Full-Stack AI Development - We build with context-first architecture by default
• Contact Us - Let us help you structure your AI development workflow