Context Management

Context is the information you provide to help the AI understand your request. Too little context leads to generic responses; too much causes confusion and wasted tokens. Mastering context management is essential for effective AI collaboration.

The Context Spectrum

Too Little                    Optimal                    Too Much
    |---------------------------|---------------------------|
Generic/wrong           Accurate/relevant           Confused/slow
responses                    output                    responses

The "Lost in the Middle" Effect

Research shows that LLMs struggle with information in the middle of long contexts. Important details placed between the beginning and end may be ignored.

Best Practice: Put critical information at the beginning or end of your prompt, not buried in the middle.

[CRITICAL: Main instruction here]

[Supporting context...]
[More details...]
[Additional information...]

[IMPORTANT: Key constraints here]

What to Include

Essential Context

Context Checklist

Before sending a prompt, verify you've included:

• Programming language and version
• Framework and version
• Relevant type definitions/interfaces
• Import statements that matter
• Existing patterns in your codebase
• Specific constraints or requirements

What to Exclude

Harmful Context

The Minimal Context Principle

Include only what's necessary to answer the question correctly. Ask yourself: "If I removed this, would the answer change?"

Too much context:

Here's my entire user service (500 lines), authentication module (300 lines), and database schema (200 lines). How do I add password reset?

Right amount:

I need to add password reset to my user service.

Current relevant code:

// User model fields
interface User {
  id: string;
  email: string;
  passwordHash: string;
}

// Current auth methods
class AuthService {
  async login(email: string, password: string): Promise<User>
  async register(email: string, password: string): Promise<User>
}

Stack: Node.js, TypeScript, PostgreSQL, nodemailer for emails.

Add password reset with:

• Token generation
• Email sending
• Token validation and password update

Context Layering Strategy

Organize context in layers, from most to least important:

Layer 1: Core Request
What you actually want

Layer 2: Immediate Context
Code directly related to the task

Layer 3: Architectural Context
Patterns, conventions, dependencies

Layer 4: Background Context
Nice-to-have information

Example:

Layer 1 - Core Request: Add input validation to this form handler.

Layer 2 - Immediate Context:

export const submitForm = async (data: FormData) => {
  const response = await api.post('/submit', data);
  return response;
};

Layer 3 - Architectural Context: We use Zod for validation throughout the codebase. Errors should follow our standard format: { field: string, message: string }[]

Layer 4 - Background Context: This form is part of our checkout flow. It handles shipping addresses.

Working with Context Windows

Context Window Limits

Different models have different context limits:

• GPT-4o: 128K tokens
• Claude 3.5 Sonnet: 200K tokens
• Gemini 1.5 Pro: Up to 2M tokens

Estimating Token Usage

Rough estimates for code:

• 1 token ≈ 4 characters
• 1 line of code ≈ 10-20 tokens
• 100 lines of code ≈ 1,000-2,000 tokens

Context Budget Strategy

Allocate your context budget intentionally:

Total Budget: 100%

- Instructions: 10%
- Core code to modify: 30%
- Related interfaces/types: 20%
- Examples of desired output: 15%
- Constraints and requirements: 10%
- Buffer for response: 15%

Providing File Context

Single File

Example prompt:

File: src/services/UserService.ts

// [file contents]

Focus on the createUser method for this task.

Multiple Related Files

Example prompt:

I need help with a bug spanning multiple files.

File 1: src/hooks/useAuth.ts

// [relevant portions only]

File 2: src/context/AuthContext.tsx

// [relevant portions only]

The issue is in how these two interact.

Referencing Without Including

My project structure:
- /src/components/  (React components)
- /src/services/    (API layer)
- /src/utils/       (Helper functions)

I'm working in /src/services/OrderService.ts which calls functions from /src/utils/pricing.ts.

Here's the relevant code from both:
[only the relevant functions]

Context for Different Task Types

Code Generation

Minimal context needed:

Tech: TypeScript, React 18, Tailwind CSS

Create a button component that:

• Accepts variant (primary, secondary, danger)
• Has loading state
• Is accessible

Existing pattern to follow:

// How we structure components
export const Input = ({ label, ...props }) => (
  <div className="flex flex-col gap-1">
    <label>{label}</label>
    <input {...props} className="border rounded p-2" />
  </div>
);

Debugging

[More context needed]

Full error with stack trace:
[paste complete error]

Relevant code path:
1. Entry point: [code]
2. Calls: [code]
3. Where error occurs: [code]

Environment: Node 20, TypeScript 5.3, PostgreSQL 15

Recent changes: Updated prisma from 5.0 to 5.5

Code Review

[Context about standards]

Review this PR against our team guidelines:

Guidelines:
- All functions must have JSDoc
- Error handling required for async operations
- No magic numbers

Code to review:
[PR diff or code]

Incremental Context Provision

For complex tasks, provide context incrementally:

Turn 1:

I need help refactoring a large module. First, let me show you the current structure:
[high-level overview]

What questions do you have before we proceed?

Turn 2:

Good questions. Here are the answers:
- [answer 1]
- [answer 2]

Now here's the specific function to refactor:
[code]

Turn 3:

One more piece of context - here's how this integrates:
[integration code]

Now please suggest the refactoring approach.

Common Context Mistakes

1. Context Dump

Bad: Pasting 10 files without guidance Good: Curating relevant portions with clear markers

2. Missing Imports

Bad: Code snippet without imports, leaving AI to guess Good: Include critical imports that affect the solution

3. Outdated Context

Bad: Including old code that's been refactored Good: Ensure context reflects current codebase state

4. Assumed Knowledge

Bad: "Fix the auth bug" (which bug?) Good: Specific description with reproduction context

Context Compression Techniques

When you need to include a lot of information:

Summarize Long Files

This 500-line file handles user management. Key sections:

Lines 1-50: Imports and types
Lines 51-150: CRUD operations (createUser, getUser, updateUser, deleteUser)
Lines 151-300: Authentication (login, logout, refreshToken)
Lines 301-500: Admin operations

Here's the specific section I need help with:
[paste only that section]

Use Pseudo-Code for Context

The calling code (simplified):

1. Get user from database
2. Validate permissions
3. [FUNCTION TO WRITE] <- this is what I need
4. Update audit log
5. Return result

Then provide the actual code for step 3.

Reference Without Including

This function will be called by our Express router at POST /api/users.
Request body is validated by Zod schema (not shown, but you can assume valid types).
Response must match our standard API format: { success: boolean, data?: T, error?: string }

Practice Exercise

Take a complex task and prepare context for it:

1. Identify all potentially relevant code
2. Filter to only what's essential
3. Layer the context by importance
4. Position critical info at beginning/end
5. Estimate token usage

Task example: "Add caching to an API endpoint"

What context would you include vs exclude?

Summary

• Include essential context: tech stack, relevant code, constraints
• Exclude: sensitive data, unrelated code, entire files
• Layer context by importance
• Put critical info at beginning or end (avoid the middle)
• Estimate tokens and budget appropriately
• Provide context incrementally for complex tasks

Next Steps

Now let's explore prompt templates and patterns—reusable structures you can use across different tasks.