Session 4 / 6 — Optimization

Prompt Caching & Memory Types

Prompt caching drastically reduces costs on long conversations. The 4 memory types are a recurring exam question — each has a precise use case.

Exercise 1

Prompt Caching — TTL, cost, and optimal structure

Objective: understand exactly how the cache works and how to structure a prompt to take advantage of it.

1

Explore the caching rules:

💬 Type in Claude Code
Explain Anthropic prompt caching precisely: 1. What is the cache TTL (Time To Live)? 2. What is the impact on cost (% reduction for cache hits)? 3. What type of content should be placed AT THE START of the prompt to maximize hits? 4. What invalidates the cache? 5. Give a concrete example of a poorly structured vs well-structured prompt for caching
✅ Key answers TTL = 5 minutes. Cache reads ≈ 10% of normal cost. Static content first (system prompt, docs, instructions). Cache invalidated if content changes or TTL expires. Dynamic content (user message) goes at the end.
2

Build an optimal prompt for caching:

💬 Type in Claude Code
A developer is building a support chatbot with: - A 5000-token system prompt (instructions + product docs) - A variable conversation history (50-200 tokens) - A user question (20-50 tokens) Draw the optimal order of elements in the API call to maximize cache hits. Explain why this order and what would happen if we put the history BEFORE the system prompt.
✅ Optimal order 1. Static system prompt (cached). 2. Static docs/context (cached). 3. History (variable, never cached). 4. Question (variable). The cache applies to the common prefix — anything that changes after it invalidates the cache for what follows.
⚠️ Exam Trap

"The cache persists indefinitely if you don't change the content" → FALSE. The TTL is 5 minutes. If more than 5 minutes elapse between two calls with the same content, the cache expires and the next call pays full cost.

Exercise 2

The 4 Memory Types — Understanding and distinguishing them

Objective: master the 4 memory types, their persistence, and the use cases for each. Frequently asked as a multi-select question.

In-Context

Current context window. Temporary, disappears at the end of the conversation.

External

External database (vector DB, SQL). Persistent, queryable via tools.

In-Weights

Model fine-tuning. Permanent, embedded in model parameters.

In-Cache

Prompt caching. Temporary (TTL 5 min), reduces costs on repeated static content.

1

Deepen the 4 types with a practical angle:

💬 Type in Claude Code
For the 4 Claude memory types (in-context, external, in-weights, in-cache), create a table with for each: - Persistence duration - Read cost (tokens / latency) - Ideal use case - Main limitation - Does the content persist between conversations?
2

Classification exercise: which memory type for each need?

💬 Type in Claude Code
For each need, which memory type to use and why? 1. An AI assistant that must remember a specific user's preferences between sessions (e.g. preferred language, tone) 2. A code review agent that needs to know company standards for ALL its analyses 3. The system instructions of a support chatbot sent on every API call 4. A company's 1M FAQ database, queryable via semantic search 5. A model specialized in medical language trained on millions of clinical cases
✅ Expected answers 1→External (persists between sessions). 2→In-Cache or In-Context (reused often in session). 3→In-Cache (static, repeated). 4→External (vector DB). 5→In-Weights (domain fine-tuning).
💡 In-Cache = the often-forgotten memory type

Prompt caching is officially classified as an "in-cache" memory type. It is temporary memory (TTL 5 min) that persists in Anthropic's infrastructure, not in the model. Different from in-context (active window) and in-weights (model parameters).

Exercise 3

In-Context vs External — When to switch to an external DB

Objective: understand the limits of in-context memory and know when to switch to an external solution (RAG, vector DB).

1

Analyze limits and transitions:

💬 Type in Claude Code
A developer is building an AI assistant for a company. When should they switch from in-context memory to external memory? Identify 4 signals that indicate in-context is no longer sufficient. For each signal, propose an appropriate external architecture.
✅ Expected signals Data > context window (200k tokens), need for persistence between sessions, multiple users with distinct memories, data that changes frequently (recomputing context is expensive).
2

Practical case: memory architecture for a complex agent:

💬 Type in Claude Code
A technical support AI agent must: - Know the product documentation (500 pages, updated monthly) - Remember a user's previous tickets (max 50 tickets) - Have its system instructions (2000 tokens, fixed) - Access the known bugs database (10,000 entries) Propose an architecture that uses all 4 memory types optimally. Justify each choice.
💡 Typical architecture

In-cache: system instructions (static, called frequently). In-context: active conversation + retrieved relevant context. External: product docs (vector search), user history (SQL). In-weights: domain specialization if extreme performance is needed.

← Previous Session: Tool Use Next Session: MCP & Skills →