The AI Product Manager's Field Manual

How to use this book

This is a manual, not a novel. It is written to be studied in the same order as your 16-week roadmap, one chapter roughly every two weeks — but each chapter also stands alone, so you can jump to whatever you're building this week.

You are not starting from zero. You have more than a decade of product management behind you — strategy, discovery, roadmaps, analytics, SQL. This book deliberately skips the generic "what is a neural network" material that fills most AI courses. It teaches the layer you actually need: enough of how the technology works to make good product decisions, and the judgment to know when a decision is good.

Read actively. Every chapter carries five kinds of margin note, and they are the point of the book as much as the prose:

At the end of each chapter is a short self-check. If you can answer those questions in your own words without notes, move on. If you can't, that's your signal to re-read — or better, to go build the thing.

The appendices are the part you'll come back to most: a bank of interview questions with model answers, reusable templates (an AI PRD, an evaluation plan, a model-selection scorecard), a glossary, and a short, curated reading list. Keep them close.

What an AI PM actually is

Before any tokens or embeddings, get the job description right — because most people, including most job posts, get it wrong.

There is a myth that to manage AI products you must become a machine-learning engineer. You don't. The companies hiring strong AI PMs are not looking for someone who can derive backpropagation; they have research scientists for that. They are looking for a product manager who is fluent in what the technology can and cannot do, who can specify behavior precisely, who insists on evaluation, who can prototype an idea before asking an engineer to build it, and who exercises judgment about where AI belongs and where it doesn't.

The five things the job adds to normal PM

Everything you already do as a PM still applies. AI adds five competencies on top:

• AI fluency — you understand tokens, context windows, embeddings, retrieval, and agents well enough to reason about cost, latency, and failure without asking an engineer to translate every sentence.
• Evaluation-first thinking — you never ship an AI feature you can't measure. You can answer "how do we know it got better?" with a dataset and a number, not a vibe.
• Prompt & prototype ability — you can build a working proof-of-concept yourself, in an afternoon, to de-risk an idea before it hits a sprint.
• Model & trade-off judgment — you can choose between a rule, a classifier, a large model, and an agent, and defend the choice on cost, quality, latency, and risk.
• Responsible-AI instinct — you think about hallucination, privacy, bias, and governance as product requirements, not legal afterthoughts.

The one skill that separates senior from junior

If you take one strategic bet from this whole book, make it this: get exceptional at AI evaluation. Almost every PM can wire up a demo that looks impressive in a meeting. Very few can tell you whether version B is actually better than version A across a thousand real cases, at what cost, with what regressions. That gap — the ability to turn "it feels better" into "faithfulness went from 0.72 to 0.88 with no latency regression" — is the clearest signal of a senior AI PM. Chapter 4 is the longest in this book for that reason.

A mental model for the whole stack

Here is the map you'll fill in over the next eight chapters. An AI feature is a pipeline: something turns a user's intent into a prompt; often you retrieve relevant context to ground it; a model generates a response; sometimes the model calls tools or runs as an agent across several steps; and around all of it sits evaluation, guardrails, and product design that decide what the user actually sees and trusts. Strategy — cost, build-vs-buy, governance — wraps the whole thing.

AI Foundations

How large language models actually work — to exactly the depth a product manager needs, and no further. By the end you can reason about cost, latency, and failure without a translator, and you can decide when the answer is not AI at all.

1.1 What a language model is doing

A large language model (LLM) has one core skill: given a stretch of text, predict what comes next. That's it. Everything impressive — writing code, summarizing a contract, answering a question — is that one skill applied at enormous scale. The model was trained on a vast amount of text and learned the statistical shape of language and, embedded in it, a great deal of world knowledge. It does not "look things up" and it does not reason like a database. It generates the most plausible continuation, one piece at a time.

1.2 Tokens — the unit of everything

Models don't see words or characters; they see tokens — chunks of text roughly ¾ of a word on average. "Product manager" might be three tokens; a rare word or a code symbol might be several. Tokens matter to you for three very practical reasons:

• You pay per token, both for the tokens you send (input) and the tokens the model generates (output). Cost is literally a token count times a price.
• Latency scales with output tokens — the model generates them one at a time, so a long answer is a slow answer.
• The context window is measured in tokens — the model can only "see" so many at once.

1.3 The context window

The context window is the maximum number of tokens the model can consider at once — the prompt you send plus the answer it generates must fit inside it. Modern models range from tens of thousands to well over a million tokens. A bigger window lets you include more source material, but it isn't free: more input tokens cost more and add latency, and models can lose track of details buried in the middle of a very long context (the "lost in the middle" effect). Bigger is not automatically better.

1.4 Embeddings — meaning as coordinates

An embedding is a list of numbers (a vector) that represents the meaning of a piece of text as a point in high-dimensional space. Texts with similar meaning land near each other, even if they share no words: "How do I cancel my plan?" and "I want to end my subscription" sit close together. This is the foundation of semantic search and of RAG. You don't compute embeddings by hand — a model produces them — but you must understand the concept, because "find the most relevant chunks" (Chapter 3) is really "find the nearest vectors."

1.5 The knobs: temperature and top-p

When the model generates each token, it has a probability distribution over what could come next. Two settings shape how it picks:

• Temperature — how much randomness. Low (near 0) makes the model pick the most likely token almost every time: consistent, predictable, good for extraction, classification, and anything factual. High (0.8–1.2) makes it sample more adventurously: good for brainstorming and creative copy, bad for anything you need to be reliable.
• Top-p (nucleus sampling) — instead of considering every possible token, only consider the smallest set whose probabilities add up to p (say 0.9), then sample from those. It's another way to trade off focus versus variety.

1.6 Hallucinations

A hallucination is when the model generates something fluent, confident, and false. It follows directly from Section 1.1: the model is producing a plausible continuation, and plausible is not the same as true. Hallucinations are not a bug to be fully patched out; they are an inherent property you manage. The main levers: ground the model in real source text (retrieval), lower temperature, ask it to cite or say "I don't know," and — crucially — evaluate for it (Chapter 4). You reduce and detect hallucinations; you don't eliminate them.

1.7 Reasoning models

A newer class of reasoning models is trained to "think" before answering — generating a chain of intermediate steps (often hidden) before the final response. They're markedly better at math, logic, coding, and multi-step planning, but they cost more and are slower, because that thinking is extra tokens and time. As a PM your job is to route: use a reasoning model for the genuinely hard, multi-step problems, and a fast standard model for the 90% of requests that are simple. Paying reasoning prices for "summarize this email" is waste.

1.8 Function calling / tool use

Function calling (or tool use) lets a model do more than talk: you describe some functions it's allowed to call — get_order_status(order_id), search_docs(query) — and the model, instead of answering directly, can emit a structured request to call one. Your code runs the function and hands the result back, and the model continues. This is the bridge from "chatbot" to "software that acts": it's how a model checks a live order, books a slot, or queries a database. It is also the foundation of agents (Chapter 6) and the reason MCP (Chapter 7) exists.

1.9 The model landscape

You don't need loyalty to one vendor; you need a map. Broadly:

The important axis for you is closed vs open. Closed models (GPT, Claude, Gemini) are accessed through an API — easiest to start with, top-tier quality, but your data leaves your walls and you pay per token. Open models (Llama, Mistral, DeepSeek) can be self-hosted — more control, data stays home, no per-token vendor bill, but you own the infrastructure and the quality/ops burden. We return to this decision in Chapter 8.

1.10 The decision that matters most: when NOT to use AI

The most valuable judgment in this chapter is knowing that an LLM is often the wrong tool. Walk down this ladder before reaching for a model:

Prompt Engineering

The prompt is the product spec the model actually reads. Prompt engineering is the discipline of writing that spec precisely, versioning it, and testing it — treating a prompt like code, not like a lucky incantation.

2.1 The anatomy of a prompt

A production prompt almost always has structure. The two most important parts:

• System prompt — the standing instructions that define the model's role, rules, tone, and constraints. It's set by you, the builder, and applies to the whole conversation: "You are a support assistant for Acme. Only answer from the provided documents. If the answer isn't there, say you don't know."
• User prompt — the actual request from the end user, or the specific task for this call.

Getting the system prompt right is most of the battle. It's where you encode behavior, guardrails, and format requirements once, instead of repeating them every turn.

2.2 Few-shot prompting

Zero-shot is asking with no examples. Few-shot is including a handful of worked examples of input → desired output right in the prompt. Models are excellent imitators, so two or three good examples often outperform paragraphs of instructions. If you want output in a specific format, or edge cases handled a certain way, show it rather than describe it.

2.3 Chain-of-thought

Chain-of-thought prompting asks the model to reason step by step before answering ("Think through this step by step, then give your answer"). For multi-step problems — math, logic, anything with intermediate reasoning — this measurably improves accuracy, because the model uses its own generated steps as scaffolding. The trade-off is more tokens (cost, latency), and for reasoning models (Section 1.7) it's partly built in. Use it where correctness on hard problems matters; skip it for simple lookups.

2.4 Structured / XML prompting

Models handle clearly delimited prompts far better than walls of text. Wrapping sections in tags — <instructions>, <context>, <example> — removes ambiguity about where one part ends and another begins, and makes it easy to ask for structured output back. This "XML prompting" style is especially reliable with some model families and is a good default for anything complex.

<role>You extract structured data from support tickets.</role>
<rules>
- Return valid JSON only.
- If a field is missing, use null.
</rules>
<ticket>{{ticket_text}}</ticket>

2.5 Tool calling in prompts

Building on Section 1.8, tool calling is often the difference between a prompt that talks about doing something and one that does it. In the prompt/config you declare the available tools and their parameters; the model decides when to call them. As a PM, the design questions are: which tools, with what permissions, and what the model should do when a tool errors or returns nothing.

2.6 Prompt versioning & evaluation prompts

Here is where amateurs and professionals split. A prompt is not something you tweak in a text box and forget — it's an artifact that changes behavior for every user, so it must be versioned (tracked, diffable, roll-back-able) and tested before it ships. When you change a prompt, you run it against a fixed set of cases and check that you improved the target behavior without regressing others. This is the bridge to Chapter 4, and it's why "prompt engineering" and "evaluation" are really one discipline.

A special category is the evaluation prompt: a prompt whose job is to judge another model's output ("Given this question and this answer, is the answer supported by the context? Reply PASS or FAIL with a reason"). This is "LLM-as-judge," and you'll meet it properly in Chapter 4.

Retrieval-Augmented Generation (RAG)

The single most important architecture pattern for enterprise AI PMs. RAG is how you make a general model answer accurately about your data — your PRDs, your policies, your knowledge base — without retraining it.

3.1 The problem RAG solves

An LLM only knows what it saw in training. It doesn't know your company's return policy, last week's incident report, or the PRD you wrote yesterday — and if you ask, it will happily hallucinate a plausible answer. You could stuff everything into the prompt, but that's expensive, hits the context limit, and buries the relevant bit. Retrieval-Augmented Generation solves this: at question time, retrieve only the most relevant pieces of your data and put those in the prompt, so the model answers from real, current source text.

3.2 The pipeline, end to end

There are two phases. Indexing happens ahead of time; retrieval happens per question.

Indexing (done once, then kept fresh)

• Chunking — split your documents into passages (say, a few hundred tokens each). Too big and retrieval is imprecise and costly; too small and each chunk loses context. Chunking strategy is one of the biggest quality levers in RAG.
• Embedding — run each chunk through an embedding model to get its vector (Section 1.4).
• Storing — put those vectors, plus the original text and metadata, into a vector database.

Retrieval (every question)

• Embed the query into the same vector space.
• Search for the nearest chunk vectors — semantic similarity, not keyword match.
• Rank / filter the candidates, optionally re-ranking for relevance and applying metadata filters (e.g. "only this customer's docs").
• Augment & generate — drop the top chunks into the prompt and ask the model to answer from them, ideally with citations.

3.3 Vector databases

A vector database stores embeddings and answers "which vectors are nearest to this one" fast, at scale. You don't need to implement one, but you should recognize the players and that they're broadly interchangeable at the concept level:

3.4 Better retrieval: hybrid search, metadata, re-ranking

• Semantic search finds meaning but can miss exact terms — product codes, names, acronyms. Keyword search nails those but misses paraphrase. Hybrid search combines both and is usually the right default for real corpora.
• Metadata filtering narrows the search before or after similarity — by date, author, department, access permission. This is also how you enforce that users only retrieve documents they're allowed to see.
• Re-ranking takes the top ~50 candidates and runs a more expensive, more accurate model to reorder them, so the best few reach the prompt. A cheap way to lift quality noticeably.

3.5 RAG vs. fine-tuning — the classic question

PMs constantly confuse these. They solve different problems and are often used together.

3.6 Where RAG breaks (so you can design for it)

• Retrieval misses — if the right chunk isn't retrieved, the model can't use it. Most "RAG is wrong" problems are actually retrieval problems, not generation problems. Measure retrieval separately.
• Bad chunking — the answer is split across two chunks and neither is complete.
• Stale index — documents changed but the index didn't. You need a refresh pipeline.
• The model ignores the context and answers from its own memory anyway — mitigated with strict prompting ("answer only from the passages") and evaluated as faithfulness (Chapter 4).

AI Evaluation

The most important chapter in this book. If you master one thing, master this. Evaluation is how "it feels better" becomes "faithfulness went from 0.72 to 0.88 with no latency regression" — and that translation is the clearest marker of a senior AI PM.

4.1 Why evaluation is the whole game

Traditional software is deterministic: given an input, the correct output is known, and a test passes or fails. AI is probabilistic: the same input can yield different, and differently-good, outputs. There's rarely one right answer — a summary can be good in many ways and bad in many others. So you cannot "QA" an AI feature by clicking around. You need a systematic way to measure quality across many cases, or you are flying blind — shipping changes with no idea whether they helped.

4.2 The golden dataset

Everything starts with a golden dataset (also "eval set" or "test set"): a curated collection of representative inputs, each paired with the ideal output or clear criteria for a good one. This is your ruler. When you change a prompt, swap a model, or adjust retrieval, you run it against the golden set and compare scores. Building this set is the highest-leverage work in AI PM — and it is squarely a PM job, because it encodes what "good" means for your product.

• Start small and real: 30–100 cases drawn from actual usage beat 1,000 synthetic ones.
• Include the hard cases, the edge cases, and the "should refuse" cases — not just the happy path.
• Ground truth is the verified-correct answer for a case. Sometimes it's exact; often it's a rubric ("must mention X, must not claim Y").
• Grow it over time: every real failure you find becomes a new row, so the same bug can never silently return.

4.3 Offline vs. online evaluation

• Offline eval runs against your golden set before you ship — a lab test. Fast, cheap, repeatable, catches regressions early.
• Online eval measures real users in production — thumbs up/down, task completion, escalation rate, A/B tests. It's the truth, but it's slow and you only learn after shipping.

You need both. Offline to iterate quickly and gate releases; online to confirm the offline gains are real and to catch what your golden set didn't anticipate.

4.4 How to actually score outputs

Different tasks need different scoring methods. Your toolkit:

LLM-as-judge, the workhorse

Human eval is the gold standard but doesn't scale to thousands of cases per release. So you use a strong model as an automated grader: give it the input, the output, and a rubric, and have it return a score with a reason. It's the practical backbone of modern AI evaluation. The catch — and the senior move — is to validate the judge against human ratings on a sample, so you trust its scores. An un-validated judge is just another opinion.

4.5 The metrics that matter

You don't need every metric; you need the right handful for your feature. The essential vocabulary:

• Precision — of the things the system flagged/returned, what fraction were correct? (Low precision = false positives / noise.)
• Recall — of the things it should have caught, what fraction did it? (Low recall = misses.)
• Precision vs. recall is a trade-off, and which you favor is a product decision: a medical alert wants high recall (never miss), a spam filter wants high precision (never flag good mail).
• Faithfulness / groundedness — is the answer actually supported by the retrieved context? The key metric for RAG; it's how you measure hallucination directly.
• Relevance — does the answer address what was asked?
• Latency — how long to respond. A quality gain that doubles response time may be a net loss.
• Cost — tokens per request times price. Always in the frame.
• Safety — rate of harmful, biased, or policy-violating outputs.

4.6 Regression testing & the eval flywheel

Regression testing means re-running your whole golden set on every change to ensure you didn't break something that used to work. AI is full of whack-a-mole: fixing one case quietly breaks another. Without regression testing you won't notice until users do.

Put it together and you get the evaluation flywheel, the operating rhythm of a good AI team:

1. Ship a version.
2. Collect real failures from production.
3. Add them to the golden set as new cases.
4. Improve the prompt / retrieval / model.
5. Re-run the full set — confirm the fix, catch regressions.
6. Ship again. Repeat.

The team with the best flywheel wins, because their product compounds in quality while competitors guess.

AI Product Design

A model is not a product. The design work is everything around the model: how the user asks, how much they should trust the answer, what happens when the model is wrong — and it's wrong often enough that designing for failure is the job, not an edge case.

5.1 Three interaction patterns

Most AI features fall into one of three shapes, and choosing the right one is an early, consequential design decision:

5.2 Designing for trust

Because the model is sometimes confidently wrong (Section 1.6), your design has to help users calibrate how much to trust any given output. The main tools:

• Confidence signals — communicate uncertainty. Sometimes a numeric score, more often UX: hedged language, "I'm not sure, but…", or surfacing multiple options instead of one false-certain answer.
• Citations & explainability — show the sources or the reasoning, so users can verify. In a RAG product, linking each claim to its source document is one of the highest-trust features you can build.
• Let users verify easily — the faster a user can check an answer, the more useful even an imperfect model becomes.

5.3 Designing for failure

The model will fail. Great AI product design treats failure as a first-class state, not an exception you hope to avoid:

• Guardrails — rules and filters around the model that block bad inputs and outputs: off-topic requests, unsafe content, attempts to make it do things it shouldn't. Guardrails are product requirements.
• Fallbacks — what happens when the model can't answer or confidence is low? A good fallback ("I couldn't find that — here's how to reach support") beats a confident wrong answer every time.
• Human handoff — a clean path to a person when the AI hits its limit. Knowing when to hand off is itself a design decision, and often a metric.
• Graceful "I don't know" — explicitly designing and rewarding the model for admitting ignorance. Counterintuitively, a model that says "I don't know" at the right times is more trusted, not less.

5.4 Memory and context

Conversation memory — how much of the prior conversation the product carries forward — is a design lever with real trade-offs. More memory means more coherent, personalized interactions, but more tokens (cost, latency), more chance of the model fixating on stale context, and privacy questions about what you retain. Deciding what to remember, for how long, and how to let users see and clear it is product design, not just engineering.

5.5 Writing a behavior specification

Traditional PMs write requirements for deterministic behavior. AI PMs write a behavior specification: a description of how the AI should act across situations, including the ambiguous and adversarial ones. What should it do when asked something off-topic? When it doesn't know? When a user is abusive? When it's uncertain? These aren't edge cases to bolt on later — they're the core spec, because they define your product's character and safety. Much of this spec is enforced through the system prompt (Chapter 2) and guardrails, and verified through evaluation (Chapter 4).

Agentic AI

The trend to bet on. An agent is a model that doesn't just answer — it pursues a goal across multiple steps, deciding for itself what to do next and using tools to do it. This is where AI stops being a feature and becomes a worker.

6.1 What makes something an "agent"

A plain LLM call is one-shot: prompt in, answer out. An agent runs a loop. Given a goal, it decides on an action, takes it (often by calling a tool), observes the result, and decides what to do next — repeating until the goal is met or it gives up. The model is the brain; tools are the hands; the loop is what turns thinking into doing.

6.2 The four capabilities

• Planning — breaking a goal into steps, and re-planning when steps fail. The harder the task, the more this matters, and it's where agents most often go wrong.
• Tool / function calling — the agent's ability to act: search, run code, query a database, call an API, send a message (Sections 1.8, 2.5). An agent's power is bounded by the tools you give it.
• Memory — short-term (the current task's working context) and long-term (facts persisted across sessions, often in a vector store — note the link back to RAG).
• Reflection — the agent evaluating its own output and correcting course: "that didn't work, let me try another way." Reflection is what separates a robust agent from one that fails silently.

6.3 Single-agent vs. multi-agent

A workflow is a fixed sequence of steps you design — reliable, predictable, and honestly the right choice more often than people admit. An agent decides its own steps — flexible, but less predictable. Multi-agent systems use several specialized agents that collaborate (a "researcher" hands to a "writer" hands to a "reviewer"). Multi-agent can tackle complex problems but multiplies cost, latency, and failure surface.

6.4 The framework landscape

You don't need to code these, but you should know the map:

6.5 Evaluating agents is harder

Everything from Chapter 4 still applies, but agents add a dimension: you evaluate not just the final answer but the trajectory — did it take sensible steps, use the right tools, avoid loops, recover from errors? An agent can reach a right answer via a reckless path (expensive, slow, lucky) or a wrong answer via a reasonable one. You measure task success rate, steps taken, cost per task, and failure modes. Agents also fail in spectacular new ways — infinite loops, calling the wrong tool, doing something irreversible — which is why permissioning and human-in-the-loop checkpoints are core design, not polish.

MCP — the Model Context Protocol

If agents are workers, MCP is the standard way to plug them into your tools and data. It's the "USB-C for AI" — one protocol so any model can connect to any system without a bespoke integration each time.

7.1 The problem MCP solves

Every useful agent needs to reach real systems — GitHub, Slack, Jira, a database, your docs. Before MCP, each connection was a custom integration: bespoke glue between a specific model and a specific tool, rebuilt for every combination. That doesn't scale. The Model Context Protocol (MCP) is an open standard, introduced by Anthropic and now widely adopted, that defines one way for AI applications to connect to external tools and data. Build an MCP server for your system once, and any MCP-compatible AI can use it.

7.2 The core concepts

• MCP server — a program that exposes a system's capabilities (your database, GitHub, a SaaS app) over the protocol. The tool's side of the plug.
• MCP client / host — the AI application (Claude Desktop, an IDE, your agent) that connects to servers and uses what they offer.
• Tools — actions the server exposes that the model can call: create_issue, run_query, send_message. (The same tool-calling idea from Chapter 1, now standardized.)
• Resources — data the server exposes for the model to read: files, records, documents.
• Prompts — reusable prompt templates a server can provide for common tasks.

7.3 Authentication and remote MCP

Connecting an AI to your Jira or your database raises the obvious question: who's allowed to do what? MCP servers handle authentication so the model acts with appropriate, scoped permissions rather than god-mode access. Remote MCP — servers running in the cloud rather than on your laptop — is what makes this viable for real organizations, with proper auth (OAuth-style flows) so a hosted AI can safely use enterprise systems. For an enterprise AI PM, the auth and permission model isn't a detail — it's the difference between a useful integration and a security incident.

7.4 Why this matters now

MCP, together with agents (Chapter 6), is the bet the whole industry is making: AI that doesn't just chat but acts across your real systems, safely and with standard plumbing. Understanding it puts you ahead of most PMs, who still think of AI as a text box. The ecosystem — GitHub, Slack, Notion, Jira, Azure DevOps, and hundreds more — is growing fast, and "which MCP servers do we expose, with what permissions?" is becoming a real product question.

AI Product Strategy

Everything so far was capability. This chapter is judgment at the portfolio level: choosing models, controlling cost, deciding build vs. buy, and justifying AI with business outcomes rather than novelty. This is where you sound like a leader, not a hobbyist.

8.1 The central trade-off triangle

Nearly every AI product decision is a balance of three forces, and you usually can't max all three:

The art is matching the point in this triangle to the use case. A customer-facing legal answer wants quality and will tolerate cost and latency. An autocomplete suggestion wants speed above all. There is no universally "best" model — only the best fit for a given job at a given price.

8.2 Open vs. closed models

Rule of thumb: start with a closed API to validate the product fast; consider open/self-hosted when data-residency demands it, or when volume is high enough that per-token costs justify running your own infrastructure.

8.3 The build-vs-buy ladder

For any AI capability, climb from cheapest/fastest to most expensive/custom, and stop as soon as it's good enough:

1. Prompt engineering on a hosted model — cheapest, fastest, try first.
2. RAG — add your knowledge without training. The default for "answer about our data."
3. Fine-tuning — when you need consistent behavior prompting can't get.
4. Training your own model — almost never the right answer; enormous cost and expertise for gains a combination of the above usually delivers.

8.4 ROI, risk, and unit economics

Enterprises don't fund technology; they fund outcomes. Frame every AI investment in business terms:

• Value — what outcome, quantified? Hours saved, deflected tickets, faster cycle time, revenue. Tie it to a metric leadership already cares about.
• Unit economics — cost per request × volume vs. value per request. An AI feature that costs more per use than it returns doesn't scale, however impressive. Know your per-request token cost and your break-even.
• Risk — hallucination in a high-stakes context, data leakage, bias, regulatory exposure, reputational damage. Weigh probability × impact, and design mitigations in (Chapter 5).

8.5 Governance and Responsible AI

For enterprise AI PMs, responsibility is a requirement, not a nice-to-have:

• Data governance — what data trains or feeds the system, where it lives, who can access it, and how retention and privacy are handled.
• Security — prompt injection (malicious instructions hidden in inputs), data exfiltration through tools, over-broad agent permissions. Treat model inputs as untrusted.
• Fairness & bias — models can reproduce and amplify bias in training data; test for it across user groups as part of evaluation.
• Transparency & accountability — users should know when they're talking to AI; someone owns what it does.

The AI PM interview — question bank

The questions you'll actually be asked, grouped by theme, with the shape of a strong answer. Rehearse these out loud until they're yours. The full model answers live in the chapters; here's the map and the compressed versions.

Fluency & fundamentals

• Why use an LLM instead of classical ML? — Unstructured language input + need for generation/flexibility vs. labeled data + need for a cheap, deterministic score. (Ch 1)
• When should you NOT use AI? — When a rule or lookup solves it deterministically and auditably; an LLM would be slower, costlier, and non-deterministic. (Ch 1)
• Why do models hallucinate? — They predict plausible next tokens, and plausible ≠ true; you manage it with grounding, low temperature, citations, and evaluation. (Ch 1)
• Explain tokens, context window, and why they matter to cost. — Unit of processing and billing; the window caps what fits; input tokens are paid every call. (Ch 1)

Architecture & building

• What is RAG and when do you use it? — Retrieve relevant passages, then generate from them; use it to answer over your own changing data with citations. (Ch 3)
• RAG vs. fine-tuning? — RAG for knowledge, fine-tuning for behavior; start with prompting. (Ch 3)
• Your RAG bot is wrong — how do you debug? — Separate retrieval from generation; check whether the right chunk was retrieved before blaming the model. (Ch 3)
• Agent vs. workflow? — Workflow when steps are fixed (more reliable/cheaper); agent only when dynamic multi-step decisions are genuinely needed, and then constrained. (Ch 6)
• What is MCP? — Open standard connecting models to tools/data; USB-C for AI; makes enterprise agents practical with scoped auth. (Ch 7)

Evaluation & judgment (expect the most depth here)

• How do you know your AI is better? — Golden dataset + offline eval + LLM-as-judge validated against humans + faithfulness/latency/cost together + regression testing + online A/B. (Ch 4)
• What is a golden dataset and who owns it? — Curated representative cases with ground truth; the PM owns it because it defines "good." (Ch 4)
• Precision vs. recall — which matters more? — Depends on the cost of a miss vs. a false alarm; it's a product decision. (Ch 4)
• How do you design for the model being wrong? — Calibrated trust, guardrails, fallbacks, human handoff, a good "I don't know," specified up front. (Ch 5)

Strategy & leadership

• How do you pick a model? — Per use case against the quality/latency/cost triangle; route by difficulty. (Ch 8)
• Build vs. buy? — Climb the ladder: prompt → RAG → fine-tune → (almost never) train your own. (Ch 8)
• How do you justify an AI investment? — Business outcome + unit economics (value per request vs. cost per request at volume) + risk with mitigations. (Ch 8)
• How do you handle Responsible AI? — Data governance, security (prompt injection), bias testing, transparency — as product requirements. (Ch 8)

Templates you'll reuse

Three artifacts that turn the concepts in this book into working documents. Copy them, fill them in, and you're producing the output an AI PM is hired to produce.

B.1 The AI PRD outline

A normal PRD with three AI-specific sections bolted on (marked ★):

1. Problem & user      — who, what pain, why now
2. Why AI              — why AI over a rule/ML; what breaks without it
3. Success metrics     — business outcome + the quality metric(s)
4. Solution & UX       — pattern (chat / copilot / agent), key flows
★ 5. AI behavior spec  — tone, refusals, uncertainty, off-topic,
                         adversarial, "I don't know" behavior
★ 6. Evaluation plan   — golden dataset, metrics, pass bar,
                         offline + online, regression strategy
★ 7. Failure & risk    — hallucination, fallbacks, human handoff,
                         guardrails, privacy/security, mitigations
8. Model & cost        — model choice + trade-off rationale,
                         per-request cost, latency target
9. Rollout             — phasing, monitoring, kill switch

B.2 The evaluation plan (one page)

Feature:            ______________________________
What "good" means:  ______________________________ (the rubric)
Golden dataset:     __ cases · sources: real usage / edge / refusal
Metrics tracked:    quality (how scored?) · faithfulness ·
                    latency (p95) · cost/request · safety
Scoring method:     rule / human / LLM-as-judge (validated vs humans? Y/N)
Pass bar to ship:   ______________________________
Offline gate:       run full set on every prompt/model/retrieval change
Online check:       metric + A/B design + guardrail alarms
Flywheel:           prod failures → new cases → re-run → ship

B.3 Model-selection scorecard

Score candidate models 1–5 per use case, weight by what the use case needs, pick the highest weighted total — not the "smartest" model:

Glossary

Fast definitions for the terms in this book. If you can define each in a sentence, your fluency is where it needs to be.

Model & generation

• Token — the chunk of text a model reads and generates; the unit of cost and context. (~¾ word)
• Context window — max tokens a model can consider at once (prompt + output).
• Embedding — a vector representing text meaning; near vectors ≈ similar meaning.
• Temperature — randomness in generation; low = consistent, high = creative.
• Top-p — nucleus sampling; consider only the most probable tokens summing to p.
• Hallucination — a fluent, confident, false output.
• Reasoning model — a model trained to think in steps before answering; better at hard tasks, slower/costlier.
• Function calling / tool use — the model emitting a structured request to run a defined function.

Retrieval

• RAG — Retrieval-Augmented Generation; retrieve relevant text, then answer from it.
• Chunking — splitting documents into passages for indexing.
• Vector database — stores embeddings and finds nearest ones fast.
• Hybrid search — semantic + keyword search combined.
• Re-ranking — a second, more accurate pass to reorder retrieved candidates.
• Fine-tuning — further-training a model to change its behavior/style.

Evaluation

• Golden dataset — curated cases with ideal answers; your ruler for quality.
• Ground truth — the verified-correct answer for a case.
• LLM-as-judge — using a model to grade another model's output against a rubric.
• Precision — of what was returned, fraction correct. Recall — of what should have been caught, fraction caught.
• Faithfulness / groundedness — is the answer supported by the provided context? (RAG's anti-hallucination metric.)
• Regression testing — re-running the full eval set to catch newly broken cases.

Systems & strategy

• Agent — an LLM that loops: plans, acts via tools, observes, repeats toward a goal.
• Multi-agent — several specialized agents collaborating.
• MCP — Model Context Protocol; open standard connecting models to tools/data.
• Guardrails — rules/filters around a model that block unsafe inputs/outputs.
• Prompt injection — malicious instructions hidden in inputs to hijack model behavior.
• Open vs. closed model — self-hostable weights vs. vendor API access.

Further reading, curated

This book is your spine. When you want to go deeper on a topic, go here — deliberately, one source at a time, not by drowning in tabs.

Books, in reading order

• Escaping the Build Trap — Melissa Perri. Re-anchors you on outcomes over output; the mindset all the AI sits inside.
• Inspired — Marty Cagan. Revisit with an AI lens: discovery and delivery still rule.
• AI Engineering — Chip Huyen. The most PM-useful technical book on building with foundation models; evaluation, RAG, cost. Your core text.
• Designing Machine Learning Systems — Chip Huyen. For the MLOps and data-strategy fluency to collaborate credibly with engineers.
• Building LLM Applications / The AI Engineering Handbook — practical, pattern-level references to keep on the desk.

Follow weekly (two items, no more)

Pick two per week from the roadmap's list — OpenAI, Anthropic, Google DeepMind, Microsoft AI, LangChain, LlamaIndex, Pinecone, Hugging Face, Chip Huyen, Simon Willison — read them properly, and stop. Depth beats a firehose.

End of the Field Manual · built from your 16-week roadmap · pair it with the daily companion.