building informative evals and iterating often
Key facts
- Purpose
- Use LLMs to draft, critique, and red-team PRDs without outsourcing product judgment.
- Red-team trigger
- NNG cites a Columbia Journalism Review test where ChatGPT misattributed 76% of quote-identification requests. For PRDs, that is the warning label: source claims must be verified.
- Best next step
- Copy the drafting and red-team prompts.
Role
Use AI as a drafter, critic, and scenario expander
The best AI-assisted PRD workflow starts after you have a rough product point of view. Give the model your notes, research excerpts, constraints, target user, and current hypothesis. Ask it to structure and challenge the material, not to decide what customers need from nothing.
AI is useful when the task has clear inputs and a checkable output: summarize research, turn notes into a first draft, identify missing sections, generate edge cases, compare one-page versus full PRD formats, and rewrite vague requirements into acceptance-test language.
It is weak where teams are tempted to use it most recklessly: inventing customer evidence, estimating market size, choosing strategy, or claiming what competitors do without sources. Those tasks require human validation and primary evidence.
- Draft
- Convert notes into a structured PRD spine.
- Critique
- Find contradictions, missing non-goals, weak metrics, and hidden assumptions.
- Expand
- Generate edge cases, failure states, stakeholder questions, and launch risks.
- Compress
- Turn a bloated PRD into a one-page decision brief.
Prompt
A prompt that works because it constrains the model
A good PRD prompt gives the model a role, source material, output structure, standards for uncertainty, and explicit prohibitions. The more your prompt resembles a review checklist, the less the model has to improvise.
Do not ask for "a great PRD for my app idea" and expect quality. Ask for a specific transformation: "Using only the notes below, draft the problem, users, goals, non-goals, metrics, risks, and open questions. Mark missing evidence as UNKNOWN. Do not invent numbers."
Then run a second prompt that attacks the first draft. PRDs improve faster when the model is asked to act as a skeptical engineer, support lead, data analyst, compliance reviewer, and customer advocate in separate passes.
Reusable block
AI PRD drafting prompt
Paste this after your notes. Keep the UNKNOWN rule.
- Role
- You are a senior product manager drafting a PRD from source notes.
- Evidence rule
- Use only the provided notes. Mark missing facts as UNKNOWN. Do not invent customers, metrics, timelines, or competitors.
- Output
- Return problem, users, goals, non-goals, success metrics, requirements, risks, open questions, and a one-paragraph launch summary.
- Quality bar
- Every requirement needs an acceptance test. Every metric needs a baseline, target, or explicit UNKNOWN.
Review
Red-team the PRD before anyone treats it as true
The red-team pass is where AI becomes most valuable. Ask the model to find every sentence that depends on an unstated assumption. Ask it to list what an engineer, designer, data analyst, support lead, and legal reviewer would challenge.
Then ask it to produce a "kill list": what evidence would make the project not worth doing, what metric would signal harm, what dependency could block launch, and what customer segment might be hurt by the proposed tradeoff.
Finally, verify all factual claims manually. If the PRD contains a stat, competitor claim, legal interpretation, or customer quote, it needs a linked source or an owner responsible for validation.
- Highlight claims without sources.
- Find requirements without acceptance tests.
- Find metrics without baselines or instrumentation.
- Find non-goals that conflict with goals.
- Find places where the PRD assumes a design or architecture decision.
- Find launch risks that are not assigned to an owner.
Judgment
The PM still owns the hard calls
AI can make a PRD look complete before the thinking is complete. That is the main risk. A polished document with invented certainty is worse than a rough document that admits what the team does not know.
The PM owns prioritization, tradeoffs, evidence quality, stakeholder alignment, and the decision to stop. Use AI to increase the number of angles you inspect; do not use it to avoid the discomfort of choosing.
A useful operating norm is simple: AI may draft language and questions, but only named humans can approve user evidence, metrics, constraints, launch commitments, and final scope.