THE SHORT ANSWER
Evaluate evidence by asking whether it is relevant to the claim, how it was produced, whether the quantity and quality are sufficient, how current and consistent it is, and what uncertainty remains. An anecdote can reveal a possibility; it usually cannot establish how common that possibility is.
Test the evidence, not just its presence
| Dimension | Question |
|---|---|
| Relevance | Does it address this claim, population, time and context? |
| Method | How were observations selected, measured and analyzed? |
| Quality | Are errors, missing data and limitations handled openly? |
| Quantity | Is this one observation or a body of evidence? |
| Consistency | Do independent lines of evidence point similarly? |
| Recency | Could the underlying conditions have changed? |
| Uncertainty | What range of conclusions remains plausible? |
Evidence & context: National Academies of Sciences, Engineering, and Medicine
An anecdote can be useful without being representative
One customer's experience may expose a failure worth investigating. It does not show how often the failure occurs or whether the same cause explains other cases. Systematic evidence uses a stated method to collect and compare observations, making broader conclusions easier to assess.
Do not discard qualitative evidence merely because it is not a large dataset. Interviews and observations can reveal mechanisms, meanings and missing questions that a metric does not capture.
One study rarely settles a broad question
A result should be interpreted with its design, uncertainty and relationship to other evidence. The National Academies notes that replication and synthesis help assess a body of research; a single successful or failed replication does not automatically settle the underlying claim.
Evidence & context: National Academies of Sciences, Engineering, and Medicine
Keep a compact evidence record
- Exact claim
- Best supporting evidence
- Method and population
- Relevant limitations
- Independent corroboration
- Current confidence
- What evidence would change the conclusion
Sources & further reading
- Reproducibility and Replicability in Science
National Academies of Sciences, Engineering, and Medicine. A consensus report on evidence, uncertainty, transparency, reproducibility and replication. Scientific standards need proportionate adaptation outside research settings.
- Civic Online Reasoning
Digital Inquiry Group. Source-evaluation teaching resources, including lateral reading: investigating a source beyond its own website.
Examples and exercises are illustrative unless attributed to a source. No independent expert review is claimed.
A correction, a counterexample or an experience worth sharing?
Join the conversation ↗