Source Evaluation & Bibliography
Evaluate, document, and maintain curated collections of references and sources.
Source Evaluation Framework
The CRAAP Test
| Criterion | Questions | Weight |
|---|---|---|
| Currency | When was it published/updated? Is the information still valid? | Medium |
| Relevance | Does it address the specific question? Who is the intended audience? | High |
| Authority | Who is the author? What credentials/affiliations? | High |
| Accuracy | Is it evidence-based? Can claims be verified? Peer-reviewed? | Critical |
| Purpose | Is it informative, persuasive, or commercial? Any bias? | Medium |
Source Tier Classification
| Tier | Source Type | Reliability | Example |
|---|---|---|---|
| S | Primary research, official specs | Highest | Peer-reviewed papers, RFC documents, official API docs |
| A | Authoritative secondary | High | Textbooks, reputable technical blogs, conference proceedings |
| B | Community knowledge | Moderate | Stack Overflow answers (high-vote), well-maintained wikis |
| C | Informal | Variable | Blog posts, tutorials, forum discussions |
| D | Unverified | Low | Social media, anonymous posts, undated content |
Evaluation Record
source:
title: "Designing Data-Intensive Applications"
author: "Martin Kleppmann"
type: book
year: 2017
url: "https://dataintensive.net/"
tier: A
evaluation:
currency: Good (concepts still current despite 2017 publication)
relevance: High (directly addresses distributed systems patterns)
authority: Strong (researcher at Cambridge, industry experience)
accuracy: Peer-reviewed content, extensively cited
purpose: Educational, no commercial bias
notes: "Definitive reference for distributed data systems. Chapter 5-9 most relevant."
tags: [distributed-systems, databases, architecture]
last_verified: 2026-03-20
Annotated Bibliography Format
Entry Structure
## {Author Last}, {First}. "{Title}." *{Publication}*, {Year}. {URL}
**Tier:** {S/A/B/C/D} | **Relevance:** {High/Medium/Low} | **Last verified:** {Date}
**Summary:** {2-3 sentences describing the content and main argument}
**Key contributions:**
- {Specific idea, framework, or finding #1}
- {Specific idea, framework, or finding #2}
**Limitations:** {Any caveats, biases, or gaps}
**Connection:** {How this source relates to other sources or the project}
Example Entry
## Kleppmann, Martin. "Designing Data-Intensive Applications." O'Reilly, 2017.
**Tier:** A | **Relevance:** High | **Last verified:** 2026-03-20
**Summary:** Comprehensive guide to distributed data systems covering replication,
partitioning, transactions, and stream processing. Bridges theoretical CS concepts
with practical engineering tradeoffs.
**Key contributions:**
- Clear taxonomy of consistency models (linearizability, causal, eventual)
- Practical comparison of batch vs. stream processing architectures
- The "unbundling the database" framing for microservice data patterns
**Limitations:** Pre-dates serverless and edge computing patterns. Some implementation
details are framework-specific and may be dated.
**Connection:** Foundation for resilience-patterns and data-pipeline-architect skills.
Complements the CAP theorem discussion in redis-patterns.
Bibliography Management
Collection Structure
references/
├── bibliography.yaml # Machine-readable catalog
├── by-topic/
│ ├── distributed-systems.md # Topic-organized annotations
│ ├── security.md
│ └── ai-agents.md
├── by-tier/
│ ├── tier-s.md # Primary sources only
│ └── tier-a.md # Authoritative secondary
└── reading-log.md # Chronological reading notes
Machine-Readable Catalog
bibliography:
- id: kleppmann2017
title: "Designing Data-Intensive Applications"
author: "Martin Kleppmann"
year: 2017
type: book
tier: A
topics: [distributed-systems, databases]
cited_by: [resilience-patterns, data-pipeline-architect]
- id: fowler2002
title: "Patterns of Enterprise Application Architecture"
author: "Martin Fowler"
year: 2002
type: book
tier: A
topics: [architecture, patterns]
cited_by: [backend-implementation-patterns]
Citation in Documents
The circuit breaker pattern [kleppmann2017, ch.8] provides fault isolation
between services. This aligns with the bulkhead pattern described in
[nygard2018, ch.5], which isolates failure domains at the resource level.
Curation Workflow
Adding New Sources
- Discover — Find source through research, recommendation, or citation chain
- Evaluate — Apply CRAAP test and assign tier
- Annotate — Write summary, key contributions, limitations
- Connect — Link to existing sources and project skills
- Catalog — Add to bibliography.yaml and topic files
Periodic Review
## Quarterly Review Checklist
- [ ] Verify URLs still resolve (automated: link-checker script)
- [ ] Check for updated editions or superseding publications
- [ ] Re-evaluate tier for sources whose domains have evolved
- [ ] Remove or demote sources that are no longer current
- [ ] Add newly discovered sources from recent research
Anti-Patterns
- Uncritical acceptance — Every source needs evaluation, even authoritative ones
- Recency bias — Older sources can be foundational; evaluate on merit, not date alone
- Single-source reliance — Triangulate claims across multiple independent sources
- No annotation — A bibliography without annotations is just a link list
- Stale references — Sources need periodic re-verification
- Collecting without connecting — Always link sources to the questions they help answer