Skip to content

Karpathy's LLM Wiki

A pattern popularized by Andrej Karpathy (co-founder of OpenAI, ex-Tesla AI lead): instead of traditional rag-vs-wiki, have an LLM compile knowledge once into a persistent, interlinked set of markdown files — a wiki — and keep it current. This very knowledge base is built on that pattern.

The core idea

Traditional RAG rediscovers knowledge from scratch on every query: retrieve chunks, stuff into context, hope relevance. A compiled wiki does the work up front — cross-references are already there, contradictions already flagged, synthesis already reflects everything ingested. The agent reads a small index.md, follows wiki-links, and reasons over curated pages.

Division of labor

  • Human curates sources and directs analysis.
  • Agent summarizes, cross-references, files, and maintains consistency.

Key facts

  • Structure: raw/ (immutable sources) → curated entities/, concepts/, comparisons/ pages → index.md + log.md navigation.
  • No vector database required — retrieval is agentic (search + read + follow links), see rag-vs-wiki.
  • Just markdown: opens in Obsidian, VS Code, or any editor; portable and owned.
  • Compounds over time — each ingest makes the whole more valuable, unlike per-query RAG.