Turn Years of Messy Notes Into Wisdom You Can Search

Install the skill below for me. Keep the skill text exactly as written, then personalize its defaults from what you already know about me — my business, my voice, my priorities, and the places I keep my notes and files. Confirm when it's installed and show me exactly how to run it.

---
name: knowledge-library-extraction
description: Extract, organize, and distill a personal knowledge library from raw notes into structured intelligence with through-lines. Use this skill whenever you say "extract my library," "process my notes," "run the extraction," "build a codex," "distill my notes," "organize my second brain," "run CODE on my notes," or any variation of wanting to turn a large collection of personal notes into organized, categorized, searchable wisdom. Also trigger when you have a folder of notes from conferences, coaching programs, books, courses, or mentors and want to find the patterns, through-lines, and best insights across all of them. This is the bridge between a messy second brain and a structured intelligence asset.
---

# Knowledge Library Extraction

Turn a raw collection of personal notes into a structured intelligence library with through-lines, attributed lessons, and searchable wisdom.

## What This Produces

From raw notes (Evernote, Notion, markdown, text files), this process produces:

1. **Clean text files**, one per note, stripped of formatting, with title and date preserved
2. **Structured JSON cards**, one per note, with lessons extracted, speakers attributed, personal reflections separated, and a top quote identified
3. **Through-lines**, the 7-12 recurring philosophies that run across the entire collection, with anchor quotes and top lessons per theme
4. **A through-lines report**, a human-readable document showing the patterns, with an observations section that surfaces what a human reading linearly would never see

## When to Use This

- Processing notes from coaching programs, conferences, masterminds, or courses
- Organizing a personal knowledge management library (second brain)
- Preparing content for a Codex-style artifact or interactive experience
- Building a Claude Project knowledge base from personal notes
- Finding patterns across years of learning from mentors, speakers, or thought leaders
- Creating a searchable, categorized wisdom library from unstructured notes

## Prerequisites

- A collection of notes in one of these formats: Evernote export (HTML), Notion export (markdown), plain text files, or markdown files
- Notes should represent a coherent body of knowledge (one mentor's world, one topic area, one program series, etc.)
- Minimum ~20 notes for meaningful through-line discovery. Sweet spot is 50-200.

## The Process: CODE

**C**apture → **O**rganize → **D**istill → **E**xecute

---

### Phase 1: CAPTURE (Export and Convert)

**Goal:** Get all notes into clean, uniform text files.

**Input:** Raw notes in their native format
**Output:** A folder of clean .txt files, one per note

#### Step 1.1: Export from source

**From Evernote:**
- Select all notes → File → Export
- Choose "Multiple Web Pages (.html)"
- Export Note Attributes: keep only Note Title, Created Date, Tags
- Deselect: Author, Note Reminders, Source URL, Tasks, Updated Date, Thumbnail
- Why: Each attribute adds HTML clutter. Title, date, and tags are the only metadata we need.

**From Notion:**
- Select pages → Export → Markdown & CSV
- Choose "Everything" for content
- Include subpages if applicable

**From other sources:**
- Get files into .txt or .md format, one file per note
- Ensure each file has a clear title (filename or first line)

#### Step 1.2: Batch convert to clean text

Run this in Claude Code. The prompt below is calibrated for Evernote HTML exports. Adjust the input parsing for other formats.

**Claude Code Prompt:**
```
Convert all HTML files in [INPUT FOLDER] from Evernote HTML export format to clean, structured text files. Each HTML file is one note exported from Evernote with "Multiple Web Pages (.html)" format.

Create output folder: [OUTPUT FOLDER]
One .txt file per input .html file, with the same filename (but .txt extension).

Evernote HTML structure:
- Note title: <meta itemprop="title" content="THE TITLE">
- Created date: <meta itemprop="created" content="20200923T235313Z">
- Tags (if any): <meta itemprop="tag" content="TAG NAME"> (multiple possible)
- Content: inside <en-note> tags
- Ignore: <style> blocks, SVG <symbol> blocks, all CSS

Output format per .txt file:
TITLE: [note title from meta tag]
DATE: [created date, reformatted to readable format like "September 23, 2020"]
TAGS: [comma-separated tags if any, or "None"]

---

[Clean text content with these rules:]
- Strip ALL HTML tags
- Preserve line breaks where block elements (p, div, li, h1-h4, br) indicate them
- Collapse 3+ consecutive newlines to 2 (one blank line)
- Preserve bullet characters (•, -, *)
- Preserve bold: text in <b> or <strong> wrapped in ** on each side
- Strip all CSS, SVG, script content
- Trim whitespace per line, remove whitespace-only lines

Use Python with html.parser (no pip installs). Print progress per file. Print summary at end: total converted, total characters, any failures.

Do not ask me any questions, just build it and verify it works.
```

**Verification:** Check 3 random output files. Title should be clean, date readable, content should be the actual note text with no HTML artifacts.

---

### Phase 2: ORGANIZE (Extract Structured Cards)

**Goal:** Turn each clean text file into a structured JSON card with lessons extracted, speakers attributed, and reflections separated.

**Input:** Folder of clean .txt files
**Output:** A single all-cards.json file with one card per note

#### Step 2.1: Define your extraction schema

Each note becomes a card with this structure:

```json
{
  "id": "note-001",
  "title": "Original note title",
  "date": "September 23, 2020",
  "source": "Program or context name",
  "source_confidence": "high | medium | low",
  "speakers": ["List of people whose wisdom appears"],
  "lessons": [
    {
      "text": "Exact lesson text. PRESERVE ORIGINAL LANGUAGE",
      "speaker": "Who said it",
      "type": "quote | principle | tactic | mindset | story | reflection"
    }
  ],
  "personal_reflections": [
    "Lines that are the note-taker's personal processing"
  ],
  "top_quote": "Single most powerful line in this note"
}
```

#### Step 2.2: Define source detection rules

Customize these for your specific library. The idea is to map a signal in each note to the program or context it came from. Examples:

- Title contains a program keyword → "[Program name]"
- Content is about a topic area you studied → "[Topic bucket name]"
- Title contains a shorthand or acronym you use → "[Program name]"
- Default → "[General bucket name]"

#### Step 2.3: Define speaker detection rules

- Lines preceded by a name + dash/colon = that speaker's content
- Default speaker = the primary teacher unless attributed otherwise
- First-person lines about the note-taker's life = "[Name] (reflection)"
- Lines starting with ** = personal commentary
- Unsure = "Unknown"

#### Step 2.4: Run extraction in batches

Process 20 notes per Claude Code prompt. Do NOT use parallel sub-agents (they crash). Sequential batches are reliable.

**First batch prompt:**
```
Process the first 20 .txt files from [CLEAN NOTES FOLDER] (alphabetically). For each note, read the file and produce a structured JSON card with: id, title, date, source, source_confidence, speakers, lessons (with text, speaker, type), personal_reflections, and top_quote.

[INSERT YOUR SOURCE DETECTION RULES]
[INSERT YOUR SPEAKER DETECTION RULES]

Lesson types: quote (standalone memorable line), principle (operating philosophy), tactic (specific actionable strategy), mindset (belief or identity statement), story (narrative, capture key point), reflection (personal processing)

CRITICAL: Preserve original language. Do not summarize or rephrase. The exact words are the product.

Save to [OUTPUT]/batch-01.json. Print files processed, lesson count per file, and top_quote from each note.

Do not ask me any questions, just build it and verify it works.
```

**Subsequent batch prompt (repeat until all notes processed):**
```
Continue processing notes from [CLEAN NOTES FOLDER]. Use the exact same extraction rules and JSON card structure from the previous batch. Process the next 20 files alphabetically (skip any already processed). Save to [OUTPUT]/batch-XX.json (increment number). Print files processed, lesson count per file, and top_quote from each.

If this is the LAST batch (fewer than 20 remaining), after saving, merge ALL batch files into a single [OUTPUT]/all-cards.json. Generate [OUTPUT]/extraction-summary.md with: total notes, breakdown by source, breakdown by speaker, total lessons, total reflections, top 20 quotes. Print summary.

Do not ask me any questions, just build it and verify it works.
```

**Verification:** Spot-check 3-4 cards against notes you know well. Are lessons preserved verbatim? Are speakers correctly attributed? Are your personal reflections separated?

---

### Phase 3: DISTILL (Through-Line Discovery)

**Goal:** Find the 7-12 recurring philosophies that run across the entire library.

**Input:** all-cards.json
**Output:** through-lines.json + through-lines-report.md

#### Step 3.1: Run the clustering

**Claude Code Prompt:**
```
Read [PATH]/all-cards.json. Analyze ALL cards and discover the natural through-lines, the 7-12 recurring themes that run across the entire corpus regardless of source, speaker, or year.

A through-line is NOT a generic category like "Leadership" or "Mindset." It is a specific recurring philosophy named in the language of the world these notes come from.

For each through-line, identify:
- Name (in the source's language, using the words the teacher would use)
- One-sentence description
- How many notes touch this theme
- Which sources/programs it appears across
- Which speakers contribute to it
- 3-5 strongest lessons (EXACT original language)
- Single most powerful anchor quote

Save:
1. [PATH]/through-lines.json (structured data with all lessons mapped)
2. [PATH]/through-lines-report.md (human-readable report with: summary, each through-line with anchor quote and top lessons, uncategorized notes list, and 2-3 paragraph Observations section surfacing what a human reading linearly would never see)

Target: 7-12 through-lines. Fewer than 7 = too broad. More than 12 = consolidate.
PRESERVE ORIGINAL LANGUAGE in all quotes.

Print all through-line names, note counts, and anchor quotes.

Do not ask me any questions, just build it and verify it works.
```

#### Step 3.2: Review and iterate

This is the human step. Read the through-lines report and ask:

1. **Does each through-line feel like the world it came from?** If a name sounds like a textbook category, rename it in the source's language.
2. **Are any through-lines actually the same idea expressed differently?** Merge them.
3. **Is anything major missing?** Run the runner-up analysis:

```
Look at all-cards.json. If you were forced to pick 5 MORE through-lines beyond the current ones, what would they be? For each: proposed name, note count, anchor quote, why it did not make the cut. Also check: is there a distinct theme around [TOPIC YOU SUSPECT IS MISSING]? Report findings, do not save files.
```

4. **Swap, add, or merge** based on your review, then re-run the clustering with updates.

#### Step 3.3: Finalize

Once through-lines are locked, run one final update prompt to regenerate both files with the final structure. This is your permanent content architecture.

---

### Phase 4: EXECUTE (Build From the Asset)

The extraction is complete. You now have a structured intelligence library that can express itself in multiple forms:

**As a Claude Project:** Load all-cards.json and through-lines.json as project knowledge. Query the library conversationally: "What have I learned about X?" "Find every lesson about Y." "Write something using the principles from Z."

**As an interactive artifact:** Build a web experience (Claude Code) that lets someone explore the through-lines visually with layered depth.

**As a randomizer:** Surface one lesson at a time from the full library. Daily wisdom from the collection.

**As a content engine:** Each through-line is a content pillar. Each anchor quote is a post. Each top lesson is a thread. The library becomes source material for months of content.

**As a searchable database:** Import the structured cards into Notion or any database tool. Filter by speaker, source, type, or through-line.

The data is the asset. The expression is the choice.

---

## Key Principles

- **PRESERVE ORIGINAL LANGUAGE.** This is the #1 rule. If someone said "Don't quit 5 minutes before the miracle," that is the lesson, word for word. Never rewrite it into "the speaker advises persistence near the finish line."
- **Sequential batches beat parallel agents.** Parallel sub-agents in Claude Code crash under heavy load. 20 notes per batch, one batch at a time, is reliable.
- **The through-lines are the irreversible decision.** Visual design can change in a day. The content architecture cannot. Get the through-lines right before building anything on top of them.
- **A master knows what to leave out.** Every note does not need to be categorized. Stubs, receipts, and action-item lists can stay uncategorized without harming the library.
- **Max subscription note:** If using Claude Code through a Max subscription (rather than a separate API key), do NOT write prompts that make API calls from scripts. All processing should happen within Claude Code's native conversation.

## Typical Scale

| Library Size | Notes | Conversion Time | Extraction Time | Clustering Time |
|-------------|-------|----------------|-----------------|-----------------|
| Small | 20-50 | 5 min | 2-3 batches, ~30 min | 1 prompt, ~10 min |
| Medium | 50-150 | 10 min | 5-8 batches, ~1 hour | 1-2 prompts, ~20 min |
| Large | 150-300 | 15 min | 10-15 batches, ~2 hours | 2-3 prompts, ~30 min |
| Massive | 300+ | 20 min | 15+ batches, ~3 hours | Multiple prompts, ~1 hour |

## Troubleshooting

- **Claude Code "Not responding"**: Stop it, start a fresh session, re-paste the same batch prompt. Each batch reads from disk, no conversation history needed.
- **Through-lines feel generic**: The names need to be in the language of the world the notes came from rather than academic categories. Ask: "Would the teacher recognize this as their language?"
- **Too many uncategorized notes**: Run a targeted analysis on the uncategorized set looking for hidden clusters of 5+ notes sharing a theme.
- **Lessons come back paraphrased when they should be verbatim**: This is the most common extraction failure. Spot-check early and add emphasis to the "PRESERVE ORIGINAL LANGUAGE" instruction if needed.