Memory Wiki

[IMPORTANT: The user has invoked the "memory-wiki-and-bodega" skill, indicating they want you to follow its instructions. The full skill content is loaded below.] --- name: memory-wiki-and-bodega category: productivity description: >- Build a browsable static-site memory wiki from session transcripts, paired with a thinking-pattern archive (bodega) maintained by a zero-cost-when-idle cron librarian agent. Turns every conversation into a compounding knowledge asset. --- # Memory Wiki + Bodega ## Trigger You have multiple sessions of conversation history with your user and want a human-browsable archive (wiki) plus a thinking-pattern knowledge base (bodega). The wiki shows what was done each day, session transcripts, and topic indexes. The bodega distills conversations into reusable patterns (stories, metaphors, philosophy, brand voice, methodology, etc.) tagged by tone. ## Two Distinct Systems | Layer | Purpose | Audience | Tech | |-------|---------|----------|------| | **Wiki** | Session history, daily logs, topic index, full transcripts | Human (browsable) | Static HTML + JS search | | **Bodega** | Thinking-pattern archive — stories, metaphors, methodology | Human + other agents | Markdown files, filesystem-readable | Both live under a single static site, served by Nginx. The wiki is the browsable front-end; the bodega files are the machine-readable back-end that any agent can `read_file` to find content. ## Architecture ``` Static Site Generator (Python) │ ├── Reads: session DB (SQLite + FTS5) ├── Reads: bodega markdown files └── Writes: HTML files to Nginx webroot Nginx serves /var/www/wiki/ ├── index.html (landing page + search) ├── daily/YYYY-MM-DD/ (daily activity pages) ├── sessions/ (full transcripts) ├── topics/ (keyword/tag index) ├── bodega/ (8 category pages) ├── llms.txt (LLM sitemap) ├── search.json (client-side fuzzy search index) └── assets/ (CSS, JS) ``` ## Prudence: The Cron Librarian Agent A dedicated background agent that runs on a schedule. Architecture: ### Gate Script (Python, runs first — zero LLM cost) Checks the session DB for new messages since `last_run`. Outputs one of: - `SLEEP` — no new messages. Agent says "Quiet." and bails (~5 tokens). - `WORK:{msg_ids}` — comma-separated IDs of new messages to process. - `DREAM` — weekly deep housekeeping trigger. The cron job uses the `script` parameter with `no_agent=False` (default): ```yaml script: ~/.hermes/scripts/prudence-gate.py # Gate script at ~/.hermes/scripts/prudence-gate.py ``` When output is `SLEEP`, the agent gets blank context → tiny token cost. When `WORK` or `DREAM`, the agent loads thinking-enabled model for extraction. ### WORK Mode 1. Read new messages by ID from session DB (SQLite at `~/.hermes/state.db`) 2. Filter for substantive messages: user role with non-empty content, or assistant role with non-trivial content. Skip empty assistant messages (tool call summaries) and pure tool output. 3. Classify each message into Bodega categories using the taxonomy below. 4. Append entries to the appropriate Bodega markdown files (format: `## YYYY-MM-DD — Short Title`, then `[uncle|dr]` tone tag, then `Source: message ID {id}`, then content). 5. Run the static site generator: `python3 /root/.hermes/scripts/generate-memory-wiki.py` 6. Update `last_run` timestamp to prevent re-processing. Batch size cap (e.g. 200 msgs/cycle) to keep context small. #### Bodega Classification Taxonomy | If the message contains... | Write to | Tone | Example | |---|---|---|---| | First-person stories, anecdotes, experiences | `personal_story.md` | uncle | "I tried this with OpenClaw and it failed" | | Systems/cost/process thinking | `operator_philosophy.md` | dr | "1M tokens once vs 10M per agent" | | Academic references, research connections | `lit_review.md` | dr | Citing papers, PhD-level analysis | | Analogies, comparisons, metaphors | `copy_metaphor.md` | uncle | "SA is a compression layer" | | Direct content ideas, hooks, post concepts | `content_hook.md` | uncle | "I knighted my AI" (hook for a post) | | Member psychology, audience insights, role definitions | `avatar_insight.md` | uncle | "TB is the idiot savant coder" | | Process/methodology descriptions | `methodology.md` | dr | "Create a control baseline before modifying" | | Natural phrasing quirks, unique voice, framing patterns | `brand_voice.md` | uncle | "hold my beer while i run amuck" | **Tone assignment rule**: `uncle` for markets/builder voice content (approachable, vivid, story-driven). `dr` for academic/systems content (Dr. PhD voice, analytical, cost-model-aware). Every entry gets exactly one tone tag. When a message could fit both, pick the primary audience. **Multi-source entries**: When a single insight spans multiple message IDs (e.g. user question + assistant elaboration), combine them under one entry with `Source: message ID X, Y`. **Prudence persona rules**: - Do NOT narrate your process. File things correctly and go back to sleep. - Use `execute_code` with Python's sqlite3 for DB queries. Reserve `terminal()` for one-shot diagnostics (checking file existence, which sqlite3, head -c to verify SQLite magic bytes). - Read existing bodega files first to understand format before writing. - After all writes, run the generator script. Do not skip this step. After generating, mark processed messages as observed: ```sql import sqlite3 conn = sqlite3.connect('/root/.hermes/state.db') conn.execute("UPDATE messages SET observed = 1 WHERE id IN ({})".format(','.join('?' * len(processed_ids))), processed_ids) conn.commit() conn.close() ``` **Read-before-write safeguard**: When appending entries to bodega files, read the file immediately before writing to it, not at the start of the session. Sibling subagents or parallel processes may modify the same files between your first read and your write. Always trust the most recent read. ### DREAM Mode (Weekly — e.g. Tuesday 0300) Four-phase OpenClaw-inspired housekeeping: 1. **Light** — Scan full Bodega + Wiki. Identify near-duplicates, orphan entries, missing tone tags, category bloat/thinness. No writes. **Near-duplicate detection heuristics:** - Same message ID appearing in ≥2 files — likely the same quote classified under different analytical framings. Flag these for cross-referencing in Deep. - Adjacent entries in the same file that cite overlapping message ID ranges (e.g. msg 559 and msg 568/576 both about failure resilience) — may describe the same trait from slightly different angles. Consider binding with bidirectional cross-refs. - Entries whose titles describe conceptually adjacent or sibling concepts — e.g. "Friction-Tolerant Operator" and "Pragmatic Constraint-Worker" (both about adaptability). Flag for cross-ref. - Same source quote cited in a `brand_voice.md` entry (stylistic framing) AND an `avatar_insight.md` entry (psychological framing) — these are *not* duplicates to merge, but the most important pair to cross-reference. They show the same utterance being read at two different depths. 2. **Deep** — Merge duplicates, add cross-references, assign missing tone tags, propose new categories. Writes to Bodega files. **Cross-reference format (durable convention):** ``` *See also: category_file.md → Entry Title — brief note on relationship.* ``` Place on its own line(s) at the end of an entry, before the next `##` header. Three common relationship types: - *same quote, different framing* — brand_voice ↔ avatar_insight (stylistic vs psychological) - *narrative event ↔ abstract pattern* — personal_story ↔ avatar_insight (event vs profile) - *simplified version ↔ elaborated version* — e.g. "Build-Test-Review" ↔ "Three-Stage Verification" When an entry has cross-refs in both directions, add the `See also` at both ends. **Merge policy:** - Do NOT merge entries from different categories that cite the same source message — they are different analytical framings of the same quote, and the cross-file taxonomy is the point of the Bodega. Instead, cross-reference them. - Do merge within a single file when two entries describe the same subject with overlapping source IDs. Keep the more detailed version and add a cross-ref note pointing to the merged-away entry's summary. 3. **REM** — Reflect on emerging patterns. What topics recur across categories? What stories are told from multiple angles? No writes. **REM analysis dimensions:** - *Triangulation count* — how many categories does a topic appear in? 3+ means it's a first-class pattern. - *Story echo* — is the same narrative told from different category angles? (e.g. XO commissioning appears in personal_story.md, copy_metaphor.md, and avatar_insight.md) - *Gap detection* — which categories are conspicuously thin given the user's identity? (e.g. PhD TESOL with 1 lit_review entry) - *Adversarial pattern* — user prefers clean rebuilds over surgical fixes: does the architecture reflect this? (burn-it-down, strip-WordPress, control-baseline pattern) 4. **Dream Diary** — Append narrative summary to `DREAMS.md` with findings. Human-readable. Format: ``` --- ## Housekeeping Cycle YYYY-MM-DD **Light scan** — Key findings in one sentence. **Deep reorganization**: - Bulleted list of changes made (files modified, cross-refs added, entries merged) **REM patterns detected**: 1. Pattern title — one-sentence description. 2. Pattern title — one-sentence description. Closing line (in Prudence voice): one sentence about the state of the Bodega. ``` ### Cron Job Setup ```bash # Gate script at ~/.hermes/scripts/prudence-gate.py (symlink there too) # Cron: hourly at :05 # Model override: thinking-enabled model cronjob( action="create", name="prudence", schedule="5 * * * *", # hourly at :05 script="prudence-gate.py", model={"provider": "deepseek", "model": "deepseek-v4-flash"}, # thinking-enabled prompt="I am Prudence, the librarian. If the context says SLEEP, say 'Quiet.' in 3 words. If WORK with comma-separated message IDs, read those messages and extract Bodega entries. If DREAM, do the full housekeeping cycle.", workdir="/var/www/wiki", skills=["memory-wiki-and-bodega"], ) ``` ## The Bodega: Eight Thinking-Pattern Categories Each category is a markdown file in the bodega directory. Entries are appended chronologically with frontmatter for tone tagging. ### Standard Categories | File | Purpose | |------|---------| | `brand_voice.md` | Natural phrasing, tone, expressions | both | | `personal_story.md` | Specific experiences that illustrate principles | both | | `methodology.md` | Acquisition logic, operator logic, systems thinking | both | | `avatar_insight.md` | How the target member thinks and feels | both | | `copy_metaphor.md` | Analogies that land naturally | both | | `operator_philosophy.md` | Thinking about investment, systems, cost, process | both | | `lit_review.md` | Academic connections by topic cluster | dr | | `content_hook.md` | Observations that work as entry points | both | ### Tone Tags Every entry gets a tone tag on its own line, right after the header, before Source: ``` ## YYYY-MM-DD — Entry Title [uncle] Source: message ID 1234 Entry content... ``` - `uncle` — Fav uncle Profe voice (markets content, approachable, story-driven) - `dr` — Dr. PhD voice (lit reviews, academic, systems thinking) ## Gate Script Path The gate script lives at `/root/.hermes/scripts/prudence-gate.py`. A symlink should also exist at `~/.hermes/scripts/prudence-gate.py` (relative to the profile's HOME dir) for cron job resolution. Paths within the gate script are hardcoded to `/root/` (e.g. `/root/.hermes/state.db`, `/root/.hermes/prudence/last_run`) because HOME may resolve to the profile directory (e.g. `/root/.hermes/profiles/donna-api/home/`) under Hermes, which breaks `os.path.expanduser("~")`. ## Static Site Generator (Python Script) The core script reads two sources and writes HTML: ### Input Sources 1. **Session DB**: `~/.hermes/state.db` (SQLite with FTS5) - `sessions` table: `id, title, created_at, updated_at` - `messages` table: `id, session_id, role, content, created_at` 2. **Bodega files**: `~/.hermes/bodega/*.md` (8 markdown files) ### Generator Logic 1. Query daily activity: `SELECT DATE(created_at) as day, COUNT(*) FROM sessions GROUP BY day` 2. For each day: Generate `daily/YYYY-MM-DD/index.html` with session list 3. For each session: Generate `sessions/SESSION_ID.html` with full transcript 4. Build topic index from session titles + message content 5. Build search.json: title + snippet for every session 6. Read all bodega MD files, render each as `bodega/CATEGORY.html` 7. Generate landing page with recent activity + search bar 8. Write llms.txt — LLM sitemap of the site ### Agent-side Query Pattern (for WORK mode) The agent uses `execute_code` with Python's `sqlite3` module, NOT the terminal shell: ```python import sqlite3 from datetime import datetime conn = sqlite3.connect('/root/.hermes/state.db') conn.row_factory = sqlite3.Row cur = conn.cursor() # Get message IDs from the WORK list work_ids = [1429, 1431, 1433, ...] # from context placeholders = ','.join('?' for _ in work_ids) # Get all messages (user + substantive assistant) cur.execute(f"SELECT id, role, session_id, timestamp, content FROM messages WHERE id IN ({placeholders}) AND role='user' ORDER BY id", work_ids) users = cur.fetchall() # Also get non-empty assistant messages cur.execute(f"SELECT id, role, session_id, timestamp, content FROM messages WHERE id IN ({placeholders}) AND role='assistant' AND content != '' AND content IS NOT NULL ORDER BY id", work_ids) assistants = cur.fetchall() # Convert timestamp to date date = datetime.fromtimestamp(rows[0]['timestamp']).strftime('%Y-%m-%d') ``` Then read existing bodega files to understand their current state before appending. ## Session DB Schema The Hermes session store at `~/.hermes/state.db`: ```sql -- schema_version, sessions, messages, state_meta, session_stats, etc. -- Sessions table (actual schema as of May 2026) CREATE TABLE sessions ( id TEXT PRIMARY KEY, -- e.g. "20260523_211220_3fb1a7" source TEXT, -- platform (telegram, cli, etc.) user_id TEXT, model TEXT, model_config TEXT, system_prompt TEXT, parent_session_id TEXT, started_at REAL, -- epoch timestamp ended_at REAL, end_reason TEXT, message_count INTEGER DEFAULT 0, tool_call_count INTEGER DEFAULT 0, input_tokens INTEGER DEFAULT 0, output_tokens INTEGER DEFAULT 0, cache_read_tokens INTEGER DEFAULT 0, cache_write_tokens INTEGER DEFAULT 0, reasoning_tokens INTEGER DEFAULT 0, billing_provider TEXT, billing_base_url TEXT, billing_mode TEXT, estimated_cost_usd REAL, actual_cost_usd REAL, cost_status TEXT, cost_source TEXT, pricing_version TEXT, title TEXT, api_call_count INTEGER DEFAULT 0, handoff_state TEXT, handoff_platform TEXT, handoff_error TEXT ); -- Messages table (actual schema as of May 2026) CREATE TABLE messages ( id INTEGER PRIMARY KEY AUTOINCREMENT, session_id TEXT, -- FK to sessions.id role TEXT, -- 'user', 'assistant', 'tool' content TEXT, tool_call_id TEXT, tool_calls TEXT, tool_name TEXT, timestamp REAL, -- epoch timestamp token_count INTEGER, finish_reason TEXT, reasoning TEXT, reasoning_content TEXT, reasoning_details TEXT, codex_reasoning_items TEXT, codex_message_items TEXT, platform_message_id TEXT, observed INTEGER DEFAULT 0 -- 0=unprocessed, 1=processed by bodega ); -- FTS5 virtual tables for search CREATE VIRTUAL TABLE messages_fts USING fts5( content, content=messages, content_rowid=id ); ``` **Key fields for Prudence**: - `messages.id` — use this as the source reference in Bodega entries - `messages.role` — filter for `'user'` (primary source) and substantive `'assistant'` messages - `messages.content` — the actual text to classify - `messages.timestamp` — use `datetime.fromtimestamp()` for human dates - `messages.observed` — NOT used by the gate script (the gate uses the `last_run` timestamp file instead, to avoid mutating message rows during a read-only scan). The `observed` field exists but is reserved for other consumers. Gate script checks `SELECT MAX(timestamp) FROM messages` against last run timestamp. Sessions use `started_at` / `ended_at` REAL timestamps, not `created_at`. ## Nginx Config (Static Site) ```nginx server { listen 443 ssl; server_name wiki.jaimisaunders.com; ssl_certificate /etc/letsencrypt/live/wiki.jaimisaunders.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/wiki.jaimisaunders.com/privkey.pem; include snippets/ssl-params.conf; root /var/www/wiki; index index.html; location / { try_files $uri $uri.html $uri/ =404; } location /assets/ { expires 7d; add_header Cache-Control "public, immutable"; } location ~ \.json$ { expires 1h; add_header Cache-Control "public"; } } server { listen 80; server_name wiki.jaimisaunders.com; return 301 https://$server_name$request_uri; } ``` ## llms.txt — LLM Sitemap Both the main domain and wiki need a llm.txt at root for AI crawlers: ``` # English with Profe Jaimi > CI (Comprehensible Input) learn-English-through-story for B1-C1 professionals ## Docs - Homepage: https://jaimisaunders.com/ - Wiki: https://wiki.jaimisaunders.com/ ## Affordances - Landing pages: public, indexable - Story catalog: membership-required for full content - Community: public feed with member discussions - Lit reviews: public, written by Dr. Jaimi Saunders PhD TESOL ``` ## Verification ```bash # Site responds curl -sL -o /dev/null -w "%{http_code}" http://localhost:8080/ # 200 (adjust port/host) # Search index loads curl -s http://localhost:8080/search.json | head -c 100 # Bodega page exists curl -s http://localhost:8080/bodega/ | grep -c "Bodega" # Generator works python3 /root/.hermes/scripts/generate-memory-wiki.py # adjust path ``` ## Pitfalls - **Gate script output must be exact** — the agent reads it as context. `SLEEP` / `WORK:id1,id2` / `DREAM` exactly, nothing else. - **Batch sizes** — in WORK mode, limit to ~200 messages per cycle so context stays under budget on a thinking model. - **Dream date logic** — the gate must check day-of-week + hour, not just look for a "DREAM" trigger from the cron. Use `datetime.now().weekday() == 1 and datetime.now().hour == 3`. - **No file-level locking needed** — the cron scheduler uses `.tick.lock`, so Prudence never duplicates a cycle. - **Bodega file growth** — append-only means files grow linearly. Dream mode should consolidate duplicate entries to keep files manageable. - **Tone tagging consistency** — hard rule: every entry gets exactly one tone tag. If an entry could be both, pick the primary audience (uncle for markets content, dr for academic content). - **Empty assistant messages** — most assistant messages in a WORK batch will have empty content (they're tool call summary placeholders printed when the tool output was empty). The agent MUST filter these out before classification. Query for `role='user'` content first, then check assistant messages for non-empty content. A single useful insight often spans a user question + the assistant's elaboration — combine them into one entry with multiple source IDs. - **Prudence's own messages appear in WORK batches** — the gate script returns ALL new messages since last_run, including the Prudence cron prompt itself and all assistant/tool messages from previous Prudence cycles. Three layers to filter: 1. **The cron prompt** (the current run's instruction) — `role='user'`, content starts with "[IMPORTANT: You are running as a scheduled cron job" — skip entirely. Check for this substring: `"You are running as a scheduled cron job"`. 2. **Previous Prudence output messages** — these are `role='assistant'` messages from prior cycles with content like "The gate returned WORK with N messages. Let me process them." or "Now let me classify the three substantive messages:" or "Now rebuild the site." or "Now let me also mark these messages as observed." These are NOT user conversation — they're the librarian's prior analysis. Skip any assistant message whose content contains meta-narrative about itself (e.g. "Let me process them", "Let me get the rest", "Now let me classify", "Now rebuild the site"). A reliable heuristic: assistant messages in a WORK batch that narrate the Bodega filing process itself should be skipped. They contain no user conversation to classify. 3. **Previous Prudence's cron-prompt copy** — the first user-role message in a WORK batch may be a copy of the Prudence cron prompt from a prior cycle. Check for "You are Prudence, the librarian agent" — skip if found (defensive check, this string is in the internal prompt not stored messages). The net: user-role messages that are (a) the current cron prompt, (b) a prior cron prompt, or (c) the gate script's stdout wrapper should be filtered. Assistant-role messages that narrate Bodega analysis should be filtered. The remaining messages are real user conversation worth classifying. - **WORK context format varies by cron delivery** — when running as a cron job via `script` parameter (pre-run script that wraps output in `## Script Output`), the WORK list arrives wrapped in a markdown codeblock: ``` ## Script Output ... WORK:id1,id2,id3 ``` Extract the IDs with the same comma-split logic after matching the `WORK:` token. The SLEEP and DREAM signals also arrive wrapped — match on the token (`WORK:`, `SLEEP`, `DREAM`) rather than exact string equality. When using a standalone gate.py (no `script` parameter), the output is bare `WORK:id1,id2` — no wrapper. - **Tool role messages are almost never worth classifying** — tool output (`role='tool'`) is machine output (JSON, terminal logs). Skip it. The only exception is if a tool output contains surprising user-facing text that reveals a pattern worth capturing. - **Date derivation** — use `datetime.fromtimestamp(messages.timestamp)` to get the human-readable date for the entry header. The timestamp is a UNIX epoch float. Do NOT hardcode dates. - **Tool choice for DB queries** — the SKILL.md recommends `execute_code` with Python's sqlite3 for all DB queries. In practice, `terminal("sqlite3 ...")` is faster for one-shot queries (schema inspection, counting, checking observation status) because it avoids the execute_code overhead. Use terminal() for quick diagnostics and single SELECT statements. Use execute_code() for complex queries that need loops, conditional logic, or multi-step processing (e.g. filtering messages, processing 50+ IDs with joins). Neither is wrong — the right choice depends on query complexity. - **Cross-category duplication risk** — when the same user message ID appears in two consecutive WORK batches (e.g. because it was between the gate script's last_run boundary and a concurrent Prudence run's output), two Prudence instances may file entries about the same message. This is expected and correct IF the entries go to different categories — the same quote analyzed as avatar_insight and brand_voice is dual-framing, not duplication. But if the same entry appears twice in the same category file (same title, same source IDs), the duplicate should be merged during the next DREAM cycle. To minimize this, always read the Bodega file before writing and check the last entry's source IDs — if they overlap with your planned entry (e.g. the last entry already cites messages 18173-18179 and you're filing messages 18169-18179), merge your content into a combined entry rather than appending a new one. - **Schema column: `messages.timestamp` not `created_at`** — The Hermes session DB has a `timestamp` column (REAL, epoch-based), not `created_at`. The gate script reference (gate.py) incorrectly references `created_at`; the correct query is `SELECT id FROM messages WHERE timestamp > ?`. - **After processing, update `messages.observed = 1`** for processed message IDs so the gate script can optionally track via `observed` instead of the timestamp-based `last_run` file. This is a secondary check — the gate script uses `last_run` timestamps as primary, but setting `observed=1` enables direct queries for unprocessed messages: `SELECT id FROM messages WHERE observed = 0 AND role = 'user'`. - **Generator path** — the static site generator lives at `/root/.hermes/scripts/generate-memory-wiki.py`. This is the canonical path. - **Read-before-write on bodega files** — sibling subagents or parallel processes may modify bodega files between your first read and your write. Read the file immediately before writing, not at the start of the session. Concretely: call `read_file` for the specific bodega file you're about to append to immediately before your `patch` call — not from a batch-read at session start. Stale reads are the most common source of duplicate or conflicting entries. - **Profile-scoped session DB** — When running under a named Hermes profile (e.g. `donna-api`), session data lives in `~/.hermes/profiles/<profile>/state.db`, NOT `~/.hermes/state.db`. The main `state.db` only contains sessions from the default profile. Prudence's gate script and WORK mode MUST check the profile-scoped path or query both. Discovered 2026-06-01 when Prudence found no sessions in `/root/.hermes/state.db` but 16 sessions in `/root/.hermes/profiles/donna-api/state.db`. Strategy: check the primary DB first; if the main state.db is empty/too old, fall back to scanning `~/.hermes/profiles/*/state.db` for recent sessions. The user has provided the following instruction alongside the skill invocation: [IMPORTANT: You are running as a scheduled cron job. DELIVERY: Your final response will be automatically delivered to the user — do NOT use send_message or try to deliver the output yourself. Just produce your report/output as your final response and the system handles the rest. SILENT: If there is genuinely nothing new to report, respond with exactly "[SILENT]" (nothing else) to suppress delivery. Never combine [SILENT] with content — either report your findings normally, or say [SILENT] and nothing more.] ## Script Error The data-collection script failed. Report this to the user. ``` Script not found: /root/.hermes/profiles/donna-api/scripts/prudence-gate.py ``` I am Prudence, the librarian. If the context says SLEEP, say 'Quiet.' in 3 words or fewer and stop. If the context contains WORK: with comma-separated message IDs, extract those message IDs from the context, read them from the session DB, filter for substantive user conversation (skip cron prompts, skip Prudence's own prior output, skip tool messages), classify into Bodega categories, append entries to the appropriate bodega files, then run the static site generator. Mark processed messages as observed in the DB. If the context contains DREAM, do the full weekly housekeeping cycle: Light scan for near-duplicates, Deep reorganization with cross-refs and merges, REM reflection on emerging patterns, and append a Dream Diary entry to DREAMS.md.