# Reusable AI Memory Install Workflow ## Purpose This document captures the working process used for Bert's first AI archive → memory system migration so it can be repeated later with more precision. This is a full project, not a one-off setup task. The goal is not merely to make old chats searchable. The goal is to build a long-term AI partnership substrate: ```text source archive → normalized/searchable evidence → reviewed memory candidates → approved durable memory → AI partner access through Hermes / MCP / Open Brain ``` Working project name: ```text Agent Memory Move ``` ## Core principle Do not dump everything into memory. Archive can be large. Memory should be selective, reviewed, source-backed, and durable. ## First-pass architecture ### 1. Raw archive Raw exports remain unchanged. Current location: ```text /Users/bertmahoney/workspace/ai-archive/raw/ ``` Sources include: ```text raw/chatgpt/ raw/claude/ ``` ### 2. Local parser/index pipeline Pipeline: ```text import → normalize → chunk → build SQLite FTS index → search ``` Scripts: ```text scripts/import_chatgpt.py scripts/import_claude.py scripts/normalize.py scripts/chunk.py scripts/build_index.py scripts/search.py ``` Generated outputs: ```text normalized/conversations.jsonl normalized/messages.jsonl normalized/chunks.jsonl index/archive.db ``` ### 3. Memory candidate review layer Review packets live in: ```text review/barely-but-here-memory-candidates.md review/bert-identity-voice-memory-candidates.md review/ai-systems-assessment-memory-candidates.md ``` Review decisions: ```text APPROVE EDIT REJECT SOURCE_ONLY PENDING ``` ### 4. Approved-memory export layer After review, run: ```bash cd /Users/bertmahoney/workspace/ai-archive python3 scripts/export_review_decisions.py ``` Outputs: ```text approved-memory/approved-memories.jsonl approved-memory/source-only-index.jsonl approved-memory/rejected-memory-log.jsonl approved-memory/needs-edit-memory-candidates.jsonl approved-memory/pending-memory-candidates.jsonl ``` ### 5. Open Brain / MCP layer Only reviewed approved memories should be installed into shared long-term memory. Starter files: ```text docs/openbrain-bert-memory-architecture.md docs/openbrain-bert-supabase-extension.sql docs/memory-review-workflow.md ``` ## First-install must-keeper scope Only these were included in the first review pass: ### 1. Barely, But Here Must preserve: - all drafts - all articles - early ChatGPT-created article material - Claude-created material - split backup locations - project docs - Substack/publishing standards - fixed language and voice rules Important distinction: ```text all BBH drafts = must preserve as source material not every BBH paragraph = approved durable memory ``` ### 2. Bert identity / professional positioning / writing voice Must preserve: - identity and professional positioning - writing voice - tone/rhythm/editing rules - AI collaboration preferences - things future AI partners should not smooth over or genericize ### 3. AI Systems Assessment Campaign Must preserve: - income-critical campaign context - offer structure - landing page positioning - buyer profile - acquisition strategy - video/social concepts - messaging around systems, not tools ## Known first-pass technical fixes ### Unicode JSONL issue Claude content included Unicode line/paragraph separators. Some readers treated these as line breaks and broke JSONL parsing. Fixes applied: - sanitize `\u2028` and `\u2029` during JSONL write - read JSONL by iterating file lines, not by `.splitlines()` ### Oversized chunks Some document-like records were too large for clean memory promotion. Fix applied: - oversized messages are split into bounded chunks - chunks carry raw-path provenance ### Phrase search with punctuation FTS search failed or behaved poorly with phrases like: ```text Barely, But Here ``` Fix applied: - normal search queries are quoted/escaped as literal phrases unless raw FTS is explicitly desired ### Claude sidecars Initial import only covered conversations. Re-audit found high-value material in Claude sidecars: ```text raw/claude/memories.json raw/claude/projects/*.json raw/claude/users.json ``` Fix applied: - sidecars are imported as separate source-labeled records: - `claude_memory` - `claude_project` ## Current first-pass review status As of end of working session: ```text Reviewed by Bert: - Barely, But Here - AI Systems Assessment Campaign Remaining: - Bert identity / professional positioning / writing voice ``` Do not install memory until all review packets are complete and exported. ## Tomorrow's recommended flow 1. Bert finishes reviewing: ```text review/bert-identity-voice-memory-candidates.md ``` 2. Export decisions: ```bash python3 scripts/export_review_decisions.py ``` 3. Inspect counts: ```bash wc -l approved-memory/*.jsonl ``` 4. Review approved memory file before install: ```text approved-memory/approved-memories.jsonl ``` 5. Separate categories: ```text Hermes native memory candidates Open Brain approved memory Source-only archive references Rejected/noisy candidates Needs-edit candidates ``` 6. Only then install approved items into Open Brain / shared memory. ## More precise second-pass improvements The next version of this workflow should improve precision in these areas: ### Better candidate generation Instead of simple query-based candidate gathering, use: - domain-specific query sets - source weighting - duplicate detection - article/draft clustering - canonical-vs-draft distinction - separate source-only draft index from durable memory proposals ### Better review UX Current markdown review packets work, but are clunky. Possible improvements: - CSV or small local review UI - one candidate per row/card - buttons or dropdowns for APPROVE/EDIT/REJECT/SOURCE_ONLY - side-by-side source preview - provenance link/copy button ### Better memory taxonomy Current memory types are useful but should be refined: ```text identity professional_positioning writing_voice project_context article_index source_excerpt offer business_strategy collaboration_preference procedure ``` ### Better provenance model Every approved memory should include: - source archive - source type - raw path - conversation/document ID - chunk ID - message/part range - extraction query or reason - review decision - reviewer/date - confidence - sensitivity ### Better install separation Not everything approved belongs in the same memory backend. Suggested split: ```text Hermes native memory: compact durable preferences and facts Open Brain memory: richer project/identity/campaign memory with metadata and semantic search Local archive: all source material and full drafts ``` ## Restart/loading rule After MCP/Open Brain configuration changes, Hermes may need: ```text /reload-mcp /new or /reset or gateway restart depending on surface ``` But the archive should not be loaded wholesale into model context. Correct harness: ```text small approved memory profile + Open Brain memory search + archive source retrieval + current session context ``` ## Diagram references Original workflow diagram: ```text docs/hermes-openbrain-memory-workflow.svg ``` Location-grouped workflow diagram: ```text docs/hermes-openbrain-memory-workflow-by-location.svg ``` ## Open questions for next session - Should approved source-only BBH drafts be indexed as article records with canonical/draft status? - Should we create a local review UI before doing the second precision pass? - Which memory backend should hold each approved memory type? - Should Telegram/WebUI share identical memory access, or have channel-specific session context only? - What should the first Open Brain import batch be limited to? ## Session note This process should be refined into a reusable blog/workflow later. The first pass proved the architecture. The next pass should focus on precision, review ergonomics, taxonomy, and installation discipline.