AI Use Case Meeting · 2026-06-08
A Model-Agnostic
Memory System
Building personal AI memory that doesn’t belong to a vendor
Brett Pollak · Executive Director, Workplace Technology
UC San Diego
The Problem

Generic AI is generic.
Useful AI knows your world.

“Who should I follow up with this week?” ASK A CHATBOT their memory of you 💬 YOUR CHATS (in their cloud) AI 🔒 “Based on past chats: maybe Sarah?” + KNOWLEDGE LAYER memory · context · history SAME MODEL · DIFFERENT BRAIN ASK YOUR AGENT your world 📅 CALENDAR 💬 MEETINGS 📋 PROJECTS 🏫 CAMPUS AI “Sarah — RFP review still waiting on her since Tuesday.”
  • ▸ Anthropic, OpenAI, and Google are racing to make context the moat — by integrating your personal and enterprise data into their memory systems.
  • ▸ The more useful they get, the harder it is to leave.
  • ▸ This talk is about building that memory layer as yours instead.

What is OpenClaw?

Open Source · MIT

An open-source agent runtime that connects chat, tools, memory, and automation into one self-hosted system. 378k+ GitHub stars · MIT licensed · runs entirely on your infrastructure.

OPEN CLAW 🧠 35+ MODELS 💬 CHANNELS 🔧 TOOLS ⚡ AGENTS 📎 MEDIA 📱 NODES ⏰ CRON
Self-Hosted Gateway
Runs on your hardware — Mac, Linux, server. Your data never leaves your control. MIT licensed, open source.
Multi-Channel
One gateway serves WhatsApp, Telegram, Discord, iMessage, Slack, Teams — simultaneously.
35+ Model Providers
Anthropic, OpenAI, Google, self-hosted (Ollama, vLLM, SGLang), or any OpenAI-compatible endpoint. Multi-model routing with automatic failover.
Agent-Native Runtime
Built-in tool use, sessions, persistent memory, cron scheduling, sub-agent orchestration, workflow pipelines.
MCP & Tool Ecosystem
First-class Model Context Protocol support. Wire in calendars, email, internal APIs, and custom tools — extensible without forking the runtime.
Workspace Files
SOUL.md, AGENTS.md, USER.md, MEMORY.md, plus daily memory logs — identity, rules, context, and curated long-term memory in plain markdown.
The Architecture

One pipeline, three components

Data sources → Knowledge & memory → Agent actions pipeline
  • Three components, one pipelineData sources → knowledge & memory → agent actions. Each piece is independently swappable.
  • Knowledge is the moatMost personal-AI projects stop at “connect to email.” The interesting work is what comes next: capture, synthesize, retrieve.
  • Synthesis is where the value lives78 cron jobs turn raw signals into durable knowledge — not just chat, a daily institutional memory build.
  • Mission Control — the surface I work fromNext.js dashboard at mission-control.vercel.app. Cron outputs land as task cards, decisions queue for review, jobs report status. Telegram pushes the urgent; the dashboard holds the rest.
  • Open formats, not vendor lockMarkdown + JSON on disk. Switch tools, keep your knowledge.
Context Loading

Three layers — and the one most personal AI projects skip

Three concentric layers: always loaded, triggered, explicit
  • Layer 1 — Always loaded (~15 KB)Identity, user profile, work patterns, today + yesterday. Cheap, fixed, every interaction.
  • Layer 2 — Triggered (270 wiki pages)Mention a colleague → reads their wiki page. Mention a vendor → reads tech-stack page. The biggest unlock.
  • Layer 3 — Explicit onlyFull transcripts, raw graph JSON, dated memory files. Rare deep-dives.
  • Most projects skip Layer 2Without triggered retrieval, an accumulating wiki sits unused. The agent only knows what fits in the opening prompt.
The Pipeline

78 cron jobs on a predictable daily rhythm

24-hour clock showing accumulation, synthesis, and consolidation jobs
  • 🌅 Morning (6–9 AM)Calendar briefing, AI news digest, opportunity scan, campus pain monitor, wiki ingest. Everything ready before I open a laptop.
  • 🌆 Evening (5:30–7:35 PM)Email triage with drafts, meeting debrief, daily reflection, context promotion to long-term memory.
  • 🌙 Night (3 AM)Memory consolidation — short-term → long-term, the way sleep does for humans.
  • 📅 Weekly (Sunday)Cross-day pattern extraction. “VPN failed Mon, Wed, Fri” → “VPN is systemically broken this week.”
  • The rhythm is the pipelineNot the individual jobs — the order they fire in is what turns raw files into institutional knowledge.
What lands on my screen — Morning

Six briefings before I open a laptop

📱 Telegram · OpenClaw Today  ·  6:30 AM → 9:00 AM
📅 Calendar briefing  ·  6:30 AM
4 meetings today. Hari at 10 — project decision pending. 11–1 free for deep work. Coffee with Doris cancelled, want me to rebook?
📰 AI news digest  ·  7:00 AM
9 items today. Top three:
• OpenAI quietly bumps GPT-5.5 reasoning — 2M context now default
• Anthropic ships MCP gateway updates relevant to your auth audit
• Google deprecates Gemma 3 (you migrated last week — no action)
brettcpollak.com/ai-digest  ↗
🎯 Opportunity scanner  ·  7:30 AM
Campus IT is rolling out a new SSO config — relevant to your OpenClaw auth audit thread (last touched Apr 22). Linking the wiki page in case it’s useful in your 10 AM with Hari.
🚨 Campus pain monitor  ·  8:00 AM
VPN failed for 12 staff over 24h — mostly engineering school, 3rd time this week. Pattern looks systemic, not user-error. Worth flagging in your 1:1 with Wayne.
📚 Wiki ingest queue  ·  8:30 AM
3 new wiki pages drafted from yesterday’s interactions: people/sarah-chen.md, vendors/snowflake.md, decisions/2026-04-29-budget.md — review when ready.
📨 UCSD AI Weekly  ·  Mondays 9:00 AM
This week’s edition is published — 8 tool updates, 2 trainings, TritonAI migration recap. brettcpollak.com/ucsd-ai-news  ↗

Same items also queue as cards on Mission Control ↗ — Telegram for the urgent, the dashboard for the rest.

The Inference Layer

Model allocation — matching jobs to capabilities

All 69 active jobs run on TritonAI (UCSD’s institutional gateway, on-prem at SDSC) — zero vendor lock-in. 78 total configured, 9 disabled/deferred. 100% model-agnostic routing through TritonAI’s LiteLLM gateway.

TierAliasProvider · ModelUse & why this model fitsJobs
Heavy agentic gpt-oss-120b TritonAI · OpenAI gpt-oss 120B
Briefings · opportunity scans · weekly signal synthesis · backlog triage · LinkedIn candidates  — Best value: powerful single-shot reasoning at TritonAI open-source cost. Handles multi-step agentic work across the full pipeline.
 22
Lightweight sync mistral-small-3.2 TritonAI · Mistral Small 3.2
Token refresh · Gmail/Drive sync · confluence sync · disk watchdog · API health pings  — Lowest latency on TritonAI, cheap per-call — perfect for stateless hygiene work that fires every 15-60 min.
 18
Mid-tier synthesis gemma-4 TritonAI · Gemma 4 26B
Daily reflection · nightly context promotion · wiki ingest · team knowledge ops · evening wrap  — Strong multimodal + 1M context window — handles synthesis, image understanding, and multi-source reasoning at low cost.
 16
Long-form generation mistral-large-3 TritonAI · Mistral Large 3 675B
UCSD AI newsletter · LinkedIn post drafts · architecture reviews · vision tracking  — Best narrative flow on TritonAI — preserves voice across long-form prose without sounding generic.
 6
Deep reasoning sonnet TritonAI · Claude Sonnet 4.6
Email triage · weekly AI deep-dive · identity drift review · provider quota monitoring  — Reserved for jobs that need careful analysis, multi-source verification, and nuanced judgment — worth the premium for low-frequency high-stakes work.
 5
Specialized code deepseek-v4 TritonAI · DeepSeek V4 Flash Max
Overnight code maintenance  — Code-specific reasoning that handles refactoring, dependency updates, and repo hygiene without drift.
 1
Health check gpt-oss-120b TritonAI · OpenAI gpt-oss 120B
Henry API health ping  — Same model as heavy agentic tier; just a single hourly status check that doesn’t need its own category.
 1
System default default TritonAI · DeepSeek V4 Flash Max
Feature reminder  — Unpinned job inherits the system default model. One catch-all that keeps complexity low.
 1
Cost across the ecosystem

Same 69 active jobs, three pricing scenarios

Read each column top-to-bottom — vendor totals up top, then how Core memory and Brett-specific jobs break down underneath.

Model-agnostic gateway
Anthropic only
OpenAI only
~$16/mo
69 jobs · actual TritonAI published API rates
Core memory · 23 jobs
~$5/mo
Mostly open-weight on-prem (Gemma 4, GPT-OSS, Mistral Small) · 1 Sonnet escalation · near-zero marginal cost
Brett-specific · 46 jobs
~$11/mo
Open-weight synthesis + agentic work on GPT-OSS · Mistral Large for long-form · 2 Sonnet escalations · zero vendor lock-in
~$130/mo  8×
69 jobs · single-vendor lock-in
Core memory · 23 jobs
~$35/mo  7×
Haiku · Sonnet · Opus across all 23 jobs · no open-source savings · no cross-provider diversity
Brett-specific · 46 jobs
~$95/mo  8.6×
All synthesis on Sonnet · long-form on Sonnet · Opus deep-dive · no open-source fallback
~$160/mo  10×
69 jobs · single-vendor lock-in
Core memory · 23 jobs
~$40/mo  8×
gpt-5.5-mini for hygiene · gpt-5.5 for agentic · no open-source savings
Brett-specific · 46 jobs
~$120/mo  11×
Synthesis + long-form on gpt-5.5 · gpt-5.5 for deep-dives · no model diversity

On-prem open-weight models on UCSD GPUs at SDSC have ~$0/mo marginal cost (sunk institutional infra). TritonAI column uses actual published API rates from model_hub_table as of June 2026. Comparison columns use published Anthropic/OpenAI API rates on the same estimated token consumption. Core vs Brett-specific split estimated from job allocation patterns.

What lands — Evening & weekly

End-of-day synthesis becomes long-term memory

Each card below is what actually lands in the corresponding output file or feed.

5:30 PM📧 Email triage + drafts
reply-priority5 messages need a response today — Doris (intake), Hari (project), Wayne (1:1 prep), 2 vendor follow-ups.
drafts-ready3 drafts saved to my Outlook Drafts folder via MS Graph — open the email, review, send. Review beats writing from scratch.
unsubscribe8 newsletters proposed for cleanup based on 30-day open rate.
6:00 PM🎙 Meeting debrief
From today’s meeting transcripts → 4 action items extracted, 2 attendee wikis updated (sarah-chen, doris-tao), 5 new edges added to the knowledge graph, 1 architecture decision logged to decisions/.
7:00 PM🪞 Daily reflection
Today: 3 meetings, 1 architecture decision, 2 follow-ups due Friday.
Pattern noticed: third week of intake delays — worth a process retro.
Promoted to long-term wiki: 4 facts, 2 relationship updates.
3:00 AM🌙 Memory consolidation
12 short-term notes promoted, 3 stale entries pruned, 47 wiki backlinks + 23 knowledge-graph edges reindexed. The way sleep does it for humans — runs while I’m off.
SUN 8 AM📊 Weekly pattern extraction
“VPN issues spiked Mon/Wed/Fri — systemic, not user error.” Drafted a Henry ticket. Cross-day patterns surfaced by traversing the knowledge graph — connections no daily report would catch.
SUN 10 AM✍️ LinkedIn weekly drafts
3 post drafts in my voice, grounded in this week’s actual decisions and work — review beats drafting from scratch.

All of these also land as reviewable cards on Mission Control ↗ — the dashboard where I triage, accept, defer, or kill each item.

Outcomes

What the system actually delivers

2–3
hours / week saved
Research, prep, triage, news scanning, follow-up tracking — the crons gather; I do the judgment.
270
wiki pages indexed
Triggered context loading — the layer most personal-AI projects skip. 80 new pages since May 1.
78
cron jobs configured
Synthesis pipeline running 24/7 — 69 active, 9 disabled/deferred. Morning briefings, evening reflection, night consolidation.
∞
cross-session continuity
Decisions write back. Patterns persist. Future sessions inherit what past ones learned.
📈Pattern recognition I’d miss — “VPN failed 5 of 7 days” doesn’t surface in any daily report. Weekly synthesis surfaces it automatically.
🧠Institutional memory — “Why did we pick this over that?” has an answer. “When did we last discuss X?” has a date and a transcript link.
🎯Never starting from zero — every conversation begins with 48 hours of context, curated long-term memory, and triggered wiki lookups.
Forward-Looking

What else becomes possible — the foundation is in place

📍 Now  ·  Proven daily
78 cron jobs configured (69 active)Morning briefings, evening synthesis, night consolidation, weekly patterns — plus 9 experimental or seasonal jobs ready to enable.
Triggered wiki retrieval270 pages, loaded on demand based on what the conversation references.
Cross-session continuityDecisions and patterns persist across days, weeks, and threads.
⏭ Next 30 days  ·  Implementation
📋 Meeting prep automationAuto-compile attendee wikis, recent interactions, and open commitments before every meeting. Walk in warm.
🔔 Proactive nudges“This problem has come up three weeks in a row — here are the options you’ve already explored.”
✍️ Draft-first comms expansionMemos, project briefs, status updates — grounded in actual context, not generic AI writing.
🌅 Beyond  ·  What unlocks
🏛 Decision archaeology“Why did we pick this two years ago?” → immediate answer with full reasoning, the voices that disagreed, the tradeoffs.
👥 Team-level memoryOpt-in shared wikis. Team patterns surface without crossing personal boundaries.
🏫 Department insightsAnonymized aggregates: “trending pain points this week” from many agents, never individual records.

Most of these are implementation work, not research. The hard part — the knowledge layer — is already built.

The Vision

Scaling to UC San Diego — what if every staff member had one?

Four-tier scaling: personal, team, department, campus
  • 1 · Personal — own agent, own dataProven daily. Every staff member gets dedicated runtime, isolated data store, governance default of “this is yours.”
  • 2 · Team — opt-in sharing onlyTeam wiki visible to members; 1:1s and personal notes stay private. Nothing crosses the boundary automatically.
  • 3 · Department — anonymized aggregates“Trending pain points this week” — patterns from many agents, not records. Identity stripped before aggregation.
  • 4 · Campus — statistical patterns onlyConnective tissue across departments. Never individual records, never attributable content.
  • 🛡️ Privacy is the preconditionSharing boundaries are first-class features, not bolt-ons. Without that guarantee at every level, nothing ships.
Discussion
Where do we go from here?
1. Data sources & MCP
These tools are clearly valuable for this kind of work — which campus data sources should we be connecting through MCP, and where would those integrations move the needle most?
2. Governing access
How do we govern access to those data sources at the campus level — sanctioned MCP servers, scoped credentials, audit trails, and a sharing model staff and faculty can trust?
3. Over to Shawn Munro
Shawn is going to talk next about opportunities to address this through our APIs and hosting platforms — making these patterns first-class on UCSD infrastructure rather than one-off personal experiments.
brettcpollak.com/ai-agent-architecture github.com/bpollak/mission-control