# When AI Knows What You Mean, Not Just What You Say This morning, I'm scanning the AI newsletter Clawdbot generates for me every day. A paper on Nemotron-3 catches my eye. "Summarize this." Clawdbot does. Technical depth tuned to what I care about. No fluff. "Share it with the team." Done. Five colleagues get it via their preferred channels (email for some, Slack for others). "Tweet it." Posted. Thread in my voice, no edits needed. Then I switch to something else while the bot deploys Nemotron to my GPU server and runs inference tests. I'm doing situps when it finishes.  ## The Shift From Instructions to Intent I didn't tell the system *how* to do any of this. I told it *what* I wanted. No "use this API" or "format like this" or "my credentials are stored here." The system already knows: - My voice (writing samples, past posts) - My contacts (who uses what channel) - My workflow patterns (what "share" means in different contexts) - My credentials (stored securely, accessed automatically) This isn't better autocomplete. It's not smarter search. It's **ambient intelligence**. Systems that don't wait for explicit commands because they already have context. > [!info] What Makes This Work > It's not AGI. It's not even that complicated. It's skills (modular capabilities) + memory (persistent context) + integration (tool access) + trust (enough autonomy to act without supervision). The AI doesn't "know" me in some mystical sense. It has structured data: writing samples, contact patterns, API keys, calendar, repos. That's enough. ## What Actually Happened Here's what ran this morning: 1. **Newsletter generation** (cron skill): Pulls AI research papers, formats digest, sends email 2. **Paper summarization** (summarize skill): Fetches PDF, extracts key points at my preferred depth 3. **Team sharing** (communication skills): Routes message to five people via their preferred channels 4. **Twitter posting** (bird skill): Generates thread matching my voice, handles auth, posts 5. **Model deployment** (SSH + deployment scripts): Connects to GPU server, pulls model, runs tests Five commands from me. Maybe 30+ sub-tasks executed. Zero explanations required. ## The Infrastructure Finally Caught Up I think we're at an inflection point. Not because the models got smarter (though they did). Because the **infrastructure around them** finally works. The combination of: - Agentic frameworks with skills, memory, and integrations - Persistent context (memory files, workspace state) - Secure credential management (no keys in config) - Tool composability (skills that chain seamlessly) This creates something qualitatively different from "chat with an AI." It's closer to having a technical co-pilot who actually knows your stack, your people, your patterns. Who can act on vague instructions because they have enough context to fill in the blanks correctly. ## The Situp Test Can your AI handle a complex, multi-step workflow while you're literally doing something else? Not "can it generate code while you review." That's just faster autocomplete. I mean: Can you give it a Tuesday morning brain dump of tasks, go do pushups, and come back to find everything done correctly? If yes, you've crossed into ambient intelligence territory. > [!tip] Delegation vs. Supervision > The difference: With autocomplete, you're still the one executing. With ambient intelligence, you describe the outcome and walk away. ## What This Actually Is Look, I'm not claiming this is AGI or the singularity. It's well-orchestrated tooling + decent models + good UX. But it *feels* qualitatively different. And the thing that gets me: it's not some proprietary megacorp platform. It's open tooling. Composable skills. Models I can swap. Infrastructure I control. The whole stack runs on my laptop. The agent has access to my tools, not the other way around. That matters. ## Meta-Moment This blog post? I'm dictating it to Clawdbot on my phone. It's writing directly to my Obsidian vault. Later, I'll refine it with Claude Code. But the first draft happened while I was thinking out loud during my workout. The loop closes. ## Technical Stack The orchestrator here is [Clawdbot](https://clawd.bot/) by [@steipete](https://steipete.me/). It's a mobile-first Claude interface that handles the conversation layer and task routing. But the real power comes from the skills system. Clawdbot ships with built-in skills (like `bird` for Twitter posting), and I've extended it with ~80 custom skills I built using Claude Code. **The Combo:** - **Clawdbot**: Handles conversation, memory, credential management, mobile interface - **Claude Code skills**: Newsletter generation, summarization, deployment scripts, Obsidian integration - **Clawdbot skills**: Twitter (`bird`), communication tools, general automation When I say "tweet this," Clawdbot uses its built-in `bird` skill. When I say "deploy the model," it hands off to my custom deployment skill. Seamless handoff between Clawdbot's capabilities and my custom tooling. **Why This Works:** - Persistent memory (Clawdbot tracks voice, contacts, preferences) - Secure credential management (1Password CLI, no keys in config) - Model-agnostic (Claude Sonnet 4.5 now, but swappable) - Local + remote execution (laptop for orchestration, GPU server for compute) **Why Local Matters:** The agent has access to my tools, not the other way around. It can't phone home. It can't leak credentials. I control what it can do. > [!warning] Security Model > This setup requires trusting the agent framework with your credentials and tool access. Only do this with systems you control and audit. ## What Changed Six months ago, I would've: 1. Read the newsletter manually 2. Copy-pasted the paper into Claude 3. Summarized it myself 4. Written individual messages to five people 5. Drafted a tweet, edited it three times 6. SSH'd into the GPU server manually 7. Run the deployment commands one by one Total time: 90 minutes, spread across the morning. Now: Five voice commands. Eight minutes total. Most of it happening while I'm doing other things. The time savings are nice. But the real shift is **cognitive load**. I don't have to context-switch between tasks. I don't have to remember which colleague prefers email vs. Slack. I don't have to look up the deployment commands. I just describe what I want. The system handles the rest. ## Building This Yourself You don't need my exact stack. The pattern is what matters: 1. **Start with Clawdbot** ([clawd.bot](https://clawd.bot/)) or similar agent framework that supports skills 2. **Use the built-in skills** (Twitter, communication, basic automation) to understand the pattern 3. **Build custom skills** for your specific workflows (Claude Code makes this straightforward) 4. **Add persistent memory** (preferences, contacts, patterns you want the agent to remember) 5. **Start with one workflow** (don't try to automate everything at once) The key: Give the system enough context to make reasonable decisions without asking you every time. If you find yourself typing the same instructions repeatedly, you need more context in memory. If the agent keeps asking for clarification, you need better examples in your prompt or more structured data about your preferences. Shout out to [@steipete](https://steipete.me/) for building Clawdbot. It's the foundation that makes this whole workflow possible. ## The Longer Game This morning's workflow is already outdated. I'm testing multi-agent handoffs (research agent → writing agent → publishing agent). I'm building better memory systems (vector DBs for past decisions, not just flat files). I'm exploring ways to let the agent learn from corrections instead of needing explicit rules. Six months from now, the "five voice commands" will probably be one: "Process my newsletter and handle the interesting stuff." The system will infer what "interesting" means based on what I've engaged with before. It'll know who to share with based on topic relevance. It'll adapt the tweet thread length based on recent engagement. That's the direction. Less instruction, more intent. --- ### Related Articles - [[AI-Development-&-Agents/building-autonomous-ai-agents|Building Autonomous AI Agents: From Concept to Production]] - [[Practical-Applications/claude-code-workflow-optimization|Optimizing Development Workflows with Claude Code]] - [[AI-Systems-&-Architecture/designing-multi-agent-systems|Designing Multi-Agent Systems That Actually Work]] --- <p style="text-align: center;"><strong>About the Author</strong>: Justin Johnson builds AI systems and writes about practical AI development.</p> <p style="text-align: center;"><a href="https://justinhjohnson.com">justinhjohnson.com</a> | <a href="https://twitter.com/bioinfo">Twitter</a> | <a href="https://www.linkedin.com/in/justinhaywardjohnson/">LinkedIn</a> | <a href="https://rundatarun.io">Run Data Run</a> | <a href="https://subscribe.rundatarun.io">Subscribe</a></p>