# DGX Spark: Week One Update - Finding the Right Stack <div class="callout" data-callout="info"> <div class="callout-title">Series Context</div> <div class="callout-content"> This is a follow-up to my Day 4 post where I documented production challenges with the DGX Spark. After systematic debugging with Claude Code, I found the right software configuration for the DGX Spark's ARM64 + Blackwell architecture. This update corrects my initial pessimistic assessment and shares concrete performance data and solutions for early adopters working with this bleeding-edge platform. </div> </div> **Read Day 4 first:** [[dgx-lab-benchmarks-vs-reality-day-4|DGX Lab: When Benchmark Numbers Meet Production Reality - Day 4]] **TL;DR:** My initial post about DGX Spark challenges was more pessimistic than warranted. After systematic debugging, I found the right software configuration. The hardware is powerful, the learning curve is steep, and working with bleeding-edge ARM64 + Blackwell has been exactly the kind of challenge I was looking for. ## What Changed Yesterday after my initial post about production challenges, I did a deep dive with Claude Code to systematically work through every issue I documented. The conclusion? Most of my problems weren't hardware limitations. They were configuration issues. The core problem: I was using PyTorch wheels compiled for CUDA 12.8 (cu128). The DGX Spark ships with CUDA 13.0, but the PyTorch cu128 wheels include an older ptxas compiler that doesn't recognize GB10's compute capability (sm_121). When I tried to use torch.compile(), I got: ``` PTXASError: ptxas fatal: Value 'sm_121a' is not defined for option 'gpu-name' ``` Switching to PyTorch compiled for CUDA 13.0 (cu130 wheels) fixed this completely. But the real discovery was testing NVIDIA's NGC containers. ## Performance Testing Results I ran benchmarks on a 120M parameter transformer model across four different PyTorch configurations: | Configuration | Eager Mode | Compiled | vs Baseline | |--------------|-----------|----------|-------------| | PyTorch cu128 | Crash | Crash | N/A | | PyTorch cu130 | 48.6ms | 38.7ms | 1.0x | | PyTorch nightly cu130 | 43.5ms | 30.7ms | 1.6x | | NGC Container 25.09 | 13.7ms | 13.5ms | 3.6x | <div class="callout" data-callout="success"> <div class="callout-title">Performance Breakthrough</div> <div class="callout-content"> The NGC container wasn't just faster. It was transformatively faster. For a 10K step training run, that's the difference between 8.1 hours and 2.3 hours. Not because of torch.compile() magic, but because NVIDIA has optimized the entire stack for this specific hardware. </div> </div> ## What Actually Works Now **Training:** - NGC containers for production workloads - PyTorch nightly cu130 for development - Standard cache clearing patterns still recommended **Inference:** - Ollama for GPU-accelerated serving (works perfectly on ARM64) - CPU inference as fallback when needed - vLLM doesn't work on ARM64 + CUDA 13.0 due to library mismatches **Memory Management:** - The fragmentation issue is real but manageable - `torch.cuda.empty_cache()` every 50 steps - Limit long-running sessions to 2-3 hours - Monitor VRAM usage ## The ARM64 + Blackwell Reality Here's what I understand now about this platform: it's genuinely new territory. The GB10 Blackwell was announced in January 2025. CUDA 13.0 was released in August 2025. ARM64 server DGX is a new product line for NVIDIA. The combination of all three means limited real-world testing and some rough edges. That's not a complaint. That's the reality of working with cutting-edge hardware, and it's exactly what makes this interesting. <div class="callout" data-callout="tip"> <div class="callout-title">Technical Insight</div> <div class="callout-content"> The memory fragmentation issue exists at the CUDA runtime level. Building PyTorch from source wouldn't help because the issue is below the PyTorch layer. Docker containers have near-zero GPU overhead on Linux, so there's no performance penalty for using NGC's pre-optimized builds. </div> </div> I expect the CUDA/driver ecosystem to mature over the next 6-12 months. When driver 590.x or 600.x releases, many of these edge cases will likely disappear. But right now, if you know the workarounds, the system is absolutely usable for production work. ## Lessons from Debugging Working through this with Claude Code was actually fun. Not in a "haha fun" way, but in the way that systematically tracking down root causes in complex systems is satisfying. Every issue had a reason. Every failure pointed to something specific. The debugging process taught me more about the ML software stack (application → PyTorch → CUDA runtime → driver → hardware) than I've learned in months of just using working systems. When something fails at the CUDA layer, you learn exactly how these layers interact. ## Correcting My Initial Take My [[dgx-lab-benchmarks-vs-reality-day-4|Day 4 post about benchmark reality]] was too negative. I was frustrated by unexpected failures and interpreted them as fundamental hardware issues. After investigation, most were software configuration problems with clear solutions. Before hitting these issues, I documented several successful implementations showing the system working well: [[dgx-lab-intelligent-gateway-heuristics-vs-ml-day-1|building an intelligent AI gateway]], [[dgx-lab-supercharged-bashrc-ml-workflows-day-2|creating ML workflow optimizations]], and [[dgx-lab-building-complete-rag-infrastructure-day-3|deploying a complete RAG infrastructure]]. The Day 4 challenges were real, but they didn't invalidate the earlier successes. The benchmarks NVIDIA publishes are accurate. I just wasn't running the right stack to achieve them. Once I switched to CUDA 13.0 and NGC containers, the performance matched expectations. **What I got wrong:** - Training wasn't broken (it worked fine all along) - Inference issues were CUDA version related, not fundamental flaws - llama.cpp and Ollama work well on this platform - The "15 hours debugging training failures" were inference testing failures **What I got right:** - Memory fragmentation is real and requires workarounds - Long training sessions need careful management - CPU inference is more stable than GPU for certain workloads - This platform requires expertise to configure correctly ## Why This Matters I bought this system specifically because it's bleeding edge. I want to work with the newest GPU architecture. I want to understand ARM64 ML infrastructure before it becomes mainstream. I want to figure out optimization patterns that will matter in six months when this stack is mature. NVIDIA is building something genuinely new here. ARM64 server GPUs with Blackwell architecture represent a different approach to ML infrastructure. There's a learning curve, yes. But the performance potential is real, and getting in early means understanding the platform deeply. Working with Claude Code through this process made it educational rather than frustrating. Having an AI pair programmer that could systematically work through CUDA versions, benchmark different configurations, and help document findings turned debugging into research. <div class="callout" data-callout="note"> <div class="callout-title">Real-World Impact</div> <div class="callout-content"> I'm working on model fine-tuning and development with substantive applications in oncology and medicine. The performance difference between 8-hour and 2-hour training runs isn't just convenience. It changes how quickly I can iterate, test hypotheses, and refine approaches. When you can run three experiments in a day instead of one, the velocity of progress fundamentally shifts. </div> </div> The debugging process itself taught me more about the ML software stack than months of just using working systems. Claude Code and I spent about 15 hours systematically working through CUDA versions, container configurations, and memory management patterns. We documented everything: which PyTorch builds work, which crash, what the error messages actually mean, how the software stack layers interact. This wasn't wasted time. Understanding the platform at this level means I know exactly what's happening when something goes wrong. I know which layer of the stack to investigate. I know when an issue is in my code versus when it's a CUDA runtime bug that needs a driver update. That knowledge is valuable for everything I'll build on this system. When ARM64 ML infrastructure is standard in two years, I'll have two years of experience optimizing for it. That's the value of working with bleeding-edge platforms. ## Current State Right now, my DGX Spark runs: - CUDA 13.0 - NGC PyTorch containers for training - Ollama for inference serving - PyTorch nightly cu130 for quick tests - Monitoring scripts for driver updates Training is fast. Inference is stable. The system feels solid for my workloads. The rough edges are still there. Memory management requires attention. Long sessions need monitoring. But these are manageable engineering challenges, not fundamental blockers. ## For Other Early Adopters <div class="callout" data-callout="tip"> <div class="callout-title">Configuration Checklist</div> <div class="callout-content"> If you're hitting similar issues, try this: 1. Verify your CUDA version (13.0 matters for Blackwell) 2. Test NGC containers (major performance difference) 3. Use Ollama instead of vLLM on ARM64 4. Implement cache clearing patterns 5. Monitor VRAM closely during long runs Your exact configuration might differ, but the pattern holds: get the software stack right for your hardware, and the performance is there. </div> </div> The system isn't "not ready." It's ready for people willing to learn the platform and work with bleeding-edge software stacks. That's a feature, not a bug. --- **Technical Environment:** - Hardware: NVIDIA DGX Spark (GB10 Blackwell, ARM64) - CUDA: 13.0 (critical for sm_121 support) - Primary: NGC PyTorch 25.09 container - Alternative: PyTorch 2.10.0.dev nightly cu130 - Inference: Ollama (GPU-accelerated) - Tools: Claude Code for systematic debugging **Current Projects:** - Gemma 2 9B fine-tuning for medical QA - Specialized oncology domain adaptation - Multi-model inference pipeline development ## Further Reading <div class="quick-nav"> ### DGX Lab Series - [[three-days-to-build-ai-research-lab-dgx-claude|Three Days to Build an AI Research Lab]] - The journey begins - [[dgx-lab-intelligent-gateway-heuristics-vs-ml-day-1|Day 1: Intelligent Gateway]] - When simple heuristics beat ML - [[dgx-lab-supercharged-bashrc-ml-workflows-day-2|Day 2: ML Workflow Optimizations]] - Shell productivity - [[dgx-lab-building-complete-rag-infrastructure-day-3|Day 3: RAG Infrastructure]] - Production deployment - [[dgx-lab-benchmarks-vs-reality-day-4|Day 4: Benchmarks vs Reality]] - The initial challenges ### Related Topics - [[agent-architectures-with-mcp|Agent Architectures with MCP]] - [[cline-roo-code-quick-start|Cline & Roo Code Quick Start]] </div> --- ### Related Articles - [[model-context-protocol-implementation|Implementing Model Context Protocol (MCP) Across AI Coding Assistants]] - [[building-markdown-rag-system|Building a Markdown RAG System: A Practical Guide to Document-Grounded AI]] - [[manus-im-system-architecture|Inside Manus.im: The Elegant Architecture Behind a Powerful AI Agent]] --- <p style="text-align: center;"><strong>About the Author</strong>: Justin Johnson builds AI systems and writes about practical AI development.</p> <p style="text-align: center;"><a href="https://justinhjohnson.com">justinhjohnson.com</a> | <a href="https://twitter.com/bioinfo">Twitter</a> | <a href="https://www.linkedin.com/in/justinhaywardjohnson/">LinkedIn</a> | <a href="https://rundatarun.io">Run Data Run</a> | <a href="https://subscribe.rundatarun.io">Subscribe</a></p>