# Stack Pulsar

> Stack Pulsar publishes practical, engineer-written guides on LLMOps, AI infrastructure, observability, FinOps, and DevOps tooling — calibrated for teams running production AI systems in 2026.

The site is a content companion to the "The Stack Pulse" newsletter. Articles focus on hands-on production patterns: monitoring, cost optimization, reliability engineering, and tooling comparisons for LLM, agent, and AI-infrastructure workloads.

## Key pages

- [Home](https://stackpulsar.com/): site overview and latest posts
- [Blog index](https://stackpulsar.com/blog/): full archive of articles
- [Start Here](https://stackpulsar.com/start-here/): curated entry points by topic
- [Newsletter](https://stackpulsar.com/newsletter/): The Stack Pulse — weekly briefing for AI infrastructure engineers
- [About](https://stackpulsar.com/about/): editorial scope and contact
- [Premium](https://stackpulsar.com/premium/): paid tier and sponsor information
- [RSS feed](https://stackpulsar.com/rss.xml)
- [Sitemap](https://stackpulsar.com/sitemap-index.xml)

## LLMOps

- [AI Coding Agent Observability 2026: Six Tools](https://stackpulsar.com/blog/ai-coding-agent-observability-2026/): Claude Code 1.0, Gemini CLI, Codex CLI, OpenCode, GitHub Copilot, AWS Kiro: per-tool OTel hooks, normalized schema, and a five-panel view
- [Per-Engineer AI Observability 2026: Beat Reflection](https://stackpulsar.com/blog/per-engineer-ai-usage-observability-2026/): OTel + LangSmith + ClickHouse reference schema for per-engineer Claude observability — session cost, four-signal dashboard, 1/3/6/12-month retention
- [Coding Agent Cost Observability 2026: One View](https://stackpulsar.com/blog/coding-agent-cost-observability-2026/): How a normalized OTel GenAI schema caught $1,600 of hidden Claude Code + Cursor overlap on an $11,400 bill — five-panel Grafana cost view
- [LLM Latency Monitoring 2026: TTFT and TPOT](https://stackpulsar.com/blog/llm-latency-monitoring-2026/): Every millisecond your users wait for an LLM response, engagement drops
- [LLM Hallucinations: Five Production Detection Methods](https://stackpulsar.com/blog/how-to-monitor-llm-hallucinations/): Five detection methods with thresholds: regex PII gates, RAGAS faithfulness >0.80, sentence-transformer drift <0.35, judge model, Prometheus SLO burn alerts
- [LLMOps Observability: Latency, Hallucinations, and Drift](https://stackpulsar.com/blog/llmops-observability-blueprint/): A blueprint for LLMOps observability: why HTTP 200 is a lie for LLM apps, the three pillars of LLM health (latency, quality, reliability), and how to…
- [AI Incident Postmortem Template: Four-Question Framework](https://stackpulsar.com/blog/ai-incident-postmortem-template/): When your AI system fails, you need answers fast
- [Five LLM Dashboards That Catch Incidents in 30 Minutes](https://stackpulsar.com/blog/llm-monitoring-dashboard-templates/): Five Grafana panels with the Prometheus queries behind them — token throughput, TTFT/TPOT, cost-per-model, error rates
- [Build Your First LLM Monitoring Stack: OTel + Prometheus](https://stackpulsar.com/blog/llm-monitoring-stack-tutorial/): A practical guide to instrumenting LLM applications with OpenTelemetry, scraping metrics with Prometheus, and visualizing token costs, latency, and quality…
- [Multi-Modal LLM Monitoring in Production: A Practical Guide](https://stackpulsar.com/blog/multimodal-llm-monitoring/): How to monitor vision, audio, and text inputs in multi-modal AI systems
- [LLM Model Drift Detection 2026: Monitoring AI Degradation](https://stackpulsar.com/blog/llm-model-drift-detection/): Our support LLM silently degraded for weeks before any alert fired
- [LLM Evaluation Frameworks: RAGAS, TruLens, and the Stack](https://stackpulsar.com/blog/llm-evaluation-frameworks/): RAGAS scores retrieval faithfulness, TruLens traces attribution per output
- [LLMOps Platform Comparison 2026: Guide to the Leading Tools](https://stackpulsar.com/blog/llmops-platform-comparison-2026/): Hands-on 2026 LLMOps comparison: Braintrust's eval gating vs LangSmith tracing vs Phoenix OTel — pricing tiers, PII gaps, and which scales further
- [Prompt Injection: Detection and Prevention Strategies](https://stackpulsar.com/blog/prompt-injection-detection/): Prompt injection is an active threat in production AI systems
- [LLM Security Hardening 2026: A Defense-in-Depth Guide](https://stackpulsar.com/blog/llm-security-hardening/): Prompt injection, jailbreaking, and model extraction threaten production AI systems
- [Helicone vs Portkey vs LangSmith: LLM Observability 2026](https://stackpulsar.com/blog/llm-observability-tools-2026/): Helicone (100K/mo free), Portkey ($50/mo at 500K routed), LangSmith Plus ($39/seat)
- [Monitoring LLM Hallucinations 2026: AI Engineer Guide](https://stackpulsar.com/blog/llm-hallucination-monitoring/): Hallucinations are the blind spot of LLM monitoring
- [Agentic Observability: Multi-Agent LLM Monitoring](https://stackpulsar.com/blog/agentic-observability/): A practical guide to observability for agentic AI systems — step-level tracing, cost accounting, reliability monitoring, and the four-layer stack you need…
- [RAG Observability 2026: What Matters in Production](https://stackpulsar.com/blog/rag-observability/): A practical guide to monitoring RAG pipelines in production — retrieval precision, context utilization, answer faithfulness, embedding drift, and the…
- [MCP Monitoring: Observability for Model Context Protocol](https://stackpulsar.com/blog/mcp-monitoring/): A practical guide to monitoring MCP (Model Context Protocol) servers in production
- [LLM Observability: Complete Implementation Guide](https://stackpulsar.com/blog/llm-observability-guide/): A practical guide to implementing LLM observability in production
- [vLLM Production Monitoring: A Practical Stack Guide](https://stackpulsar.com/blog/vllm-production-monitoring/): GPU cache utilization, KV cache hit rate, TTFT/TPOT metrics, and a complete Prometheus + Grafana monitoring setup for vLLM inference servers — updated for v0.19
- [Arize Phoenix 15.4.0: Open Source LLM Observability](https://stackpulsar.com/blog/arize-phoenix/): A practitioner's guide to Arize Phoenix 15.4.0: embedding drift detection, RAG trace analysis, the agent toolset, and wiring it into a self-hosted LLM stack
- [Monitor LLMs Without Per-Token Fees: 5 Open-Source Tools](https://stackpulsar.com/blog/open-source-llm-monitoring-stack/): Replaced our $4,200 Helicone bill with OTel + Prometheus + Grafana + Loki + Tempo — $180 flat at 10B tokens/month, PagerDuty SLOs, vLLM p99 1.4s TTFT
- [Beyond the Stack Trace: AI Debugging Paradigm](https://stackpulsar.com/blog/beyond-the-stack-trace-ai-debugging/): Why stack traces fail for non-deterministic AI

## AI Infrastructure

- [Mesh Inference on iroh: GPUs in Three Offices and a Closet](https://stackpulsar.com/blog/mesh-inference-iroh-2026/): Mesh LLM turns scattered GPUs into one OpenAI-compatible API
- [Custom AI Silicon 2026: Meta MTIA, Trainium2, TPU, Maia](https://stackpulsar.com/blog/custom-ai-silicon-comparison-2026/): Meta MTIA 300/450/Iris, AWS Trainium2/Inferentia2, Google TPU v5e/v6, Microsoft Maia 100 vs NVIDIA H100/B100 — vendor-neutral, dollar-per-token
- [Agent Sandbox vs. Agent Substrate: CNCF Runtime 2026](https://stackpulsar.com/blog/agent-sandbox-vs-agent-substrate-2026/): Two CNCF SIG Apps projects — agent-sandbox and agent-substrate — and the OTel pattern that makes both debuggable in production
- [The Agentic Harness for AI Incident Response](https://stackpulsar.com/blog/agentic-incident-harness/): PagerDuty CAIO: AI incident tools are missing a critical layer
- [Agentic Ops Platform 2026: Enterprise Reference Architecture](https://stackpulsar.com/blog/agentic-ops-platform-2026/): Enterprise architecture for 200+ internal AI agents: per-agent RBAC, audit logs, sandboxed tools, prompt-injection defense, and the Kubernetes operator pattern
- [AI Operational Debt 2026: 3 Forms That Break AI Strategy](https://stackpulsar.com/blog/ai-operational-debt/): Prompt debt, eval debt, tool debt: the three forms of operational debt unique to AI systems, and the audit pattern that finds them before they break production
- [The Google Remy Leak: AI Agent Stack Risk in 2026](https://stackpulsar.com/blog/google-remy-leak-enterprise-ai-agents/): Google's Gemini Workspace agent stack leaked via OAuth over-scoping, calendar side-channels, and draft-state recovery — a pattern, not a single CVE
- [AI SLO/SLA Contracts: A Practical Guide for Infra Teams](https://stackpulsar.com/blog/ai-slo-sla-contracts/): TTFT p99 targets, composite SLA math, model deprecation clauses, and RAG recall SLOs — a practitioner's guide to AI service SLAs in production
- [SGLang Production Monitoring: A Complete Practical Guide](https://stackpulsar.com/blog/sglang-production-monitoring/): Monitor SGLang in production: RadixAttention architecture, KV cache metrics, prefill/decode throughput, TTFT, Prometheus + Grafana instrumentation, and a…
- [Cut vLLM GPU Costs 40% with KEDA Queue-Depth Autoscaling](https://stackpulsar.com/blog/kubernetes-ai-autoscaling-keda/): I cut our vLLM GPU bill 40% (8xH100, $11.4k → $6.8k/mo) by replacing HPA with KEDA queue-depth autoscaling
- [vLLM vs Triton: Real H100 Throughput and Migration Cost](https://stackpulsar.com/blog/vllm-triton-comparison/): PagedAttention gives vLLM 2-5x throughput vs Triton on Llama-3 70B H100; Triton wins for multi-model serving
- [Kubernetes GPU Operator: A Production Setup Guide](https://stackpulsar.com/blog/kubernetes-gpu-operators/): GPU pods not scheduling
- [How to Monitor Ollama in Production: The Observability Stack](https://stackpulsar.com/blog/ollama-production-monitoring/): A 4-hour outage from a silent Ollama CPU fallback: TTFT 30x slowdown, every probe green
- [AI Agent Reliability 2026: Failure Modes + Observability](https://stackpulsar.com/blog/ai-agent-reliability-monitoring/): What actually breaks an agent at 2am: four failure modes, the OTel span hierarchy that surfaces them, and the CrewAI v0.5 + LangGraph fixes that work
- [Agentic AI Infrastructure for DevOps and Platform Engineers](https://stackpulsar.com/blog/agentic-ai-infrastructure/): From stateless LLM calls to autonomous multi-step agents — a practical guide to the infrastructure patterns that make agentic AI production-ready
- [LiteLLM Production Monitoring 2026: Gateway + Cost Tracking](https://stackpulsar.com/blog/litellm-production-monitoring/): Monitor LiteLLM 1.95.0-dev.1: critical SSRF fix, Rust gateway routes, MCP DCR, spend controls, signed images, and the current stable pin
- [AI Model Monitoring vs Traditional APM in 2026](https://stackpulsar.com/blog/ai-model-monitoring-vs-apm/): Four fundamental differences between AI and software monitoring — non-deterministic output, token-based cost, multi-component latency, and stateful context…
- [LLM Incident Postmortem 2026: Lessons from AI Failures](https://stackpulsar.com/blog/llm-incident-postmortem/): Real incident retrospectives from legal RAG, medical AI, and customer support AI failures
- [SRE Best Practices for AI/LLM Systems in 2026](https://stackpulsar.com/blog/sre-best-practices-ai-llm-systems/): A practical SRE playbook for operating AI and LLM systems in production
- [vLLM vs TGI vs TensorRT-LLM on H100s: The Benchmarks](https://stackpulsar.com/blog/llm-inference-engine-comparison/): vLLM, TGI, and TRT-LLM benchmarks plus Qwopus3.6 and Neutrino-8B serving paths, VRAM, quantization, and engine support
- [Terraform vs Pulumi: AI Infrastructure Decisions](https://stackpulsar.com/blog/terraform-vs-pulumi-ai-infrastructure/): Terraform vs Pulumi for AI/ML infra: dynamic GPU clusters, Kubernetes, multi-cloud routing, and the Terraform 1.17 alpha caveat
- [State of AI Infrastructure 2026: From Hype to Production](https://stackpulsar.com/blog/state-of-ai-infrastructure-2026/): A practical analysis of the AI infrastructure landscape in 2026 — GPU providers, inference frameworks, SLM adoption, and the FinOps reality check that…
- [The Rise of eBPF 2026: A New Era for System Observability](https://stackpulsar.com/blog/ebpf-observability-guide/): eBPF is rewriting the rules of Linux observability
- [eBPF for AI Networking: GPU Workload Visibility](https://stackpulsar.com/blog/ebpf-for-ai-networking/): How eBPF, Cilium, and Hubble deliver kernel-level observability for AI infrastructure: GPU scheduler events, NCCL/RDMA latency, and inference pod traffic
- [K8s GPU Scheduling: Stop NUMA Crossings Killing Training](https://stackpulsar.com/blog/kubernetes-gpu-scheduling-ml-workloads/): NUMA crossings killed 38% of our 70B training throughput (11d → 7d with NVLink pinning)
- [OpenClaw Reliability: Production AI Agent Patterns](https://stackpulsar.com/blog/openclaw-reliability-guide/): A senior SRE perspective on OpenClaw failure modes in AI agent production environments, with hardening patterns and monitoring strategies for DevOps and AI…
- [GPU Monitoring for AI Inference: A Practical Guide for 2026](https://stackpulsar.com/blog/gpu-monitoring-ai-inference/): Monitor GPU utilization, VRAM, temperature, and power draw for AI inference
- [Vector Database Comparison 2026: Pinecone, Milvus, Weaviate](https://stackpulsar.com/blog/vector-database-comparison-2026/): Pinecone vs Milvus vs Weaviate 1.39 for production RAG: Namespaces, live reindex, REST search, gRPC web, cost, and operational trade-offs
- [vLLM, SGLang, or Ollama: Self-Hosted LLM Engines Benchmarked](https://stackpulsar.com/blog/vllm-sglang-ollama-comparison/): vLLM tops throughput, SGLang adds multi-model routing, Ollama wins on simplicity
- [Monitoring the Unseen: Observability for AI/ML Pipelines](https://stackpulsar.com/blog/ai-ml-pipeline-observability/): LLMs, vector databases, and RAG pipelines introduce new failure modes
- [Multi-LLM Routing: Cut Costs 40% Without Quality Loss](https://stackpulsar.com/blog/multi-llm-routing-ensembling/): 200K requests across GPT-4o, Claude 3.5, and Gemini 2.0 Flash: cost-plus-latency routing saved 44% with no measurable quality drop
- [Multi-Dimensional AI Retrieval: Beyond Vector Search](https://stackpulsar.com/blog/multi-dimensional-ai-retrieval-2026/): Vector search is 30% of production RAG
- [Fine-tuning in Production: The Infrastructure Guide for 2026](https://stackpulsar.com/blog/fine-tuning-in-production/): Axolotl, Unsloth, TRL, QLoRA and the eval pipeline that catches bad checkpoints — the fine-tuning stack that actually works in production in 2026
- [Trainium2 vs Inferentia2: When AWS Custom Silicon Beats H100](https://stackpulsar.com/blog/aws-trainium-inferentia/): 2TB HBM fits Llama 70B on one Trainium2 host: EKS + SageMaker numbers, NeuronLink collectives, Neuron SDK compile times, $0.30/M-token Inferentia2 vs H100
- [Backup and Restore for Vector Databases: A Production Guide](https://stackpulsar.com/blog/backup-restore-for-vector-dbs/): How Pinecone, Weaviate, Qdrant, Milvus, and Chroma handle backup, restore, PITR, and disaster recovery — with concrete RTO/RPO numbers and a runbook
- [Inference API Gateways 2026: LiteLLM vs BentoML vs Ray Serve](https://stackpulsar.com/blog/inference-api-gateways/): A practitioner's comparison of three inference gateway and serving stacks — LiteLLM, BentoML, and Ray Serve — when to use each, and the limits
- [MCP Enterprise Authorization 2026: The Missing Auth Layer](https://stackpulsar.com/blog/mcp-enterprise-authorization-2026/): Cloudflare, Auth0, Stytch and Stargate ship MCP server auth primitives in 2026
- [Agent Evidence Packet Analytics 2026: The Audit Trail](https://stackpulsar.com/blog/agent-evidence-packet-analytics-2026/): EU AI Act Article 12 compliance: the 5-tuple evidence-packet schema, Sigstore Rekor for agent decisions, the Codex encryption crisis, and the replay path

## Observability

- [OpenAI Just Made Your Agent a Black Box (and What to Do)](https://stackpulsar.com/blog/codex-multi-agent-v2-encryption-black-box-2026/): OpenAI's Codex multi-agent-v2 encrypts the parent→subagent payload (July 14, 2026)
- [Agent Observability at 1,200+ Agents: OTel Stack 2026](https://stackpulsar.com/blog/agent-observability-1200-agents-otel-2026/): AWS AgentCore vs
- [Probabilistic Observability 2026: AI Debugging Discipline](https://stackpulsar.com/blog/probabilistic-observability-2026/): The 4 primitives for debugging non-deterministic AI: output distributions, semantic traces, statistical regression, hallucination-as-metric
- [Prometheus vs Grafana: Fix Alert Fatigue and Unknown Pods](https://stackpulsar.com/blog/prometheus-vs-grafana-2026/): Your Kubernetes pods are eating through budget, your on-call is drowning in alert fatigue, and your dashboards show 'unknown' for half your services
- [OpenTelemetry for AI Inference: Tracing LLM Pipelines](https://stackpulsar.com/blog/opentelemetry-ai-inference-tracing/): How to instrument LLM inference pipelines with OpenTelemetry — from prompt injection to token streaming, from model serving to downstream tool calls, using…
- [The State of Observability in 2026: Trends and Tech](https://stackpulsar.com/blog/observability-2026/): 2026 AI observability playbook: OTel + eBPF + vLLM, with SLO targets — TTFT under 1.5s, KV-cache at 95%, RAG top-1 cosine above 0.75 in production

## FinOps

- [Carbon-Aware AI Inference 2026: Cut Energy 30-50%](https://stackpulsar.com/blog/carbon-aware-ai-inference-2026/): A 1000-H100 inference fleet spends $850K/year on electricity
- [AI Cost by Workflow 2026: The Tokenmaxxing Layer](https://stackpulsar.com/blog/ai-token-cost-by-workflow/): Per-workflow token attribution: tag every LLM call with workflow_id, build per-business-process cost dashboards, route workflows to cheaper models
- [AI Coding Agent FinOps 2026: Copilot, Cursor, Devin Cost](https://stackpulsar.com/blog/ai-coding-agent-finops/): Per-engineer token costs, per-LOC and per-PR attribution, anomaly detection, and enterprise policy for AI coding agents: Copilot, Cursor, Devin
- [Multimodal LLM Cost Optimization 2026: Vision and Audio AI](https://stackpulsar.com/blog/multimodal-llm-cost-optimization/): GPT-4V costs 4x GPT-4o, a 1024px image can burn 16K tokens
- [AWS Savings Plans vs Reserved Instances: 2026 FinOps Guide](https://stackpulsar.com/blog/reserved-instances-savings-plans/): AWS Compute Savings Plans beat EC2 RIs for GPU inference
- [LLM Cost Monitoring Tools 2026 — Complete Guide](https://stackpulsar.com/blog/llm-cost-monitoring-tools-2026/): Stop guessing where your LLM spend goes
- [Datadog Migration: From $15K to $3K/mo Playbook](https://stackpulsar.com/blog/datadog-migration-playbook/): Three client migrations from Datadog to Grafana + Prometheus + Tempo
- [LLM Context Window Optimization: Cut Costs, Keep Quality](https://stackpulsar.com/blog/llm-context-window-optimization/): A practical guide to reducing LLM inference costs by 40-70% using semantic truncation, context compression, dynamic sizing, and hybrid retrieval — with code…
- [Kubernetes Cost Optimization: Cutting Cloud Bills in Half](https://stackpulsar.com/blog/kubernetes-cost-optimization/): Practical strategies to cut Kubernetes spend by 40-60%: right-sizing nodes, Spot instance mixing, cluster autoscaling, namespace quotas, storage tiering…
- [Cloud FinOps in 2026: From Chaos to Controlled Spend](https://stackpulsar.com/blog/cloud-finops-guide/): A practical guide to cloud waste reduction without sacrificing performance — covering tagging strategies, reserved capacity, and cost-aware architecture
- [LLM FinOps 2026: Cut Your AI Bill, Keep Performance](https://stackpulsar.com/blog/llm-finops-strategies/): A practical guide to reducing LLM inference costs by 60-80% using tiered model routing, semantic caching, prompt optimization, and self-hosting — without…
- [AWS FinOps Agent 2026: The First Frontier Agent for FinOps](https://stackpulsar.com/blog/aws-finops-agent-2026/): AWS FinOps Agent investigates cost anomalies in plain English inside Slack and Jira

## Kubernetes

- [Catch GPU Throttling at 83°C: Prometheus + Grafana + eBPF](https://stackpulsar.com/blog/kubernetes-monitoring-stack/): Above 83°C, vLLM pods throttle silently until OOMKilled

## Tooling

- [DevOps Supply Chain Security 2026: CPU-Z Compromise Lessons](https://stackpulsar.com/blog/supply-chain-security-devops-2026/): The April 2026 CPU-Z/HWMonitor supply chain attack exposed how even trusted developer tools can become attack vectors
- [Datadog Alternatives 2026: 5 Cost-Effective Picks](https://stackpulsar.com/blog/datadog-alternatives-2026/): Five Datadog alternatives that deliver better value at scale — covering LLM inference, Kubernetes, and cloud cost observability for 2026

## Other

- [AI Agent Security Posture 2026: A Practitioner's Framework](https://stackpulsar.com/blog/ai-agent-security-posture-2026/): Agent security posture is not a CSPM problem

## Crawl guidance

- Articles update with corrections and new data — `lastmod` in the sitemap reflects revisions
- All article URLs use a trailing slash; non-trailing variants 301 to the canonical
- Internal links use canonical URLs; redirect stubs are excluded from the sitemap
- Cloudflare Pages serves the site; `/cdn-cgi/*` paths are platform infrastructure and not site content