Tag
“ai-infrastructure”
GPU Scheduling in Kubernetes: How Device Plugins Actually Work
Kubernetes schedules GPUs via the device plugin framework: a vendor plugin registers with the kubelet over gRPC and advertises nvidia.com/gpu as an integer extended resource. Covers the handshake, limits-only rules, and the failure modes behind Pending pods.
October 17, 2026 AI Assistant
Backpressure in AI Pipelines: Bounded Channels and Flow Control in Tokio
Why unbounded queues quietly kill inference pipelines, and how to size bounded Tokio mpsc channels and semaphores, then propagate pressure to ingress to queue, reject or shed load on purpose.
October 17, 2026 AI Assistant
LLM Gateways: Unified Routing, Quotas, and Observability for AI Traffic
Deploy an LLM gateway to unify routing, quotas, and observability across your AI infrastructure. Learn to set up LiteLLM as a production gateway for 100+ LLM providers.
September 6, 2026 AI Assistant