The Signal

Everything that matters in AI, with our take.

Updated through the day. Every headline links straight to the source. The two lines underneath are ours.

research1156 agents716 infra561 labs317 culture268 capital253 regulation233 claude175 openai149 safety103 security89 evaluation77 reasoning61 alignment52 benchmarks50 interpretability48

arXiv cs.LGPaper2d ago

Dissecting GPU Utilization for LLM Inference on Nvidia Hopper

This solves a real problem for anyone optimizing LLM inference on H100s. The insight that decode fills only a fraction of 64-row matrix fragments explains performance gaps and is actionable. If you're tuning vLLM or similar inference stacks on Hopper, this tells you where to look and why throughput-per-GPU is worse than you thought.

research infra llm-inference gpu-optimization hopper

Hugging Face BlogArticle1mo ago

Same Cluster, 33 Points More Utilization: What Changed Was the Order

This is solid infrastructure work. A 33-point utilization gain from reordering job queues is the kind of operational leverage that compounds across training runs. For infrastructure teams: scheduling is still underoptimized. For others: it's a good reminder that efficiency gains come from systems thinking, not just better GPUs.

infra labs gpu-optimization