arXiv cs.LGPaper
Efficiently Estimating Optimal Hyperparameter Scaling Laws through Power-Law Entropy Search
The efficiency gain is real if you're training LLMs at scale and need to predict optimal configs at new model sizes. But this is a narrow tool for a narrow audience: if you're not doing frontier model training, it doesn't change your day. Even among LLM labs, the question is whether the sampling budget PLES saves outweighs the latency cost of Bayesian optimization.