Kilian Q. Weinberger

Associate Professor of Computer Science

Automatic Hyper-parameter Tuning


Download Matlab CODE

Hyper parameter optimization is one of the biggest problems in many settings within and outside of computer science.
The slowest method is to meticulously try out many different hyper parameter settings (also often referred to grid search).
You can typically do better with random search. Often the best approach is to train a small machine learning algorithm to predict which parameter setting is most promising
and then explore the parameter space with the help of these predictions.

Relevant publications:
[PDF][CODE][BIBTEX] Bayesian Optimization with Inequality Constraints.
Jacob Gardner, Matt Kusner, Zhixiang (Eddie) Xu, Kilian Q. Weinberger, John Cunningham
International Conference on Machine Learning (ICML), Beijing China. JMLR W&CP 32 (1) :937-945, 2014.