Home NewsPublicationsTeachingSoftwareAwardsOther

Lars Kotthoff

Lars Kotthoff

Assistant Professor


EN 4071A
Department of Computer Science
University of Wyoming
Dept 3315, 1000 E University Ave
Laramie, WY 82071-2000

My research combines artificial intelligence and machine learning to build robust systems with state-of-the-art performance. I develop techniques to induce models of how algorithms for solving computationally difficult problems behave in practice. Such models allow to select the best algorithm and choose the best parameter configuration for solving a given problem. I lead the Meta-Algorithmics, Learning and Large-scale Empirical Testing (MALLET) lab and have acquired more than $400K in external funding to date.

More broadly, I am interested in innovative ways of modelling and solving challenging problems and applying such approaches to the real world. Part of this is making cutting edge research available to and usable by non-experts. Machine learning often plays a crucial role in this, and I am also working on making machine learning more accessible and easier to use.

Interested in coming to beautiful Wyoming and joining MALLET? There are several funded PhD positions available. Please drop me an email or, if you are already here, come by my office. I also have Master's projects and projects for undergraduates seeking research experience available. There are usually several of them posted on the board opposite of my office.

Follow me on Twitter.



For citation numbers, please see my Google Scholar page.


  • Kotthoff, Lars, Alexandre Fréchette, Tomasz P. Michalak, Talal Rahwan, Holger H. Hoos, and Kevin Leyton-Brown. “Quantifying Algorithmic Improvements over Time.” In 27th International Joint Conference on Artificial Intelligence (IJCAI) Special Track on the Evolution of the Contours of AI, 2018. preprint PDF bibTeX abstract

    Assessing the progress made in AI and contribu- tions to the state of the art is of major concern to the community. Recently, Fr´ echette et al. [2016] advocated performing such analysis via the Shapley value, a concept from coalitional game theory. In this paper, we argue that while this general idea is sound, it unfairly penalizes older algorithms that advanced the state of the art when introduced, but were then outperformed by modern counterparts. Driven by this observation, we introduce the tem- poral Shapley value, a measure that addresses this problem while maintaining the desirable properties of the (classical) Shapley value. We use the tempo- ral Shapley value to analyze the progress made in (i) the different versions of the Quicksort algorithm; (ii) the annual SAT competitions 2007–2014; (iii) an annual competition of Constraint Programming, namely the MiniZinc challenge 2014–2016. Our analysis reveals novel insights into the development made in these important areas of research over time.


  • Kotthoff, Lars, Barry Hurley, and Barry O’Sullivan. “The ICON Challenge on Algorithm Selection.” AI Magazine 38, no. 2 (2017): 91–93. preprint PDF bibTeX

  • Kotthoff, Lars, Chris Thornton, Holger H. Hoos, Frank Hutter, and Kevin Leyton-Brown. “Auto-WEKA 2.0: Automatic Model Selection and Hyperparameter Optimization in WEKA.” Journal of Machine Learning Research 18, no. 25 (2017): 1–5. preprint PDF bibTeX abstract

    WEKA is a widely used, open-source machine learning platform. Due to its intuitive interface, it is particularly popular with novice users. However, such users often find it hard to identify the best approach for their particular dataset among the many available. We describe the new version of Auto-WEKA, a system designed to help such users by automatically searching through the joint space of WEKA's learning algorithms and their respective hyperparameter settings to maximize performance, using a state-of-the-art Bayesian optimization method. Our new package is tightly integrated with WEKA, making it just as accessible to end users as any other learning algorithm.
  • Lindauer, Marius, Jan N. van Rijn, and Lars Kotthoff, eds. Proceedings of the Open Algorithm Selection Challenge. Vol. 79. Proceedings of Machine Learning Research. PMLR, 2017. bibTeX

  • ———. “Open Algorithm Selection Challenge 2017: Setup and Scenarios.” In Proceedings of the Open Algorithm Selection Challenge, edited by Marius Lindauer, Jan N. van Rijn, and Lars Kotthoff, 79:1–7. Proceedings of Machine Learning Research. Brussels, Belgium: PMLR, 2017. preprint PDF bibTeX abstract

    The 2017 algorithm selection challenge provided a snapshot of the state of the art in algorithm selection and garnered submissions from four teams. In this chapter, we describe the setup of the challenge and the algorithm scenarios that were used.
See all


  • Maintainer of the FSelector R package.

  • Author and maintainer of LLAMA, an R package to simplify common algorithm selection tasks such as training a classifier as portfolio selector.

  • Core contributor to the mlr R package (Github) for all things machine learning in R.

  • Leading the Auto-WEKA project, which brings automated machine learning to WEKA.


  • I am teaching COSC 3020 (Algorithms and Data Structures) this semester. Lecture materials, assignments, announcements, etc. are available on WyoCourses.
  • I am teaching a practical machine learning course using mlr. The slides are available here.
  • If you are interested in the AI reading group, check out the list of proposed papers here.



Apart from my main affiliation, I am a research associate with the Maya Research Program. If I'm not in the office, it's possible that you can find me in the jungle of Belize excavating and/or mapping Maya ruins. Check out the interactive map.

I am also involved with the OpenML project project and a core contributor to ASlib, the benchmark library for algorithm selection.

While you're here, have a look at my overview of the Algorithm Selection literature. For something more visual, have a look at my pictures on Flickr.