Abstract

Instance Space Analysis (ISA) is a recently developed methodology to support objective testing of algorithms. Rather than reporting algorithm performance on average across a chosen set of test problems, as is standard practice, ISA offers a more nuanced understanding of the unique strengths and weaknesses of algorithms across different regions of the instance space that may otherwise be hidden on average. It also facilitates objective assessment of any bias in the chosen test instances, and provides guidance about the adequacy of benchmark test suites and the generation of more diverse and comprehensive test instances to span the instance space. This talk provides an overview of the ISA methodology, and the online software tools that are enabling its worldwide adoption in many disciplines. A case study comparing algorithms for university timetabling is presented to illustrate the methodology and tools, with several other applications to optimisation, machine learning, computer vision and quantum computing highlighted.

About the Speaker

Kate Smith-Miles is a Melbourne Laureate Professor of Applied Mathematics in the School of Mathematics and Statistics at The University of Melbourne. She is also Director of a doctoral training centre for Optimisation Technologies, Integrated Methodologies and Applications (OPTIMA, see optima.org.au), and Associate Dean (Enterprise & Innovation) for the Faculty of Science. Kate obtained a B.Sc(Hons) in Mathematics and a Ph.D. in Electrical Engineering, both from The University of Melbourne. She has published 2 books on neural networks and data mining, and over 280 refereed journal and international conference papers in the areas of neural networks, optimisation, data mining, and various applied mathematics topics. She has supervised 30 PhD students to completion, and has been awarded over AUD$20 million in competitive grants, including a prestigious Australian Laureate Fellowship from the Australian Research Council, which enabled her Instance Space Analysis methodology to be advanced and expanded into an online tool (matilda.unimelb.edu.au). Her awards include the Australian Mathematical Society Medal in 2010 for distinguished research; the EO Tuck Medal from ANZIAM in 2017 for outstanding research and distinguished service; and the Ren Potts Medal for outstanding research in the theory and practice of operations research from the Australian Society for Operations Research in 2019. She is frequently invited as keynote speaker at leading international conferences including IFORS, GECCO, and CPAIOR.