Bachelor theses
Meta-learning for automatic model selection and hyperparameter optimization
Author
Ivan Rychtera
Year
2020
Type
Bachelor thesis
Supervisor
Ing. Markéta Jůzlová
Reviewers
Ing. Karel Klouda, Ph.D.
Department
Summary
Model selection and hyperparameter optimisation is an important step in creating well-performing machine learning models. Optimizing hyperparameters for a given task can be enhanced with information from previous tasks - meta-learning. This thesis looks into combining meta-learning with algorithms for hyperparameter optimisation and model selection. We look at several state-of-the-art hyperparameter optimisation and meta-learning methods. We assess the change in performance for 3 of the reviewed hyperparameter optimisation algorithms when incorporating meta-learning. The results does not prove any significant improvement of meta-learning compared to vanilla model selection methods. The subsequent analysis implies that this is due to the capability of hyperparameter optimisation and model selection methods to find well-performing configurations on their own in the given time frame.
Model-agnostic methods for explaining local predictions of a black-box classifier
Author
Adam Skluzáček
Year
2020
Type
Bachelor thesis
Supervisor
Ing. Markéta Jůzlová
Reviewers
Ing. Kamil Dedecius, Ph.D.
Department
Summary
Local model-agnostic explanation methods aim to explain a single prediction of an arbitrary machine learning model by studying the model only through its inputs and corresponding outputs. Explaining predictions of a complex machine learning model helps practitioner to debug the model and build user's trust in the predictions.
This thesis reviews and describes three of the state-of-the-art local model-agnostic explanation methods -- LIME, Anchors and SHAP. The described methods are evaluated in terms of faithfulness of their explanaions to the model being explained. Evaluation is performed on various classifiers trained on artificially generated datasets as well as a real-world divorce dataset. The artificial datasets are generated based on known dependencies which allows to calculate optimal explanations and compare them to the explanations produced by the explanation methods. The experiments show that SHAP is the most robust out of the considered explanation methods. LIME and Anchors fail to produce faithful explanations in specific cases, however, they both managed to produce faithful explanations in experiment with real-world dataset.