Ing. Jan Koza

Stať ve sborníku

Katedra aplikované matematiky

Active learning plays a crucial role in black-box optimization, especially for objective functions that are expensive to evaluate. Continuous black-box optimization has adopted an approach called surrogate modelling, where the original black-box objective is approximated with a regression model. An active learning task in this context is to decide which points should be evaluated using the original objective to update the surrogate model. Apart from low-order polynomials, the first surrogate models were artificial neural networks of the kinds multilayer perceptron and radial basis function network. In the late 2000s, neural networks have been superseded by other kinds of surrogate models, primarily Gaussian processes. However, over the last 15 years, neural networks have seen significant and successful development, suggesting that they once again have the potential to serve as promising surrogate models. This paper reviews possible research directions concerning that potential, and recalls initial results from investigations in some of these directions. Finally, it contributes to those results by investigating the state-of-the-art black-box optimizer CMA-ES surrogate-assisted by two variants of random-activation-function neural network ensembles.

Stať ve sborníku

Katedra aplikované matematiky

This paper explores the use of Gaussian processes (GPs) in the covariance matrix adaptation evolution strategy (CMA-ES) for black-box optimization. GPs are powerful probabilistic models that capture complex relationships, making them suitable for modeling uncertain objective functions. Integrating GPs into the CMA-ES improves exploration and adaptation in the search space, enhancing convergence speed and solution quality. The paper describes a novel implementation framework allowing to use GPs as surrogate models for the CMA-ES. That framework findings encourage further research to advance the application of GPs in black-box optimization.

Stať ve sborníku

10.14428/esann/2022.ES2022-113

Katedra aplikované matematiky

The paper presents a novel application of artificial neural networks (ANNs) in the context of surrogate models for black-box optimization, i.e. optimization of objective functions that are accessed through empirical evaluation. For active learning of surrogate models, a very important role plays learning of multidimensional normal distributions, for which Gaussian processes (GPs) have been traditionally used. On the other hand, the research reported in this paper evaluated the applicability of two ANN-based methods to this end: combining GPs with ANNs and learning normal distributions with evidential ANNs. After methods sketch, the paper brings their comparison on a large collection of data from surrogate-assisted black-box optimization. It shows that combining GPs using linear covariance functions with ANNs yields lower errors than the investigated methods of evidential learning.

Stať ve sborníku

10.1145/3449639.3459358

Katedra aplikované matematiky

Surrogate regression models have been shown as a valuable technique in evolutionary optimization to save evaluations of expensive black-box objective functions. Each surrogate modelling method has two complementary components: the employed model and the control of when to evaluate the model and when the true objective function, aka evolution control. They are often tightly interconnected, which causes difficulties in understanding the impact of each component on the algorithm performance. To contribute to such understanding, we analyse what constitutes the evolution control of three surrogate-assisted versions of the state-of-the-art algorithm for continuous black-box optimization --- the Covariance Matrix Adaptation Evolution Strategy. We implement and empirically compare all possible combinations of the regression models employed in those methods with the three evolution controls encountered in them. An experimental investigation of all those combinations allowed us to asses the influence of the models and their evolution control separately. The experiments are performed on the noiseless and noisy benchmarks of the Comparing-Continuous-Optimisers platform and a real-world simulation benchmark, all in the expensive scenario, where only a small budget of evaluations is available.

Stať ve sborníku

Katedra aplikované matematiky

Semi-supervised learning is characterized by using the additional information from the unlabeled data. In this paper, we compare two semi-supervised algorithms for deep neural networks on a large real-world malware dataset. Specifically, we evaluate the performance of a rather straightforward method called Pseudo-labeling, which uses unlabeled samples, classified with high confidence, as if they were the actual labels. The second approach is based on an idea to increase the consistency of the network’s prediction under altered circumstances. We implemented such an algorithm called Π-model, which compares outputs with different data augmentation and different dropout setting. As a baseline, we also provide results of the same deep network, trained in the fully supervised mode using only the labeled data. We analyze the prediction accuracy of the algorithms in relation to the size of the labeled part of the training dataset.

Stať ve sborníku

Katedra teoretické informatiky

The current archives of large-scale spectroscopic surveys, such as SDSS or LAMOST, contain millions of spectra. As some interesting objects (e.g. emission line stars or quasars) can be identified only by checking the shapes of certain spectral lines, machine learning techniques have to be applied, complemented by flexible visualisation of results.