prof. Ing. RNDr. Martin Holeňa, CSc.

Proceedings paper

10.14428/esann/2022.ES2022-113

Department of Applied Mathematics

The paper presents a novel application of artificial neural networks (ANNs) in the context of surrogate models for black-box optimization, i.e. optimization of objective functions that are accessed through empirical evaluation. For active learning of surrogate models, a very important role plays learning of multidimensional normal distributions, for which Gaussian processes (GPs) have been traditionally used. On the other hand, the research reported in this paper evaluated the applicability of two ANN-based methods to this end: combining GPs with ANNs and learning normal distributions with evidential ANNs. After methods sketch, the paper brings their comparison on a large collection of data from surrogate-assisted black-box optimization. It shows that combining GPs using linear covariance functions with ANNs yields lower errors than the investigated methods of evidential learning.

Proceedings paper

Department of Applied Mathematics

This paper provides an insight into the possibility of how to find ontologies most relevant to scientific texts using artificial neural networks. The basic idea of the presented approach is to select a representative paragraph from a source text file, embed it to a vector space by a pre-trained fine-tuned transformer, and classify the embedded vector according to its relevance to a target ontology. We have considered different classifiers to categorize the output from the transformer, in particular random forest, support vector machine, multilayer perceptron, k-nearest neighbors, and Gaussian process classifiers. Their suitability has been evaluated in a use case with ontologies and scientific texts concerning catalysis research. From results we can say the worst results have random forest. The best results in this task brought support vector machine classifier.

Invited/Awarded proceedings paper

10.1145/3477911.3477922

Department of Applied Mathematics

In the last decade, we have seen a significant uprise of deep neural networks in image processing tasks and many other research areas. However, while various neural architectures have successfully solved numerous tasks, they constantly demand more and more processing time and training data. Moreover, the current trend of using existing pre-trained architectures just as backbones and attaching new processing branches on top not only increases this demand but diminishes the explainability of the whole model. Our research focuses on combinations of explainable building blocks for the image processing tasks, such as object tracking. We propose a combination of Mask R-CNN, state-of-the-art object detection and segmentation neural network, with our previously published method of sparse feature tracking. Such a combination allows us to track objects by connecting detected masks using the proposed sparse feature tracklets. However, this method cannot recover from complete object occlusions and has to be assisted by an object re-identification. To this end, this paper uses our feature tracking method for a slightly different task: an unsupervised extraction of object representations that we can directly use to fine-tune an object re-identification algorithm. As we have to use objects masks already in the object tracking, our approach utilises the additional information as an alpha channel of the object representations, which further increases the precision of the re-identification. An additional benefit is that our fine-tuning method can be employed even in a fully online scenario.

Proceedings paper

Department of Applied Mathematics

Recently, the amount of generated time series data has been increasing rapidly in many areas such as healthcare, security, meteorology and others. However, it is very rare that those time series are annotated. For this reason, unsupervised machine learning techniques such as anomaly detection are often used with such data. There exist many unsupervised algorithms for anomaly detection ranging from simple statistical techniques such as moving average or ARIMA till complex deep learning algorithms such as LSTM-autoencoder. For a nice overview of the recent algorithms we refer to read. Difficulties with the unsupervised approach are: defining an anomaly score to correctly represent how anomalous is the time series, and setting a threshold for that score to distinguish between normal and anomaly data. Supervised anomaly detection, on the other hand, needs an expensive involvement of a human expert. An additional problem with supervised anomaly detection is usually the occurrence of very low ratio of anomalies, yielding highly imbalanced data. In this extended abstract, we propose an active learning extension for an anomaly detector based on a LSTM-autoencoder. It performs active learning using various classification algorithms and addresses data imbalance with oversampling and under-sampling techniques. We are currently testing it on the ECG5000 dataset from the UCR time series classification archive.

Book

10.1007/978-3-030-36962-0

Department of Applied Mathematics

This book explores internet applications in which a crucial role is played by classification, such as spam filtering, recommender systems, malware detection, intrusion detection and sentiment analysis. It explains how such classification problems can be solved using various statistical and machine learning methods, including K nearest neighbours, Bayesian classifiers, the logit method, discriminant analysis, several kinds of artificial neural networks, support vector machines, classification trees and other kinds of rule-based methods, as well as random forests and other kinds of classifier ensembles. The book covers a wide range of available classification methods and their variants, not only those that have already been used in the considered kinds of applications, but also those that have the potential to be used in them in the future. The book is a valuable resource for post-graduate students and professionals alike.

Proceedings paper

Department of Applied Mathematics

Semi-supervised learning is characterized by using the additional information from the unlabeled data. In this paper, we compare two semi-supervised algorithms for deep neural networks on a large real-world malware dataset. Specifically, we evaluate the performance of a rather straightforward method called Pseudo-labeling, which uses unlabeled samples, classified with high confidence, as if they were the actual labels. The second approach is based on an idea to increase the consistency of the network’s prediction under altered circumstances. We implemented such an algorithm called Π-model, which compares outputs with different data augmentation and different dropout setting. As a baseline, we also provide results of the same deep network, trained in the fully supervised mode using only the labeled data. We analyze the prediction accuracy of the algorithms in relation to the size of the labeled part of the training dataset.

Proceedings paper

Department of Applied Mathematics

Classification serves an important role in domains such as network security or health care. Although these domains require understanding of the classifier’s decision, there are only a few classification methods trying to justify or explain their results. Classification rules and decision trees are generally considered comprehensible. Therefore, this study compares the classification performance and comprehensibility of a random forest classifier with classification rules extracted by Frequent Item Set Mining, Logical Item Set Mining and by the Explainer algorithm, which was previously proposed by the authors.

Proceedings paper

10.1109/SITIS.2018.00075

Department of Applied Mathematics

Fundamental approaches in motion tracking are based on registration of pixel patches from one frame to another. To ensure invariance to some changes in the image and improve the speed of discovering a match, a pyramidal approach is used to steer the process faster to optima. However, registration of the patches in high resolution is still computationally expensive. Because we require the algorithm to process Ultra HD video content in real time on commonly available hardware, especially on mid-tier graphics processing units, approaches using matching of pixel patches are not feasible. In this paper, we present and evaluate an approach inspired by motion tracking on an image pyramid. However, instead of comparing pixel patches one to another, we utilise binary image descriptors that are much shorter and inherently use a Hamming distance for their direct comparison. Evaluation of our implementation, which is available on GitHub, was carried out on the Multiple Object Tracking challenge dataset.

Proceedings paper

Department of Applied Mathematics

One of the key preprocessing tasks in informa- tion retrieveal from video is the segmentation of the scene, primarily its segmentation into foreground objects and the background. This is actually a classification task, but with the specific property that it is very time consuming and costly to obtain human-labelled training data for classifier training. That suggests to use semisupervised classifiers to this end. The presented work in progress reports the inves- tigation of semisupervised classification methods based on cluster regularization and on fuzzy c-means in connection with the foreground / background segmentation task. To classify as many video frames as possible using only a single human-based frame, the semisupervised classifica- tion is combined with a frequently used keypoint detec- tor based on a combination of a corner detection method with a visual descriptor method. The paper experimentally compares both methods, and for the first of them, also clas- sifiers with different delays between the human-labelled video frame and classifier training.

Proceedings paper

Department of Applied Mathematics

The recognition of emotional states in speech is starting to play an increasingly important role. However, it is a complicated process, which heavily relies on the extraction and selection of utterance features related to the emotional state of the speaker. In the reported research, MPEG-7 low level audio descriptors[10] serve as features for the recognition of emotional categories. To this end, a methodology combining MPEG-7 with several important kinds of classifiers is elaborated.

Proceedings paper

Department of Applied Mathematics

This paper studies reverse Turing tests to distinguish humans and computers, called CAPTCHA. Contrary to classical Turing tests, in this case the judge is not a human but a computer. The main purpose of such tests is securing user logins against the dictionary or brute force password guessing, avoiding automated usage of various services, preventing bots from spamming on forums and many others. Typical approaches to solving text-based CAPTCHA automatically are based on a scheme specific pipeline containing hand-designed pre-processing, denoising, segmentation, post processing and optical character recognition. Only the last part, optical character recognition, is usually based on some machine learning algorithm. We present an approach using neural networks and a simple clustering algorithm that consists of only two steps, character localisation and recognition. We tested our approach on 11 different schemes selected to present very diverse security features. We experimentally show that using convolutional neural networks is superior to multi-layered perceptrons.

Article

Department of Theoretical Computer Science

In application domains where there exists a large amount of unlabelled data but obtaining labels is expensive, active learning is a useful way to select which data should be labelled. In addition to its traditional successful use in classification and regression tasks, active learning has been also applied to sequence labelling. According to the standard active learning approach, sequences for which the labelling would be the most informative should be labelled. However, labelling the entire sequence may be inefficient as for some its parts, the labels can be predicted using a model. Labelling such parts brings only a little new information. Therefore in this paper, we investigate a sequence labelling approach in which in the sequence selected for labelling, the labels of most tokens are predicted by a model and only tokens that the model can not predict with sufficient confidence are labelled. Those tokens are identified using the k-best Viterbi algorithm.

Proceedings paper

10.15439/2017F417

Department of Theoretical Computer Science

Currently used motion estimation is usually based on a computation of optical flow from individual images or short sequences. As these methods do not require an extraction of the visual description in points of interest, correspondence can be deduced only by the position of such points. In this paper, we propose an alternative motion estimation method solely using a binary visual descriptor. By tuning the internal parameters, we achieve either a detection of longer time series or a higher number of shorter series in a shorter time. As our method uses the visual descriptors, their values can be directly used in more complex visual detection tasks.

Proceedings paper

Department of Theoretical Computer Science

Databases of video content traditionally rely on annotations and meta-data imported by a person, usually the uploader. This is supposedly due to a lack of an universal approach to the automated multimedia content annotation. As it may be hard or impossible to find a single classifier for all encountered combinations of different modalities or even a network of the classifiers, current interest of our research is to use meta-learning for multiple stages of the multimedia content classification. With this, we hope to handle correctly all modalities involved including their overlaps. Successively, the extracted classes will be used to build the index and later used for searching and discovery in the multimedia.

Proceedings paper

Department of Theoretical Computer Science

This paper studies reverse Turing tests to tell humans and computers apart. Contrary to classical Turing tests, the judge is not a human but a computer. These tests are often called Completely Automated Public Turing tests to tell Computers and Humans Apart (CAPTCHA). The main purpose of such test is avoiding automated usage of various services, preventing bots from spamming on forums, securing user logins against dictionary or brute-force password guessing and many others. During years, a diversity of tests appeared. In this paper, we focused on the two most classical and widespread schemes, which are text-based and audio-based CAPTCHA, and on their use in the Czech internet environment. The goal of this paper is to point out flaws and weak spots of often used solutions and consequent security risks. To this end, we pipelined several relatively easy algorithms like flood fill algorithm and k-nearest neighbours, to overcome CAPTCHA challenges at several web pages, including state administration.

Proceedings paper

Department of Theoretical Computer Science

Open narrative approach enables the creators of multimedia content to create multi-stranded, navigable narrative environments. The viewer is able to navigate such space depending on author’s predetermined constraints, or even browse the open narrative structure arbitrarily based on their interests. This philosophy is used with great advantage in the collaborative open narrative system NARRA. The platform creates a possibility for documentary makers, journalists, activists or other artists to link their own audiovisual material to clips of other authors and finally create a navigable space of individual multimedia pieces. To help authors focus on building the narratives themselves, a set of automated tools have been proposed. Most obvious ones, as speech-to-text, are already incorporated in the system. However other, more complicated authoring tools, primarily focused on creating metadata for the media objects, are yet to be developed. Most complex of them involve an object description in media (with unrestricted motion, action or other features) and detection of near-duplicates of video content, which is the focus of our current interest. In our approach, we are trying to use motion-based features and register them across the whole clip. Using GridCut algorithm to segment the image, we then try to select only parts of the motion picture, that are of our interest for further processing. For the selection of suitable description methods, we are developing a meta-learning approach. This will supposedly enable automatic annotation based not only on clip similarity per se, but rather on detected objects present in the shot.

Article

Department of Theoretical Computer Science

The objectives of this article are to model behavior of individual animals and to cluster the resulting models in order to group animals with similar behavior patterns. Hidden Markov models are considered suitable for clustering purposes. Their clustering is well studied, however, only if the observable variables can be assumed to be Gaussian mixtures, which is not valid in our case. Therefore, we use the Kullback-Leibler divergence to cluster hidden Markov models with observable variables that have an arbitrary distribution. Hierarchical and spectral clustering is applied. To evaluate the modeling approach, an experiment was performed and an accuracy of 83.86% was reached in predicting behavioral sequences of individual animals. Results of clustering were evaluated by means of statistical descriptors of the animals and by a domain expert, both methods confirm that the results of clustering are meaningful.

Proceedings paper

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science

Discovering anomalies within data is nowadays very important, because it helps to uncover interesting events. Consequently, a considerable amount of anomaly detection algorithms was proposed in the last few years. Only a few papers about anomaly detection at least mentioned why some samples were labelled as anomalous. Therefore, we proposed a method allowing to extract rules explaining the anomaly from an ensemble of specifically trained decision trees, called sapling random forest. Our method is able to interpret the output of an arbitrary anomaly detector. The explanation is given as conjunctions of atomic conditions, which can be viewed as antecedents of association rules. In this work we focus on selection, post processing and evaluation of those rules. The main goal is to present a small number of the most important rules. To achieve this, we use quality measures such as lift and confidence boost. The resulting sets of rules are experimentally and empirically evaluated on two artificial datasets and one real-world dataset.

Proceedings paper

Department of Theoretical Computer Science

Proceedings paper

Department of Applied Mathematics

The main objective of outlier detection is find- ing samples considerably deviating from the majority. Such outliers, often referred to as anomalies, are nowadays more and more important, because they help to uncover in- teresting events within data. Consequently, a considerable amount of statistical and data mining techniques to iden- tify anomalies was proposed in the last few years, but only a few works at least mentioned why some sample was la- belled as an anomaly. Therefore, we propose a method based on specifically trained decision trees, called sapling random forest. Our method is able to interpret the output of arbitrary anomaly detector. The explanation is given as a subset of features, in which the sample is most deviating, or as con- junctions of atomic conditions, which can be viewed as antecedents of logical rules easily understandable by hu- mans. To simplify the investigation of suspicious samples even more, we propose two methods of clustering anoma- lies into groups. Such clusters can be investigated at once saving time and human efforts. The feasibility of our ap- proach is demonstrated on several synthetic and one real world datasets.

Proceedings paper

Department of Theoretical Computer Science

The main objective of outlier detection is find- ing samples considerably deviating from the majority. Such outliers, often referred to as anomalies, are nowadays more and more important, because they help to uncover in- teresting events within data. Consequently, a considerable amount of statistical and data mining techniques to iden- tify anomalies was proposed in the last few years, but only a few works at least mentioned why some sample was la- belled as an anomaly. Therefore, we propose a method based on specifically trained decision trees, called sapling random forest. Our method is able to interpret the output of arbitrary anomaly detector. The explanation is given as a subset of features, in which the sample is most deviating, or as con- junctions of atomic conditions, which can be viewed as antecedents of logical rules easily understandable by hu- mans. To simplify the investigation of suspicious samples even more, we propose two methods of clustering anoma- lies into groups. Such clusters can be investigated at once saving time and human efforts. The feasibility of our ap- proach is demonstrated on several synthetic and one real world datasets.

Proceedings paper

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science

Article

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science

Proceedings paper

10.1109/ISDA.2010.5687260

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science

Book chapter

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science

Proceedings paper

Department of Theoretical Computer Science