Mgr. Alexander Kovalenko, Ph.D.

Theses

Bachelor theses

Machine Learning Techniques for Source Code Pattern Recognition

Author
Rudolf Raevskiy
Year
2022
Type
Bachelor thesis
Supervisor
Mgr. Alexander Kovalenko, Ph.D.
Reviewers
Pierre Donat-Bouillud, Ph.D.
Summary
The automated understanding of code semantics is crucial for helping developers write reliable and optimized code. In recent years, there has been a growing interest in applying machine learning to source code, with the aim of automatically discovering bugs, commenting or understanding and improving the code. This work reports deep learning techniques applied to various levels of abstraction of the source code. We experiment with a dataset consisting of R language source code. R language has a large community of mostly statisticians. However, R libraries are prone to have suboptimal code. The main contribution of this work is a model trained on a large R dataset, which is the first step toward an automated tool to write a better R code. We primarily focus on Abstract Syntax Trees (AST), considering other representations forms just as well. Different abstractions add a structure to the input and therefore help to better generalize across the dataset. We train and evaluate several models based on various code representations. The transformer-based architecture was chosen as a backbone model for the current task, as it outperforms its counterparts in this domain. Training a model on a large R dataset is the first step toward automatized tool to write a better R code. As a result of RASTaBERTa, which is, to the best of our knowledge, the state-of-the-art transformer-based model for the R language and can be used for further training for specific tasks such as classification, bugs, and anomalies detection, bug-fix, etc.

Self-supervised model for efficient sound recognition trained on aggregated data

Author
Vojtěch Houska
Year
2021
Type
Bachelor thesis
Supervisor
Mgr. Alexander Kovalenko, Ph.D.
Reviewers
doc. Ing. Pavel Kordík, Ph.D.
Summary
The thesis summarizes state-of-the-art approaches in deep learning. It discusses application of self-supervised autoencoders and pre-processing techniques used in sound recognition. YouTube platform served as a source of weakly-labeled data to train such models. Latent space properties of proposed autoencoders were compared and tested using K-means clustering. Implementation of Adversarially Constrained Autoencoder Interpolation failed to outperform randomly initialized autoencoder. The reasons are further discussed and several recommendations for future research are proposed.

Master theses

Dictated numbers recognition model for an interactive voice response (IVR) company

Author
Martin Nykodem
Year
2022
Type
Master thesis
Supervisor
Mgr. Alexander Kovalenko, Ph.D.
Reviewers
doc. Ing. Pavel Kordík, Ph.D.
Summary
This thesis focuses on the problem of automatic speech recognition (ASR). Namely, the specific task of this work is to create a machine learning model to recognize numbers in the Czech language, dictated in a phone call. ASR systems face specific domain-related problems of speech recognition. Therefore, to meet certain requirements peculiar to the Czech language, a custom approach for the preprocessing and model development has to be applied. Based on the survey of the popular state-of-the-art and trending approaches in the ASR field, the model applicable for the above-mentioned task is developed. Specificity of the domain, including data preprocessing and model fine-tuning, is discussed. Additionally, a specific domain dataset extension using the available Czech language datasets is presented. Finally, the development progress and improvements discovered during the development process are described. The results show that an 10-fold improvement in the correct recognition of recordings containing a sequence of dictated numbers is attained. The model vastly outperforms the current best solution from Czech speech recognition companies, as well as solutions from Google and Microsoft. Additionally, the lowest WER score of the available non-commercial models for the domain-agnostic dataset for the Czech language Common Voice 8 is achieved.

Machine Learning for Wafer Bin Map Defect Pattern Classification

Author
Jan Šefčík
Year
2022
Type
Master thesis
Supervisor
Mgr. Alexander Kovalenko, Ph.D.
Reviewers
Mgr. Petr Šimánek
Summary
Automatic classification of defect patterns in wafer bin maps is a challenging problem for semiconductor manufacturers. Recently, progress with supervised approaches has been made, but labeled datasets are usually small and of poor quality. The creation of high-quality datasets is expensive and time-consuming, limiting early production. This work analyzes a selfsupervised/semi-supervised learning approaches that use unlabeled data. Based on the resizing problem analysis, this thesis proposed a smaller model that focuses on improving defect classification performance with diverse-sized wafers. The substantial improvement was made with the minor classes, in particular, with Scratch class.

Anomaly detection on the CERN data centre monitoring data

Author
Antonín Dvořák
Year
2022
Type
Master thesis
Supervisor
Mgr. Alexander Kovalenko, Ph.D.
Reviewers
doc. Ing. Kamil Dedecius, Ph.D.
Summary
One of the many tasks of CERN cloud service operators is to make sure that the desired computational power is delivered to all users of the scientific community. This task is accomplished by carefully setting threshold-based alarming on top of the infrastructure performance time series metrics. In order to maximize the efficiency of the cloud infrastructure and to reduce the monitoring effort for service operators, we have developed a fully automated Anomaly Detection System that leverages unsupervised machine learning methods for time series metrics. Moreover, adopting ensemble methods, we combine traditional (Isolation forest) and deep learning (Gated recurrent unit/Long short-term memory Autoencoders) approaches. This work presents a description of the CERN monitoring infrastructure, problem formulation, design of the Anomaly Detection Pipeline, description of used models, creation of the dataset and performance of the implemented models compared to the performance of the Current Alarming System.

Advancing Microrobotics for Biomedical Applications through Machine Learning

Author
Daniil Pastukhov
Year
2024
Type
Master thesis
Supervisor
Mgr. Alexander Kovalenko, Ph.D.
Reviewers
doc. Ing. Kamil Dedecius, Ph.D.
Summary
This thesis explores the integration of machine learning techniques in microrobotics, focusing on biological microrobots utilizing sperm cells as a platform. The investigation includes a detailed analysis of relevant works in microrobotics and machine learning in the biomedical context, laying the groundwork for a multifaceted exploration. Key contributions include curating and annotating datasets tailored for training and evaluating models. Object detection models were developed and considered for precisely identifying sperm cells and their heads, while a keypoint estimation model was employed to detect flagellum keypoints. Additionally, an object-tracking system was implemented and evaluated to track the dynamic movements of sperm cell heads, enhancing the understanding of their interactions in dynamic environments. Further, a trajectory prediction model was trained and evaluated. This study marks a notable advancement in the integration of machine learning and microrobotics, offering innovative perspectives and approaches that can be utilized in various biomedical and technological fields. The work contributes to the current understanding of biological microrobots and lays the foundation for future advancements, unlocking the potential for precise control mechanisms and expanding applications in various fields.

The Study of Linear Self-Attention Mechanism in Transformer

Author
Uladzislau Yorsh
Year
2022
Type
Master thesis
Supervisor
Mgr. Alexander Kovalenko, Ph.D.
Reviewers
doc. Ing. Pavel Kordík, Ph.D.
Summary
As the quadratic complexity of an attention mechanism in the Transformer architecture places a high demand on processing long sequences, the goal of this research is to explore possibilities of linear attention in Transformer-like architecture and implement new methods.

Machine Learning Techniques for Laser-Plasma Acceleration Optimization

Author
Matěj Jech
Year
2024
Type
Master thesis
Supervisor
Mgr. Alexander Kovalenko, Ph.D.
Reviewers
doc. Ing. Ivan Šimeček, Ph.D.
Summary
The thesis deals with the analysis of data from the laser-plasma particle accelerator in collaboration with the scientific institution ELI Beamlines. In the scope of the work, a data pre-processing process was designed and a generative model simulating the course of physics experiments was developed. The model is conditioned on a vector of experimental parameters and generates image data showing the energy spectrum of the accelerated electron beam. The developed model can be used as a partial substitute for real experiments, which are costly in terms of time and finances. It can also be used as a simulation of real experiments for various optimization methods. This thesis defines the process of training and testing candidate models with three different architectures and based on four hyperparameters. The resulting model can generate data at a rate of 1.8 images per second and has been evaluated based on a number of metrics, including the expert opinion of scientists, as a trustworthy tool to simulate the electron acceleration process.

Developing an automatic speech recognition system based on Czech spoken language

Author
Richard Werner
Year
2020
Type
Master thesis
Supervisor
Mgr. Alexander Kovalenko, Ph.D.
Reviewers
Ing. Mgr. Ladislava Smítková Janků, Ph.D.
Summary
This thesis deals with automatic speech recognition (ASR) using recurrent neural networks (RNN). The goal is to analyze the state-of-the-art in those fields and propose a suitable Czech open-source voice dataset and an RNN model. Next, train the model on the dataset and use to trained model to transcribe another appropriate source of speech data. The output is a trained speech-to-text model, a new open-source dataset, and a system allowing accessible data preprocessing and further extension of datasets. The dataset of choice is the Czech Parliament meetings (CPM) transcribed recordings, and the model used is the DeepSpeech open-source project. The secondary source of speech data is the rest of the recording gathered from the CPM website. Part of the preprocessing relied on the usage of a voice activity detection (VAD) model, which was used as a reference for the audio segmentation. The trained model achieved 12.66 % WER (Word Error Rate) and 4.63 % CER (Character Error Rate), which were sufficient values for the final dataset transcription. After preprocessing, the final dataset consisted of over 580000 speech utterances of ranging length roughly from 1 up to 70 seconds. The project is designed as a Docker image with prepared custom tools and other means to preprocess datasets and feed them to an RNN. Therefore, the output is a trained RNN model, an open-source dataset consisting of labeled recordings, and a ready-to-use Docker image with a toolkit for data preprocessing.