Meningioma microstructure assessed by diffusion MRI: An investigation of the source of mean diffusivity and fractional anisotropy by quantitative histology

Authors
Brabec, J.; Friedjungová, M.; Vašata, D.; Englund, E.; Bengzon, J.; Knutsson, L.; Szczepankiewicz, F.; van Westen, D.; Sundgren, P.C.; Nilsson, M.
Year
2023
Published
Neuroimage: Clinical. 2023, 37 ISSN 2213-1582.
Type
Article
Annotation
Mean diffusivity (MD) and fractional anisotropy (FA) from diffusion MRI (dMRI) have been associated with cell density and tissue anisotropy across tumors, but it is unknown whether these associations persist at the microscopic level. To quantify the degree to which cell density and anisotropy, as determined from histology, account for the intra-tumor variability of MD and FA in meningioma tumors. Furthermore, to clarify whether other histological features account for additional intra-tumor variability of dMRI parameters. We performed ex-vivo dMRI at 200 μm isotropic resolution and histological imaging of 16 excised meningioma tumor samples. Diffusion tensor imaging (DTI) was used to map MD and FA, as well as the in-plane FA (FAIP). Histology images were analyzed in terms of cell nuclei density (CD) and structure anisotropy (SA; obtained from structure tensor analysis) and were used separately in a regression analysis to predict MD and FAIP, respectively. A convolutional neural network (CNN) was also trained to predict the dMRI parameters from histology patches. The association between MRI and histology was analyzed in terms of out-of-sample (R2OS) on the intra-tumor level and within-sample R2 across tumors. Regions where the dMRI parameters were poorly predicted from histology were analyzed to identify features apart from CD and SA that could influence MD and FAIP, respectively. Cell density assessed by histology poorly explained intra-tumor variability of MD at the mesoscopic level (200 μm), as median R2OS = 0.04 (interquartile range 0.01–0.26). Structure anisotropy explained more of the variation in FAIP (median R2OS = 0.31, 0.20–0.42). Samples with low R2OS for FAIP exhibited low variations throughout the samples and thus low explainable variability, however, this was not the case for MD. Across tumors, CD and SA were clearly associated with MD (R2 = 0.60) and FAIP (R2 = 0.81), respectively. In 37% of the samples (6 out of 16), cell density did not explain intra-tumor variability of MD when compared to the degree explained by the CNN. Tumor vascularization, psammoma bodies, microcysts, and tissue cohesivity were associated with bias in MD prediction based solely on CD. Our results support that FAIP is high in the presence of elongated and aligned cell structures, but low otherwise. Cell density and structure anisotropy account for variability in MD and FAIP across tumors but cell density does not explain MD variations within the tumor, which means that low or high values of MD locally may not always reflect high or low tumor cell density. Features beyond cell density need to be considered when interpreting MD.

Learning to Optimize with Dynamic Mode Decomposition

Year
2022
Published
2022 International Joint Conference on Neural Networks (IJCNN). Vienna: IEEE Industrial Electronic Society, 2022. p. 1-8. ISSN 2161-4407. ISBN 978-1-7281-8671-9.
Type
Proceedings paper
Annotation
Designing faster optimization algorithms is of ever-growing interest. In recent years, learning to learn methods that learn how to optimize demonstrated very encouraging results. Current approaches usually do not effectively include the dynamics of the optimization process during training. They either omit it entirely or only implicitly assume the dynamics of an isolated parameter. In this paper, we show how to utilize the dynamic mode decomposition method for extracting informative features about optimization dynamics. By employing those features, we show that our learned optimizer generalizes much better to unseen optimization problems in short. The improved generalization is illustrated on multiple tasks where training the optimizer on one neural network generalizes to different architectures and distinct datasets.

Spatiotemporal Prediction of Vehicle Movement Using Artificial Neural Networks

Year
2022
Published
Proceedings of 2022 IEEE Intelligent Vehicles Symposium (IV). Piscataway: IEEE, 2022. p. 734-739. ISSN 1931-0587. ISBN 978-1-6654-8821-1.
Type
Proceedings paper
Annotation
Prediction of the movement of all traffic participants is a very important task in autonomous driving. Well-predicted behavior of other cars and actors is crucial for safety. A sequence of bird’s-eye view artificially rasterized frames are used as input to neural networks which are trained to predict the future behavior of the participants. The Lyft Motion Prediction for Autonomous Vehicles dataset is explored and adapted for this task. We developed and applied a novel approach where the prediction problem is viewed as a problem of spatiotemporal prediction and we use methods based on convolutional recurrent neural networks.

Discriminant Analysis on a Stream of Features

Authors
Motl, J.; Kordík, P.
Year
2022
Published
Engineering Applications of Neural Networks. Springer, Cham, 2022. p. 223-234. ISSN 1865-0929. ISBN 978-3-031-08222-1.
Type
Proceedings paper
Annotation
Online learning is a well-established problem in machine learning. But while online learning is commonly concerned with learning on a stream of samples, this article is concerned with learning on a stream on features. An online quadratic discriminant analysis (QDA) is proposed because it is fast, capable of modeling feature interactions, and it can still return an exact solution. When a new feature is inserted into a training set, the proposed implementation of QDA showed a 1000-fold speed up to scikit-learn QDA. Fast learning on a stream of features provides a data scientist with timely feedback about the importance of new features during the feature engineering phase. In the production phase, it reduces the cost of updating a model when a new source of potentially useful features appears.

Adapting the Size of Artificial Neural Networks Using Dynamic Auto-Sizing

Authors
Cahlík, V.; Kordík, P.; Čepek, M.
Year
2022
Published
IEEE 17th International Conference on Computer Science and Information Technologies. Dortmund: IEEE, 2022. p. 592-596. ISBN 979-8-3503-3431-9.
Type
Proceedings paper
Annotation
We introduce dynamic auto-sizing, a novel approach to training artificial neural networks which allows the models to automatically adapt their size to the problem domain. The size of the models can be further controlled during the learning process by modifying the applied strength of regularization. The ability of dynamic auto-sizing models to expand or shrink their hidden layers is achieved by periodically growing and pruning entire units such as neurons or filters. For this purpose, we introduce weighted L1 regularization, a novel regularization method for inducing structured sparsity. Besides analyzing the behavior of dynamic auto-sizing, we evaluate predictive performance of models trained using the method and show that such models can provide a predictive advantage over traditional approaches.

Road Quality Classification

Authors
Lank, M.; Friedjungová, M.
Year
2022
Published
Image Analysis and Processing – ICIAP 2022. Cham: Springer International Publishing, 2022. p. 553-563. Lecture Notes in Computer Science. vol. 13232. ISSN 1611-3349. ISBN 978-3-031-06430-2.
Type
Proceedings paper
Annotation
Road quality significantly influences safety and comfort while driving. Especially for most kinds of two-wheelers, road damage is a real threat, where vehicle components and enjoyment are heavily impacted by road quality. This can be avoided by planning a route considering the surface quality. We propose a new publicly available and manually annotated dataset collected from Google Street View photos. This dataset is devoted to a road quality classification task considering six levels of damage. We evaluated some preprocessing methods such as shadow removal, CLAHE, and data augmentation. We adapted several pre-trained networks to classify road quality. The best performance was reached by MobileNet using augmented dataset (75.55%).

Highways in Warehouse Multi-Agent Path Finding: A Case Study.

Year
2022
Published
Proceedings of the 14th International Conference on Agents and Artificial Intelligence. Madeira: SciTePress, 2022. p. 274-281. ISBN 978-989-758-547-0.
Type
Proceedings paper
Annotation
Orchestrating warehouse sorting robots each transporting a single package from the conveyor belt to its destination is a NP-hard problem, often modeled Multi-agent path-finding (MAPF) where the environment is represented as a graph and robots as agents in vertices of the graph. However, in order to maintain the speed of operations in such a setup, sorting robots must be given a route to follow almost at the moment they obtain the package, so there is no time to perform difficult offline planning. Hence in this work, we are inspired by the approach of enriching conflict-based search (CBS) optimal MAPF algorithm by so-called highways that increase the speed of planning towards on-line operations. We investigate whether adding highways to the underlying graph will be enough to enforce global behaviour of a large number of robots that are controlled locally. If we succeed, the slow global planning step could be omitted without significant loss of performance.

Image Inpainting Using Wasserstein Generative Adversarial Imputation Network

Authors
Year
2021
Published
Artificial Neural Networks and Machine Learning – ICANN 2021. Cham: Springer, 2021. p. 575-586. 1. vol. 12892. ISSN 1611-3349. ISBN 978-3-030-86339-5.
Type
Proceedings paper
Annotation
Image inpainting is one of the important tasks in computer vision which focuses on the reconstruction of missing regions in an image. The aim of this paper is to introduce an image inpainting model based on Wasserstein Generative Adversarial Imputation Network. The generator network of the model uses building blocks of convolutional layers with different dilation rates, together with skip connections that help the model reproduce fine details of the output. This combination yields a universal imputation model that is able to handle various scenarios of missingness with sufficient quality. To show this experimentally, the model is simultaneously trained to deal with three scenarios given by missing pixels at random, missing various smaller square regions, and one missing square placed in the center of the image. It turns out that our model achieves high-quality inpainting results on all scenarios. Performance is evaluated using peak signal-to-noise ratio and structural similarity index on two real-world benchmark datasets, CelebA faces and Paris StreetView. The results of our model are compared to biharmonic imputation and to some of the other state-of-the-art image inpainting methods.

Transfer learning based few-shot classification using optimal transport mapping from preprocessed latent space of backbone neural network

Authors
Chobola, T.; Vašata, D.; Kordík, P.
Year
2021
Published
AAAI Workshop on Meta-Learning and MetaDL Challenge. Proceedings of Machine Learning Research, 2021. p. 29-37. vol. 140. ISSN 2640-3498.
Type
Proceedings paper
Annotation
The MetaDL Challenge 2020 focused on image classification tasks in few-shot settings. This paper describes second best submission in the competition. Our meta learning approach modifies the distribution of classes in a latent space produced by a backbone network for each class in order to better follow the Gaussian distribution. After this operation which we call Latent Space Transform algorithm, centers of classes are further aligned in an iterative fashion of the Expectation Maximisation algorithm to utilize information in unlabeled data that are often provided on top of few labelled instances. For this task, we utilize optimal transport mapping using the Sinkhorn algorithm. Our experiments show that this approach outperforms previous works as well as other variants of the algorithm, using K-Nearest Neighbour algorithm, Gaussian Mixture Models, etc.

Dynamic Neural Diversification: Path to Computationally Sustainable Neural Networks

Year
2021
Published
Artificial Neural Networks and Machine Learning – ICANN 2021. Cham: Springer, 2021. p. 235-247. 1. vol. 12892. ISSN 1611-3349. ISBN 978-3-030-86339-5.
Type
Proceedings paper
Annotation
Small neural networks with a constrained number of trainable parameters, can be suitable resource-efficient candidates for many simple tasks, where now excessively large models are used. However, such models face several problems during the learning process, mainly due to the redundancy of the individual neurons, which results in sub-optimal accuracy or the need for additional training steps. Here, we explore the diversity of the neurons within the hidden layer during the learning process, and analyze how the diversity of the neurons affects predictions of the model. As following, we introduce several techniques to dynamically reinforce diversity between neurons during the training. These decorrelation techniques improve learning at early stages and occasionally help to overcome local minima faster. Additionally, we describe novel weight initialization method to obtain decorrelated, yet stochastic weight initialization for a fast and efficient neural network training. Decorrelated weight initialization in our case shows about 40% relative increase in test accuracy during the first 5 epochs.

Unsupervised Latent Space Translation Network

Authors
Year
2020
Published
ESANN 2020 - Proceedings. Louvain la Neuve: Ciaco - i6doc.com, 2020. p. 13-18. ISBN 978-2-87587-074-2.
Type
Proceedings paper
Annotation
One task that is often discussed in a computer vision is the mapping of an image from one domain to a corresponding image in another domain known as image-to-image translation. Currently there are several approaches solving this task. In this paper, we present an enhancement of the UNIT framework that aids in removing its main drawbacks. More specifically, we introduce an additional adversarial discriminator on the latent representation used instead of VAE, which enforces the latent space distributions of both domains to be similar. On MNIST and USPS domain adaptation tasks, this approach greatly outperforms competing approaches.

Missing Features Reconstruction Using a Wasserstein Generative Adversarial Imputation Network

Authors
Year
2020
Published
Computational Science - ICCS 2020. Cham: Springer, 2020. p. 225-239. vol. 12140. ISSN 0302-9743. ISBN 978-3-030-50422-9.
Type
Proceedings paper
Annotation
Missing data is one of the most common preprocessing problems. In this paper, we experimentally research the use of generative and non-generative models for feature reconstruction. Variational Autoencoder with Arbitrary Conditioning (VAEAC) and Generative Adversarial Imputation Network (GAIN) were researched as representatives of generative models, while the denoising autoencoder (DAE) represented non-generative models. Performance of the models is compared to traditional methods k-nearest neighbors (k-NN) and Multiple Imputation by Chained Equations (MICE). Moreover, we introduce WGAIN as the Wasserstein modification of GAIN, which turns out to be the best imputation model when the degree of missingness is less than or equal to 30%. Experiments were performed on real-world and artificial datasets with continuous features where different percentages of features, varying from 10% to 50%, were missing. Evaluation of algorithms was done by measuring the accuracy of the classification model previously trained on the uncorrupted dataset. The results show that GAIN and especially WGAIN are the best imputers regardless of the conditions. In general, they outperform or are comparative to MICE, k-NN, DAE, and VAEAC.

Constructing a Data Visualization Recommender System

Authors
Kubernátová, P.; Friedjungová, M.; van Duijn, M.
Year
2019
Published
Data Management Technologies and Applications. Cham: Springer International Publishing, 2019. p. 1-25. vol. 862. ISSN 1865-0937. ISBN 978-3-030-26636-3.
Type
Proceedings paper
Annotation
Choosing a suitable visualization for data is a difficult task. Current data visualization recommender systems exist to aid in choosing a visualization, yet suffer from issues such as low accessibility and indecisiveness. In this study, we first define a step-by-step guide on how to build a data visualization recommender system. We then use this guide to create a model for a data visualization recommender system for non-experts that aims to resolve the issues of current solutions. The result is a question-based model that uses a decision tree and a data visualization classification hierarchy in order to recommend a visualization. Furthermore, it incorporates both task-driven and data characteristics-driven perspectives, whereas existing solutions seem to either convolute these or focus on one of the two exclusively. Based on testing against existing solutions, it is shown that the new model reaches similar results while being simpler, clearer, more versatile, extendable and transparent. The presented guide can be used as a manual for anyone building a data visualization recommender system. The resulting model can be applied in the development of new data visualization software or as part of a learning tool.

Missing Features Reconstruction and Its Impact on Classification Accuracy

Year
2019
Published
Computational Science – ICCS 2019. Springer, Cham, 2019. p. 207-220. vol. 11538. ISBN 978-3-030-22744-9.
Type
Proceedings paper
Annotation
In real-world applications, we can encounter situations when a well-trained model has to be used to predict from a damaged dataset. The damage caused by missing or corrupted values can be either on the level of individual instances or on the level of entire features. Both situations have a negative impact on the usability of the model on such a dataset. This paper focuses on the scenario where entire features are missing which can be understood as a specific case of transfer learning. Our aim is to experimentally research the influence of various imputation methods on the performance of several classification models. The imputation impact is researched on a combination of traditional methods such as k-NN, linear regression, and MICE compared to modern imputation methods such as multi-layer perceptron (MLP) and gradient boosted trees (XGBT). For linear regression, MLP, and XGBT we also propose two approaches to using them for multiple features imputation. The experiments were performed on both real world and artificial datasets with continuous features where different numbers of features, varying from one feature to 50%, were missing. The results show that MICE and linear regression are generally good imputers regardless of the conditions. On the other hand, the performance of MLP and XGBT is strongly dataset dependent. Their performance is the best in some cases, but more often they perform worse than MICE or linear regression.

Chameleon 2: An Improved Graph-Based Clustering Algorithm

Authors
Bartoň, T.; Brůna, T.; Kordík, P.
Year
2019
Published
ACM Transactions on Knowledge Discovery from Data. 2019, 13(1), ISSN 1556-4681.
Type
Article
Annotation
Traditional clustering algorithms fail to produce human-like results when confronted with data of variable density, complex distributions, or in the presence of noise. We propose an improved graph-based clustering algorithm called Chameleon 2, which overcomes several drawbacks of state-of-the-art clustering approaches. We modified the internal cluster quality measure and added an extra step to ensure algorithm robustness. Our results reveal a significant positive impact on the clustering quality measured by Normalized Mutual Information on 32 artificial datasets used in the clustering literature. This significant improvement is also confirmed on real-world datasets. The performance of clustering algorithms such as DBSCAN is extremely parameter sensitive, and exhaustive manual parameter tuning is necessary to obtain a meaningful result. All hierarchical clustering methods are very sensitive to cutoff selection, and a human expert is often required to find the true cutoff for each clustering result. We present an automated cutoff selection method that enables the Chameleon 2 algorithm to generate high-quality clustering in autonomous mode.

Comparing Offline and Online Evaluation Results of Recommender Systems

Authors
Kordík, P.; Řehořek, T.; Bíža, O.; Bartyzal, R.; Podsztavek, O.; Povalyev, I.P.
Year
2018
Published
REVEAL RecSyS 2018 workshop proceedings. New York: ACM, 2018.
Type
Proceedings paper
Annotation
Recommender systems are usually trained and evaluated on historical data. Offline evaluation is, however, tricky and offline performance can be an inaccurate predictor of the online performance measured in production due to several reasons. In this paper, we experiment with two offline evaluation strategies and show that even a reasonable and popular strategy can produce results that are not just biased, but also in direct conflict with the true performance obtained in the online evaluation. We investigate offline policy evaluation techniques adapted from reinforcement learning and explain why such techniques fail to produce an unbiased estimate of the online performance in the “watch next” scenario of a large-scale movie recommender system. Finally, we introduce a new evaluation technique based on Jaccard Index and show that it correlates with the online performance.

An Overview of Transfer Learning Focused on Asymmetric Heterogeneous Approaches

Year
2018
Published
Data Management Technologies and Applications. Cham: Springer International Publishing, 2018. p. 3-26. vol. 814. ISSN 1865-0929. ISBN 978-3-319-94809-6.
Type
Proceedings paper
Annotation
In practice we often encounter classification tasks. In order to solve these tasks, we need a sufficient amount of quality data for the construction of an accurate classification model. However, in some cases, the collection of quality data poses a demanding challenge in terms of time and finances. For example in the medical area, we encounter lack of data about patients. Transfer learning introduces the idea that a possible solution can be combining data from different domains represented by different feature spaces relating to the same task. We can also transfer knowledge from a different but related task that has been learned already. This overview focuses on the current progress in the novel area of asymmetric heterogeneous transfer learning. We discuss approaches and methods for solving these types of transfer learning tasks. Furthermore, we mention the most used metrics and the possibility of using metric or similarity learning.

Discovering predictive ensembles for transfer learning and meta-learning

Authors
Kordík, P.; Frýda, T.; Černý, J.
Year
2018
Published
Machine Learning. 2018, 107(1), 177-207. ISSN 0885-6125.
Type
Article
Annotation
Recent meta-learning approaches are oriented towards algorithm selection, optimization or recommendation of existing algorithms. In this article we show how data-tailored algorithms can be constructed from building blocks on small data sub-samples. Building blocks, typically weak learners, are optimized and evolved into data-tailored hierarchical ensembles. Good-performing algorithms discovered by evolutionary algorithm can be reused on data sets of comparable complexity. Furthermore, these algorithms can be scaled up to model large data sets. We demonstrate how one particular template (simple ensemble of fast sigmoidal regression models) outperforms state-of-the-art approaches on the Airline data set. Evolved hierarchical ensembles can therefore be beneficial as algorithmic building blocks in meta-learning, including meta-learning at scale.

Asymmetric Heterogeneous Transfer Learning: A Survey

Year
2017
Published
Proceedings of the 6th International Conference on Data Science, Technology and Applications. Porto: SciTePress - Science and Technology Publications, 2017. p. 17-27. vol. 1. ISBN 978-989-758-255-4.
Type
Proceedings paper
Annotation
One of the main prerequisites in most machine learning and data mining tasks is that all available data originates from the same domain. In practice, we often can’t meet this requirement due to poor quality, unavailable data or missing data attributes (new task, e.g. cold-start problem). A possible solution can be the combination of data from different domains represented by different feature spaces, which relate to the same task. We can also transfer the knowledge from a different but related task that has been learned already. Such a solution is called transfer learning and it is very helpful in cases where collecting data is expensive, difficult or impossible. This overview focuses on the current progress in the new and unique area of transfer learning - asymmetric heterogeneous transfer learning. This type of transfer learning considers the same task solved using data from different feature spaces. Through suitable mappings between these different feature spaces we can get more data for solving data mining tasks. We discuss approaches and methods for solving this type of transfer learning tasks. Furthermore, we mention the most used metrics and the possibility of using metric or similarity learning.

Neural Turing Machine for Sequential Learning of Human Mobility Patterns

Authors
Kordík, P.; Tkačík, J.
Year
2016
Published
2016 International Joint Conference on Neural Networks (IJCNN). San Francisco: American Institute of Physics and Magnetic Society of the IEEE, 2016. p. 2790-2797. ISSN 2161-4407. ISBN 978-1-5090-0620-5.
Type
Proceedings paper
Annotation
The capacity of recurrent neural networks to learn complex sequential patterns is improving. Recent developments such as Clockwork RNN, Stack RNN, Memory networks and Neural Turing Machine all aim to increase long-term memory capacity of recurrent neural networks. In this study, we investigate properties of Neural Turing Machine, compare it with ensembles of Stack RNN on artificial benchmarks and applied it to learn human mobility patterns. We show, that Neural Turing Machine based predictor outperformed not only n-gram based prediction, but also neighborhood based predictor, that was designed to solve this particular problem. Our models will be deployed in anti-drug police department to predict mobility of suspects.

The person responsible for the content of this page: doc. Ing. Štěpán Starosta, Ph.D.