Machine Learning Metrics for Network Datasets Evaluation
Autoři
Rok
2024
Publikováno
ICT Systems Security and Privacy Protection. Cham: Springer, 2024. p. 307-320. ISSN 1868-4238. ISBN 978-3-031-56325-6.
Typ
Stať ve sborníku
Pracoviště
Anotace
High-quality datasets are an essential requirement for leveraging machine learning (ML) in data processing and recently in network security as well. However, the quality of datasets is overlooked or underestimated very often. Having reliable metrics to measure and describe the input dataset enables the feasibility assessment of a dataset. Imperfect datasets may require optimization or updating, e.g., by including more data and merging class labels. Applying ML algorithms will not bring practical value if a dataset does not contain enough information. This work addresses the neglected topics of dataset evaluation and missing metrics. We propose three novel metrics to estimate the quality of an input dataset and help with its improvement or building a new dataset. This paper describes experiments performed on public datasets to show the benefits of the proposed metrics and theoretical definitions for more straightforward interpretation. Additionally, we have implemented and published Python code so that the metrics can be adopted by the worldwide scientific community.
Meningioma microstructure assessed by diffusion MRI: An investigation of the source of mean diffusivity and fractional anisotropy by quantitative histology
Autoři
Brabec, J.; Friedjungová, M.; Vašata, D.; Englund, E.; Bengzon, J.; Knutsson, L.; Szczepankiewicz, F.; van Westen, D.; Sundgren, P.C.; Nilsson, M.
Rok
2023
Publikováno
Neuroimage: Clinical. 2023, 37 ISSN 2213-1582.
Typ
Článek
Anotace
Mean diffusivity (MD) and fractional anisotropy (FA) from diffusion MRI (dMRI) have been associated with cell density and tissue anisotropy across tumors, but it is unknown whether these associations persist at the microscopic level.
To quantify the degree to which cell density and anisotropy, as determined from histology, account for the intra-tumor variability of MD and FA in meningioma tumors. Furthermore, to clarify whether other histological features account for additional intra-tumor variability of dMRI parameters.
We performed ex-vivo dMRI at 200 μm isotropic resolution and histological imaging of 16 excised meningioma tumor samples. Diffusion tensor imaging (DTI) was used to map MD and FA, as well as the in-plane FA (FAIP). Histology images were analyzed in terms of cell nuclei density (CD) and structure anisotropy (SA; obtained from structure tensor analysis) and were used separately in a regression analysis to predict MD and FAIP, respectively. A convolutional neural network (CNN) was also trained to predict the dMRI parameters from histology patches. The association between MRI and histology was analyzed in terms of out-of-sample (R2OS) on the intra-tumor level and within-sample R2 across tumors. Regions where the dMRI parameters were poorly predicted from histology were analyzed to identify features apart from CD and SA that could influence MD and FAIP, respectively.
Cell density assessed by histology poorly explained intra-tumor variability of MD at the mesoscopic level (200 μm), as median R2OS = 0.04 (interquartile range 0.01–0.26). Structure anisotropy explained more of the variation in FAIP (median R2OS = 0.31, 0.20–0.42). Samples with low R2OS for FAIP exhibited low variations throughout the samples and thus low explainable variability, however, this was not the case for MD. Across tumors, CD and SA were clearly associated with MD (R2 = 0.60) and FAIP (R2 = 0.81), respectively. In 37% of the samples (6 out of 16), cell density did not explain intra-tumor variability of MD when compared to the degree explained by the CNN. Tumor vascularization, psammoma bodies, microcysts, and tissue cohesivity were associated with bias in MD prediction based solely on CD. Our results support that FAIP is high in the presence of elongated and aligned cell structures, but low otherwise.
Cell density and structure anisotropy account for variability in MD and FAIP across tumors but cell density does not explain MD variations within the tumor, which means that low or high values of MD locally may not always reflect high or low tumor cell density. Features beyond cell density need to be considered when interpreting MD.
Challenges In Train Evacuation Modelling Using Pathfinder
Autoři
Hrabák, P.; Kmec, J.; Najmanová, H.; Vašata, D.
Rok
2022
Publikováno
Proceedings of the 6th Fire and Evacuation Modeling Technical Conference. Manhattan, KS: Thunderhead Engineering, 2022.
Typ
Stať ve sborníku
Anotace
In comparison to buildings, public transportation vehicles including passenger trains represent specific environments typical for cramped interior layouts with narrow aisles, limited number of exits, and various operation conditions resulting in non-standard exit paths such as direct egress to the track level. Therefore, in evacuation modelling, safety analysis of passenger trains may denote a challenging task, especially when using evacuation models originally developed for simulations of evacuation processes from buildings (such as Pathfinder). In this case, several issues can be encountered: lack of validation against relevant experimental data, missing features to simulate non-standard exit path, lack of knowledge which key parameters to adjust and how.
This contribution addresses these issues while performing an egress simulation analysis of a rail car of double-deck electric unit class 471 (CityElefant) using Pathfinder. The analysis was based on a full-scale controlled evacuation experiment carried out in Prague in 2018. We introduce a technique for adjusting the Pathfinder model that simulates rail car egress to match the experimental results, and we include sensitivity analysis of key parameters influencing total evacuation time (TET).
From the obtained results several conclusions can be drawn:
1. Application of the SFPE curve for a speed-density relationship had a significant impact on TET due to narrow aisles and limited space available;
2. The parameters of agent diameter and squeeze factor have to be handled together (when associating the agent diameter with the width of passenger shoulders, the value of squeeze factor must be decreased);
3. While egressing to the track level, the time delay caused by the preparation and realization of the jump must be implemented adequately to different passenger abilities;
The impact of seating preferences of passengers with heterogeneous moving abilities prevails the influence of maximal speed or diameter distribution on TET.
Evacuation trials from a double-deck electric train unit: Experimental data and sensitivity analysis
Autoři
Najmanová, H.; Kuklík, L.; Pešková, V.; Bukáček, M.; Vašata, D.; Hrabák, P.
Rok
2022
Publikováno
Safety Science. 2022, 146 ISSN 0925-7535.
Typ
Článek
Anotace
Passenger trains represent a challenging environment in emergencies, with specific evacuation conditions resulting from the typical layout and interior design inherent to public transportation vehicles. This paper describes a dataset obtained in a full-scale controlled experiment emulating the emergency evacuation of a double-deck electric unit railcar carried out in Prague in 2018. 15 evacuation trials involving 91 participants were conducted under various evacuation scenarios considering different compositions of passenger crowd, exit widths, and exit types (egress to a high platform, to an open rail line using stairs, and a 750 mm jump without any supporting equipment). The study’s main goals were to collect experimental data on the movement conditions in the railcar and to study the impact of various boundary conditions on evacuation process and total evacuation time. Movement characteristics (exit flows, speeds) and human behaviour (pre-movement activities, exiting behaviours) were also analysed. The data obtained was used to validate and adjust a Pathfinder model to capture important aspects of evacuation from the railcar. Furthermore, a series of simulations using this model was performed to provide sensitivity analysis of the influence of crowd composition, exit width, and exit type on total evacuation time. As a key finding, we can conclude that for the case of a standard exit path (platform or stairs) the width of the main exit had the greatest impact on total evacuation time, however, crowd composition played the prevailing role in evacuation scenarios involving a jump.
Learning to Optimize with Dynamic Mode Decomposition
Autoři
Rok
2022
Publikováno
2022 International Joint Conference on Neural Networks (IJCNN). Vienna: IEEE Industrial Electronic Society, 2022. p. 1-8. ISSN 2161-4407. ISBN 978-1-7281-8671-9.
Typ
Stať ve sborníku
Anotace
Designing faster optimization algorithms is of ever-growing interest. In recent years, learning to learn methods that learn how to optimize demonstrated very encouraging results. Current approaches usually do not effectively include the dynamics of the optimization process during training. They either omit it entirely or only implicitly assume the dynamics of an isolated parameter. In this paper, we show how to utilize the dynamic mode decomposition method for extracting informative features about optimization dynamics. By employing those features, we show that our learned optimizer generalizes much better to unseen optimization problems in short. The improved generalization is illustrated on multiple tasks where training the optimizer on one neural network generalizes to different architectures and distinct datasets.
Image Inpainting Using Wasserstein Generative Adversarial Imputation Network
Autoři
Vašata, D.; Halama, T.; Friedjungová, M.
Rok
2021
Publikováno
Artificial Neural Networks and Machine Learning – ICANN 2021. Cham: Springer, 2021. p. 575-586. 1. vol. 12892. ISSN 1611-3349. ISBN 978-3-030-86339-5.
Typ
Stať ve sborníku
Anotace
Image inpainting is one of the important tasks in computer vision which focuses on the reconstruction of missing regions in an image. The aim of this paper is to introduce an image inpainting model based on Wasserstein Generative Adversarial Imputation Network. The generator network of the model uses building blocks of convolutional layers with different dilation rates, together with skip connections that help the model reproduce fine details of the output. This combination yields a universal imputation model that is able to handle various scenarios of missingness with sufficient quality. To show this experimentally, the model is simultaneously trained to deal with three scenarios given by missing pixels at random, missing various smaller square regions, and one missing square placed in the center of the image. It turns out that our model achieves high-quality inpainting results on all scenarios. Performance is evaluated using peak signal-to-noise ratio and structural similarity index on two real-world benchmark datasets, CelebA faces and Paris StreetView. The results of our model are compared to biharmonic imputation and to some of the other state-of-the-art image inpainting methods.
Transfer learning based few-shot classification using optimal transport mapping from preprocessed latent space of backbone neural network
Autoři
Chobola, T.; Vašata, D.; Kordík, P.
Rok
2021
Publikováno
AAAI Workshop on Meta-Learning and MetaDL Challenge. Proceedings of Machine Learning Research, 2021. p. 29-37. vol. 140. ISSN 2640-3498.
Typ
Stať ve sborníku
Anotace
The MetaDL Challenge 2020 focused on image classification tasks in few-shot settings. This paper describes second best submission in the competition. Our meta learning approach modifies the distribution of classes in a latent space produced by a backbone network for each class in order to better follow the Gaussian distribution. After this operation which we call Latent Space Transform algorithm, centers of classes are further aligned in an iterative fashion of the Expectation Maximisation algorithm to utilize information in unlabeled data that are often provided on top of few labelled instances. For this task, we utilize optimal transport mapping using the Sinkhorn algorithm. Our experiments show that this approach outperforms previous works as well as other variants of the algorithm, using K-Nearest Neighbour algorithm, Gaussian Mixture Models, etc.
Missing Features Reconstruction Using a Wasserstein Generative Adversarial Imputation Network
Autoři
Rok
2020
Publikováno
Computational Science - ICCS 2020. Cham: Springer, 2020. p. 225-239. vol. 12140. ISSN 0302-9743. ISBN 978-3-030-50422-9.
Typ
Stať ve sborníku
Anotace
Missing data is one of the most common preprocessing problems. In this paper, we experimentally research the use of generative and non-generative models for feature reconstruction. Variational Autoencoder with Arbitrary Conditioning (VAEAC) and Generative Adversarial Imputation Network (GAIN) were researched as representatives of generative models, while the denoising autoencoder (DAE) represented non-generative models. Performance of the models is compared to traditional methods k-nearest neighbors (k-NN) and Multiple Imputation by Chained Equations (MICE). Moreover, we introduce WGAIN as the Wasserstein modification of GAIN, which turns out to be the best imputation model when the degree of missingness is less than or equal to 30%. Experiments were performed on real-world and artificial datasets with continuous features where different percentages of features, varying from 10% to 50%, were missing. Evaluation of algorithms was done by measuring the accuracy of the classification model previously trained on the uncorrupted dataset. The results show that GAIN and especially WGAIN are the best imputers regardless of the conditions. In general, they outperform or are comparative to MICE, k-NN, DAE, and VAEAC.
Unsupervised Latent Space Translation Network
Autoři
Rok
2020
Publikováno
ESANN 2020 - Proceedings. Louvain la Neuve: Ciaco - i6doc.com, 2020. p. 13-18. ISBN 978-2-87587-074-2.
Typ
Stať ve sborníku
Anotace
One task that is often discussed in a computer vision is the mapping of an image from one domain to a corresponding image in another domain known as image-to-image translation. Currently there are several approaches solving this task. In this paper, we present an enhancement of the UNIT framework that aids in removing its main drawbacks. More specifically, we introduce an additional adversarial discriminator on the latent representation used instead of VAE, which enforces the latent space distributions of both domains to be similar. On MNIST and USPS domain adaptation tasks, this approach greatly outperforms competing approaches.
Missing Features Reconstruction and Its Impact on Classification Accuracy
Autoři
Rok
2019
Publikováno
Computational Science – ICCS 2019. Springer, Cham, 2019. p. 207-220. vol. 11538. ISBN 978-3-030-22744-9.
Typ
Stať ve sborníku
Anotace
In real-world applications, we can encounter situations when a well-trained model has to be used to predict from a damaged dataset. The damage caused by missing or corrupted values can be either on the level of individual instances or on the level of entire features. Both situations have a negative impact on the usability of the model on such a dataset. This paper focuses on the scenario where entire features are missing which can be understood as a specific case of transfer learning. Our aim is to experimentally research the influence of various imputation methods on the performance of several classification models. The imputation impact is researched on a combination of traditional methods such as k-NN, linear regression, and MICE compared to modern imputation methods such as multi-layer perceptron (MLP) and gradient boosted trees (XGBT). For linear regression, MLP, and XGBT we also propose two approaches to using them for multiple features imputation. The experiments were performed on both real world and artificial datasets with continuous features where different numbers of features, varying from one feature to 50%, were missing. The results show that MICE and linear regression are generally good imputers regardless of the conditions. On the other hand, the performance of MLP and XGBT is strongly dataset dependent. Their performance is the best in some cases, but more often they perform worse than MICE or linear regression.
Cantor spectra of magnetic chain graphs
Autoři
Exner, P.; Vašata, D.
Rok
2017
Publikováno
Journal of Physics A: Mathematical and Theoretical. 2017, 50(16), ISSN 1751-8113.
Typ
Článek
Anotace
We demonstrate a one-dimensional magnetic system can exhibit a Cantortype
spectrum using an example of a chain graph with δ coupling at the
vertices exposed to a magnetic field perpendicular to the graph plane and
varying along the chain. If the field grows linearly with an irrational slope,
measured in terms of the flux through the loops of the chain, we demonstrate
the character of the spectrum relating it to the almost Mathieu operator.
Generalizations of the centroid with an application in stochastic geometry
Autoři
Rok
2017
Publikováno
Mathematische Nachrichten. 2017, 290(2-3), 452-473. ISSN 0025-584X.
Typ
Článek
Anotace
The centroid of a subset of math formula with positive volume is a well-known characteristic. An interesting task is to generalize its definition to at least some sets of zero volume. In the presented paper we propose two possible ways how to do that. The first is based on the Hausdorff measure of an appropriate dimension. The second is given by the limit of centroids of ε-neighbourhoods of the particular set when ε goes to 0. For both generalizations we discuss their existence and basic properties. Then we focus on sufficient conditions of existence of the second generalization and on conditions when both generalizations coincide. It turns out that they can be formulated with the help of the Minkowski content, rectifiability, and self-similarity. Since the centroid is often used in stochastic geometry as a centre function for certain particle processes, we present properties that are needed for both generalizations to be valid centre functions. Finally, we also show their continuity on compact convex m-sets with respect to the Hausdorff metric topology.
On long-range dependence of random measures
Autoři
Rok
2016
Publikováno
Advances in Applied Probability. 2016, 48(4), 1235-1255. ISSN 0001-8678.
Typ
Článek
Anotace
This paper deals with long-range dependence of random measures on ℝd. By examples, it is demonstrated that one must be careful in order to define it consistently. Therefore, we define long-range dependence by a rather specific second-order condition and provide an equivalent formulation involving the asymptotic behaviour of the Bartlett spectrum near the origin. Then it is shown that the defining condition may be formulated less strictly when the additional isotropy assumption holds. Finally, we present an example of a long-range dependent random measure based on the 0-level excursion set of a Gaussian random field for which the corresponding spectral density and its asymptotics are explicitly derived.