Multiscale Image Interrogation
Our research in image representation and interrogation aims to transform standard biomedical imaging into novel biomarker discovery. By extracting multiscale information from both radiological images (i.e., radiomics) and digital pathology images (i.e., pathomics), we are able to characterize the appearance and behavior of disease across different spatial, temporal, and functional domains. Our team develops both classical feature engineering solutions (i.e., well-defined mathematical descriptors of image texture, morphology, etc.), and deep learning solutions (i.e., where features take the form of learnable imaging filters). We often integrate both strategies into our research as complementary approaches to image representation and knowledge extraction. We aim to improve diagnostics, quantify treatment response, and enable personalized therapy. In addition, we apply our technology to interrogate the underlying biology of images, characterize tissue microenvironments, and study radiogenomic interactions. Because our research is general, it can be applied to various areas of biology and medicine. Current projects include applications in head and neck cancer, lung cancer, diffuse liver disease, chronic kidney disease, and transplantation.
Figure 1. Multi-scale mathematical representation of head and neck cancer. (a) Computational biomarking the metabolic response of oropharyngeal cancer from intra-treatment radiomic expression. Quantitative imaging features associated with oropharyngeal cancer recurrence are identifed by high-throughput radiomic sequencing. (b) An illustrating example comparing the images and radiomic signatures of two different patients demonstrating the importance of metabolic heterogeneity to treatment resistance. (c) Computational characterization of the inflammatory microenvironment of head and neck cancer based on deep learning and graph theory. A whole slide image of H&E stained head and neck cancer (left) and correspnding inflammatory signature (right). (d) High magnitude patch (left) with corresponding deep learning immune cell detection (middle) and graph-based infalmmatory signature (right).
Images as Dynamical Systems
Our research in high-dimensional data analysis is motivated by the large p, small n problems that frequently arise in computational imaging. We predominantly work on unsupervised learning formalisms, where data is modeled and studied based on its intrinsic properties. This is a powerful first-line approach to understanding high-dimensional data, particularly when supervised machine learning is limited due to small samples sizes of labelled data (as is often the case with rare diseases, novel therapies, and clinical trial data). In particular, we focus on nature-inspired computational methods and soft-computing paradigms to search for latent knowledge embedded in images. Our research on this topic incorporates the applied analysis of stochastic differential equations, self-organization, and quantum machine learning (i.e., an emerging branch of research that explores methodological and structural similarities between quantum systems and learning systems).
Figure 2. Data characterization of imaging features modeled as dynamical systems. (Left) A self-consistent potential function, V(x), is constructed directly from data (i.e., high-dimensional imaging features). When V(x) has several distinct minima, if we propagate the trajectories starting from the data points of a given data set, with a high probability, those trajectories will aggregate into several groups within a short time. Further, if the profile of the potential function intrinsically indicates the density distribution of the data, the data points that end up in the neighborhood of the same minima of V (q) after O(1) time can be considered to be in the same cluster. While the forces from the potential surface push data points towards potential minima, corresponding Brownian fluctuations allow them to jump small potential barriers and escape saddle points into locations of the potential surface otherwise forbidden. (Top Right) Numerical verification of ergodic sampling at critical temperature. Marginal sampling distributions obtained across a projection of Hilbert space are shown relative to the analytic marginal Gibbs distribution left. Data are shown to be more localized at sub-critical temperature right due to non-ergodic dynamical behavior, and do not fully sample the potential landscape. (Bottom Right) A 2D projection of Hilbert space demonstrating ergodic behavior at critical temperature left and self-organization of data (i.e., clustering) at sub-critical temperature right.