Lably
Skip to content

Topological data analysis

Analysis of datasets using techniques from topology

  • 21 min read
  • 10 chapters

Related on Lably

Through Topology, Scikit-learn and Grayscale

The initial motivation is to study the shape of data. TDA has combined algebraic topology and other tools from pure mathematics to allow mathematically rigorous study of "shape". The main tool is persistent homology, an adaptation of homology to point cloud data. Persistent homology has been applied to many types of data across many fields. Moreover, its mathematical foundation is also of theoretical importance. The unique features of TDA make it a promising bridge between topology and geometry.

Basic theory

Intuition

TDA is premised on the idea that the shape of data sets contains relevant information. Real high-dimensional data is typically sparse, and tends to have relevant low dimensional features. One task of TDA is to provide a precise characterization of this fact. For example, the trajectory of a simple predator-prey system governed by the Lotka–Volterra equations forms a closed circle in state space. TDA provides tools to detect and quantify such recurrent motion.

Many algorithms for data analysis, including those used in TDA, require setting various parameters. Without prior domain knowledge, the correct collection of parameters for a data set is difficult to choose. The main insight of persistent homology is to use the information obtained from all parameter values by encoding this huge amount of information into an understandable and easy-to-represent form. With TDA, there is a mathematical interpretation when the information is a homology group. In general, the assumption is that features that persist for a wide range of parameters are "true" features. Features persisting for only a narrow range of parameters are presumed to be noise, although the theoretical justification for this is unclear.

Early history

Precursors to the full concept of persistent homology appeared gradually over time. In 1990, Patrizio Frosini introduced a pseudo-distance between submanifolds, and later the size function, which on 1-dim curves is equivalent to the 0th persistent homology. Nearly a decade later, Vanessa Robins studied the images of homomorphisms induced by inclusion. Finally, shortly thereafter, Herbert Edelsbrunner et al. introduced the concept of persistent homology together with an efficient algorithm and its visualization as a persistence diagram. Gunnar Carlsson et al. reformulated the initial definition and gave an equivalent visualization method called persistence barcodes, interpreting persistence in the language of commutative algebra.

In algebraic topology the persistent homology has emerged through the work of Sergey Barannikov on Morse theory. The set of critical values of smooth Morse function was canonically partitioned into pairs "birth-death", filtered complexes were classified, their invariants, equivalent to persistence diagram and persistence barcodes, together with the efficient algorithm for their calculation, were described under the name of canonical forms in 1994 by Barannikov.

Concepts

Some widely used concepts are introduced below. Note that some definitions may vary from author to author.

A point cloud is often defined as a finite set of points in some Euclidean space, but may be taken to be any finite metric space.

The Čech complex of a point cloud is the nerve of the cover of balls of a fixed radius around each point in the cloud.

A persistence module U indexed by Z is a vector space Ut for each t∈Z, and a linear map uts:Us→Ut whenever s≤t, such that utt=1 for all t and utsusr=utr whenever r≤s≤t. An equivalent definition is a functor from Z considered as a partially ordered set to the category of vector spaces.

The persistent homology group PH of a point cloud is the persistence module defined as PHk(X)=∏Hk(Xr), where Xr is the Čech complex of radius r of the point cloud X and Hk is the homology group.

A persistence barcode is a multiset of intervals in R, and a persistence diagram is a multiset of points in Δ(:={(u,v)∈R2∣u,v≥0,u≤v}).

The Wasserstein distance between two persistence diagrams X and Y is defined as Wp[Lq](X,Y):=infφ:X→Y[∑x∈X(‖x−φ(x)‖q)p]1/pwhere 1≤p,q≤∞ and φ ranges over bijections between X and Y. Please refer to figure 3.1 in Munch for illustration.

The bottleneck distance between X and Y is W∞[Lq](X,Y):=infφ:X→Ysupx∈X‖x−φ(x)‖q. This is a special case of Wasserstein distance, letting p=∞.

Basic property

Structure theorem

The first classification theorem for persistent homology appeared in 1994 via Barannikov's canonical forms. The classification theorem interpreting persistence in the language of commutative algebra appeared in 2005: for a finitely generated persistence module C with field F coefficients, H(C;F)≃⨁ixti⋅F[x]⊕(⨁jxrj⋅(F[x]/(xsj⋅F[x]))). Intuitively, the free parts correspond to the homology generators that appear at filtration level ti and never disappear, while the torsion parts correspond to those that appear at filtration level rj and last for sj steps of the filtration (or equivalently, disappear at filtration level sj+rj).

Persistent homology is visualized through a barcode or persistence diagram. The barcode has its root in abstract mathematics. Namely, the category of finite filtered complexes over a field is semi-simple. Any filtered complex is isomorphic to its canonical form, a direct sum of one- and two-dimensional simple filtered complexes.

Stability

Stability is desirable because it provides robustness against noise. If X is any space which is homeomorphic to a simplicial complex, and f,g:X→R are continuous tame functions, then the persistence vector spaces {Hk(f−1([0,r]))} and {Hk(g−1([0,r]))} are finitely presented, and W∞(D(f),D(g))≤‖f−g‖∞, where W∞ refers to the bottleneck distance and D is the map taking a continuous tame function to the persistence diagram of its k-th homology.

Workflow

The basic workflow in TDA is:

  1. If X is a point cloud, replace X with a nested family of simplicial complexes Xr (such as the Čech or Vietoris-Rips complex). This process converts the point cloud into a filtration of simplicial complexes. Taking the homology of each complex in this filtration gives a persistence module Hi(Xr0)→Hi(Xr1)→Hi(Xr2)→⋯
  2. Apply the structure theorem to obtain the persistent Betti numbers, persistence diagram, or equivalently, barcode.

Graphically speaking,

Sign in to keep reading

9 more chapters are free with a Lably account.

More in Statistics