Maximally informative dimensions

Mathematical formulation

Neural stimulus-response functions are typically given as the probability of a neuron generating an action potential, or spike, in response to a stimulus ${\displaystyle \mathbf{s}}$. The goal of maximally informative dimensions is to find a small relevant subspace of the much larger stimulus space that accurately captures the salient features of ${\displaystyle \mathbf{s}}$. Let ${\displaystyle D}$ denote the dimensionality of the entire stimulus space and ${\displaystyle K}$ denote the dimensionality of the relevant subspace, such that ${\displaystyle K\ll D}$. We let ${\displaystyle \{{\mathbf{v}}^K\}}$ denote the basis of the relevant subspace, and ${\displaystyle {\mathbf{s}}^K}$ the projection of ${\displaystyle \mathbf{s}}$ onto ${\displaystyle \{{\mathbf{v}}^K\}}$. Using Bayes' theorem we can write out the probability of a spike given a stimulus:

${\displaystyle P(spike|{\mathbf{s}}^K)=P(spike)f({\mathbf{s}}^K)}$

where

${\displaystyle f({\mathbf{s}}^K)=\frac{P({\mathbf{s}}^K|spike)}{P({\mathbf{s}}^K)}}$

is some nonlinear function of the projected stimulus.

In order to choose the optimal ${\displaystyle \{{\mathbf{v}}^K\}}$, we compare the prior stimulus distribution ${\displaystyle P(\mathbf{s})}$ with the spike-triggered stimulus distribution ${\displaystyle P(\mathbf{s}|spike)}$ using the Shannon information. The average information (averaged across all presented stimuli) per spike is given by

${\displaystyle I_{spike}=\sum _{\mathbf{s}}P(\mathbf{s}|spike)lo{g}_2[P(\mathbf{s}|spike)/P(\mathbf{s})]}$.

Now consider a ${\displaystyle K=1}$ dimensional subspace defined by a single direction ${\displaystyle \mathbf{v}}$. The average information conveyed by a single spike about the projection ${\displaystyle x=\mathbf{s}\cdot \mathbf{v}}$ is

${\displaystyle I(\mathbf{v})=\int dx{P}_{\mathbf{v}}(x|spike)log2[P_{\mathbf{v}}(x|spike)/P_{\mathbf{v}}(x)]}$,

where the probability distributions are approximated by a measured data set via ${\displaystyle P_{\mathbf{v}}(x|spike)=⟨\delta (x-\mathbf{s}\cdot \mathbf{v})|spike{⟩}_{\mathbf{s}}}$ and ${\displaystyle P_{\mathbf{v}}(x)=⟨\delta (x-\mathbf{s}\cdot \mathbf{v}){⟩}_{\mathbf{s}}}$, i.e., each presented stimulus is represented by a scaled Dirac delta function and the probability distributions are created by averaging over all spike-eliciting stimuli, in the former case, or the entire presented stimulus set, in the latter case. For a given dataset, the average information is a function only of the direction ${\displaystyle \mathbf{v}}$. Under this formulation, the relevant subspace of dimension ${\displaystyle K=1}$ would be defined by the direction ${\displaystyle \mathbf{v}}$ that maximizes the average information ${\displaystyle I(\mathbf{v})}$.

This procedure can readily be extended to a relevant subspace of dimension ${\displaystyle K>1}$ by defining

${\displaystyle P_{{\mathbf{v}}^K}(\mathbf{x}|spike)=⟨\prod _{i=1}^K\delta (x_i-\mathbf{s}\cdot {\mathbf{v}}_i)|spike{⟩}_{\mathbf{s}}}$

and

${\displaystyle P_{{\mathbf{v}}^K}(\mathbf{x})=⟨\prod _{i=1}^K\delta (x_i-\mathbf{s}\cdot {\mathbf{v}}_i){⟩}_{\mathbf{s}}}$

and maximizing ${\displaystyle I({\mathbf{v}}^K)}$.