Conditional probability distribution

Conditional continuous distributions

Similarly for continuous random variables, the conditional probability density function of ${\displaystyle Y}$ given the occurrence of the value ${\displaystyle x}$ of ${\displaystyle X}$ can be written as

${\displaystyle f_{Y\mid X}(y\mid x)=\frac{f_{X,Y}(x,y)}{f_X(x)}}$

where ${\displaystyle f_{X,Y}(x,y)}$ gives the joint density of ${\displaystyle X}$ and ${\displaystyle Y}$, while ${\displaystyle f_X(x)}$ gives the marginal density for ${\displaystyle X}$. Also in this case it is necessary that ${\displaystyle f_X(x)>0}$.

The relation with the probability distribution of ${\displaystyle X}$ given ${\displaystyle Y}$ is given by:

${\displaystyle f_{Y\mid X}(y\mid x)f_X(x)=f_{X,Y}(x,y)=f_{X|Y}(x\mid y)f_Y(y).}$

The concept of the conditional distribution of a continuous random variable is not as intuitive as it might seem: Borel's paradox shows that conditional probability density functions need not be invariant under coordinate transformations.

Example

The graph shows a bivariate normal joint density for random variables ${\displaystyle X}$ and ${\displaystyle Y}$. To see the distribution of ${\displaystyle Y}$ conditional on ${\displaystyle X=70}$, one can first visualize the line ${\displaystyle X=70}$ in the ${\displaystyle X,Y}$ plane, and then visualize the plane containing that line and perpendicular to the ${\displaystyle X,Y}$ plane. The intersection of that plane with the joint normal density, once rescaled to give unit area under the intersection, is the relevant conditional density of ${\displaystyle Y}$.

${\displaystyle Y\mid X=70\sim \mathcal{N}\left({\mu }_Y+\frac{{\sigma }_Y}{{\sigma }_X}\rho (70-{\mu }_X),(1-{\rho }^2){\sigma }_Y^2\right).}$

Relation to independence

Random variables ${\displaystyle X}$, ${\displaystyle Y}$ are independent if and only if the conditional distribution of ${\displaystyle Y}$ given ${\displaystyle X}$ is, for all possible realizations of ${\displaystyle X}$, equal to the unconditional distribution of ${\displaystyle Y}$. For discrete random variables this means ${\displaystyle P(Y=y|X=x)=P(Y=y)}$ for all possible ${\displaystyle y}$ and ${\displaystyle x}$ with ${\displaystyle P(X=x)>0}$. For continuous random variables ${\displaystyle X}$ and ${\displaystyle Y}$, having a joint density function, it means ${\displaystyle f_Y(y|X=x)=f_Y(y)}$ for all possible ${\displaystyle y}$ and ${\displaystyle x}$ with ${\displaystyle f_X(x)>0}$.

Properties

Seen as a function of ${\displaystyle y}$ for given ${\displaystyle x}$, ${\displaystyle P(Y=y|X=x)}$ is a probability mass function and so the sum over all ${\displaystyle y}$ (or integral if it is a conditional probability density) is 1. Seen as a function of ${\displaystyle x}$ for given ${\displaystyle y}$, it is a likelihood function, so that the sum (or integral) over all ${\displaystyle x}$ need not be 1.

Additionally, a marginal of a joint distribution can be expressed as the expectation of the corresponding conditional distribution. For instance, ${\displaystyle p_X(x)=E_Y[p_{X|Y}(x|Y)]}$.

Measure-theoretic formulation

Let ${\displaystyle (\mathrm{\Omega },\mathcal{F},P)}$ be a probability space, ${\displaystyle \mathcal{G}\subseteq \mathcal{F}}$ a ${\displaystyle \sigma }$-field in ${\displaystyle \mathcal{F}}$. Given ${\displaystyle A\in \mathcal{F}}$, the Radon–Nikodym theorem implies that there is a ${\displaystyle \mathcal{G}}$-measurable random variable ${\displaystyle P(A\mid \mathcal{G}):\mathrm{\Omega }\to \mathbb{R}}$, called the conditional probability, such that$${\displaystyle {\int }_{G}P(A\mid \mathcal{G})(\omega )dP(\omega )=P(A\cap G)}$$for every ${\displaystyle G\in \mathcal{G}}$, and such a random variable is uniquely defined up to sets of probability zero. A conditional probability is called regular if ${\displaystyle \mathrm{P}(\cdot \mid \mathcal{G})(\omega )}$ is a probability measure on ${\displaystyle (\mathrm{\Omega },\mathcal{F})}$ for all ${\displaystyle \omega \in \mathrm{\Omega }}$ a.e.

Special cases:

• For the trivial sigma algebra ${\displaystyle \mathcal{G}=\{\mathrm{\varnothing },\mathrm{\Omega }\}}$, the conditional probability is the constant function ${\displaystyle \mathrm{P}\left(A\mid \{\mathrm{\varnothing },\mathrm{\Omega }\}\right)=\mathrm{P}(A).}$
• If ${\displaystyle A\in \mathcal{G}}$, then ${\displaystyle \mathrm{P}(A\mid \mathcal{G})=1_A}$, the indicator function (defined below).

Let ${\displaystyle X:\mathrm{\Omega }\to E}$ be a ${\displaystyle (E,\mathcal{E})}$-valued random variable. For each ${\displaystyle B\in \mathcal{E}}$, define $${\displaystyle {\mu }_{X|\mathcal{G}}(B|\mathcal{G})=\mathrm{P}(X^{-1}(B)|\mathcal{G}).}$$For any ${\displaystyle \omega \in \mathrm{\Omega }}$, the function ${\displaystyle {\mu }_{X|\mathcal{G}}(\cdot |\mathcal{G})(\omega ):\mathcal{E}\to \mathbb{R}}$ is called the conditional probability distribution of ${\displaystyle X}$ given ${\displaystyle \mathcal{G}}$. If it is a probability measure on ${\displaystyle (E,\mathcal{E})}$, then it is called regular.

For a real-valued random variable (with respect to the Borel ${\displaystyle \sigma }$-field ${\displaystyle {\mathcal{R}}^1}$ on ${\displaystyle \mathbb{R}}$), every conditional probability distribution is regular. In this case,${\displaystyle E[X\mid \mathcal{G}]={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}x{\mu }_{X\mid \mathcal{G}}(dx,\cdot )}$ almost surely.

Relation to conditional expectation

For any event ${\displaystyle A\in \mathcal{F}}$, define the indicator function:

${\displaystyle {\mathbf{1}}_A(\omega )=\{\begin{array}{ll}1& \text{if }\omega \in A,\\ 0& \text{if }\omega \notin A,\end{array}}$

which is a random variable. Note that the expectation of this random variable is equal to the probability of A itself:

${\displaystyle \mathrm{E}({\mathbf{1}}_A)=\mathrm{P}(A).}$

Given a ${\displaystyle \sigma }$-field ${\displaystyle \mathcal{G}\subseteq \mathcal{F}}$, the conditional probability ${\displaystyle \mathrm{P}(A\mid \mathcal{G})}$ is a version of the conditional expectation of the indicator function for ${\displaystyle A}$:

${\displaystyle \mathrm{P}(A\mid \mathcal{G})=\mathrm{E}({\mathbf{1}}_A\mid \mathcal{G})}$

An expectation of a random variable with respect to a regular conditional probability is equal to its conditional expectation.

Interpretation of conditioning on a Sigma Field

Consider the probability space ${\displaystyle (\mathrm{\Omega },\mathcal{F},\mathbb{P})}$ and a sub-sigma field ${\displaystyle \mathcal{A}\subset \mathcal{F}}$. The sub-sigma field ${\displaystyle \mathcal{A}}$ can be loosely interpreted as containing a subset of the information in ${\displaystyle \mathcal{F}}$. For example, we might think of ${\displaystyle \mathbb{P}(B|\mathcal{A})}$ as the probability of the event ${\displaystyle B}$ given the information in ${\displaystyle \mathcal{A}}$.

Also recall that an event ${\displaystyle B}$ is independent of a sub-sigma field ${\displaystyle \mathcal{A}}$ if ${\displaystyle \mathbb{P}(B|A)=\mathbb{P}(B)}$ for all ${\displaystyle A\in \mathcal{A}}$. It is incorrect to conclude in general that the information in ${\displaystyle \mathcal{A}}$ does not tell us anything about the probability of event ${\displaystyle B}$ occurring. This can be shown with a counter-example:

Consider a probability space on the unit interval, ${\displaystyle \mathrm{\Omega }=[0,1]}$. Let ${\displaystyle \mathcal{G}}$ be the sigma-field of all countable sets and sets whose complement is countable. So each set in ${\displaystyle \mathcal{G}}$ has measure ${\displaystyle 0}$ or ${\displaystyle 1}$ and so is independent of each event in ${\displaystyle \mathcal{F}}$. However, notice that ${\displaystyle \mathcal{G}}$ also contains all the singleton events in ${\displaystyle \mathcal{F}}$ (those sets which contain only a single ${\displaystyle \omega \in \mathrm{\Omega }}$). So knowing which of the events in ${\displaystyle \mathcal{G}}$ occurred is equivalent to knowing exactly which ${\displaystyle \omega \in \mathrm{\Omega }}$ occurred! So in one sense, ${\displaystyle \mathcal{G}}$ contains no information about ${\displaystyle \mathcal{F}}$ (it is independent of it), and in another sense it contains all the information in ${\displaystyle \mathcal{F}}$.