Notation in probability and statistics

Probability theory

Random variables are usually written in upper case Roman letters, such as $X$ or $Y$ and so on. Random variables, in this context, usually refer to something in words, such as "the height of a subject" for a continuous variable, or "the number of cars in the school car park" for a discrete variable, or "the colour of the next bicycle" for a categorical variable. They do not represent a single number or a single category. For instance, if $P (X = x)$ is written, then it represents the probability that a particular realisation of a random variable (e.g., height, number of cars, or bicycle colour), X, would be equal to a particular value or category (e.g., 1.735 m, 52, or purple), $x$ . It is important that $X$ and $x$ are not confused into meaning the same thing. $X$ is an idea, $x$ is a value. Clearly they are related, but they do not have identical meanings.
Particular realisations of a random variable are written in corresponding lower case letters. For example, $x_{1}, x_{2}, \dots, x_{n}$ could be a sample corresponding to the random variable $X$ . A cumulative probability is formally written $P (X \leq x)$ to distinguish the random variable from its realization.
The probability is sometimes written $P$ to distinguish it from other functions and measure P to avoid having to define "P is a probability" and $P (X \in A)$ is short for $P ({ω \in Ω : X (ω) \in A})$ , where $Ω$ is the sample space, $X$ is a random variable that is a function of $ω$ (i.e., it depends upon $ω$ ), and $ω$ is some outcome of interest within the domain specified by $Ω$ (say, a particular height, or a particular colour of a car). $Pr (A)$ notation is used alternatively.
$P (A \cap B)$ or $P [B \cap A]$ indicates the probability that events A and B both occur. The joint probability distribution of random variables X and Y is denoted as $P (X, Y)$ , while joint probability mass function or probability density function as $f (x, y)$ and joint cumulative distribution function as $F (x, y)$ .
$P (A \cup B)$ or $P [B \cup A]$ indicates the probability of either event A or event B occurring ("or" in this case means one or the other or both).
σ-algebras are usually written with uppercase calligraphic (e.g. $F$ for the set of sets on which we define the probability P)
Probability density functions (pdfs) and probability mass functions are denoted by lowercase letters, e.g. $f (x)$ , or $f_{X} (x)$ .
Cumulative distribution functions (cdfs) are denoted by uppercase letters, e.g. $F (x)$ , or $F_{X} (x)$ .
Survival functions or complementary cumulative distribution functions are often denoted by placing an overbar over the symbol for the cumulative: $\bar{F} (x) = 1 - F (x)$ , or denoted as $S (x)$ ,
In particular, the pdf of the standard normal distribution is denoted by $φ (z)$ , and its cdf by $Φ (z)$ .
Some common operators:

$E [X]$ : expected value of X
$var [X]$ : variance of X
$cov [X, Y]$ : covariance of X and Y

X is independent of Y is often written $X ⊥ Y$ or $X ⊥ ⊥ Y$ , and X is independent of Y given W is often written

X ⊥ ⊥ Y | W

X ⊥ Y | W

$P (A ∣ B)$ , the conditional probability, is the probability of $A$ given $B$

4 more chapters are free with a Lably account.

Statistics

Greek letters (e.g. θ, β) are commonly used to denote unknown parameters (population parameters).
Some commonly used symbols for population or distribution parameters are given below:
- the population mean $μ$ ,
- the population variance $σ^{2}$ ,
- the population standard deviation $σ$ ,
- the population correlation $ρ$ ,
- the population cumulants $κ_{r}$ ,
A tilde (~) denotes "has the probability distribution of".
Placing a hat, or caret (also known as a circumflex), over a true parameter denotes an estimator of it, e.g., $\hat{θ}$ is an estimator for $θ$ .
The arithmetic mean of a series of values $x_{1}, x_{2}, \dots, x_{n}$ is often denoted by placing an "overbar" over the symbol, e.g. $\bar{x}$ , pronounced " $x$ bar".
Some commonly used symbols for sample statistics are given below:
- the sample mean $\bar{x}$ ,
- the sample variance $s^{2}$ ,
- the sample standard deviation $s$ ,
- the sample correlation coefficient $r$ ,
- the sample cumulants $k_{r}$ .
$x_{(k)}$ is used for the $k^{th}$ order statistic, where $x_{(1)}$ is the sample minimum and $x_{(n)}$ is the sample maximum from a total sample size $n$ .

Critical values

The α-level upper critical value of a probability distribution is the value exceeded with probability $α$ , that is, the value $x_{α}$ such that $F (x_{α}) = 1 - α$ , where $F$ is the cumulative distribution function. There are standard notations for the upper critical values of some commonly used distributions in statistics:

$z_{α}$ or $z (α)$ for the standard normal distribution
$t_{α, ν}$ or $t (α, ν)$ for the t-distribution with $ν$ degrees of freedom
${χ_{α, ν}}^{2}$ or $χ^{2} (α, ν)$ for the chi-squared distribution with $ν$ degrees of freedom
$F_{α, ν_{1}, ν_{2}}$ or $F (α, ν_{1}, ν_{2})$ for the F-distribution with $ν_{1}$ and $ν_{2}$ degrees of freedom

Linear algebra

Matrices are usually denoted by boldface capital letters, e.g. $A$ .
Column vectors are usually denoted by boldface lowercase letters, e.g. $x$ .
The transpose operator is denoted by either a superscript T (e.g. $A^{T}$ ) or a prime symbol (e.g. $A^{'}$ ).
A row vector is written as the transpose of a column vector, e.g. $x^{T}$ or $x^{'}$ .

Abbreviations

Common abbreviations include:

a.e. almost everywhere
a.s. almost surely
cdf cumulative distribution function
cmf cumulative mass function
df degrees of freedom (also $ν$ )
i.i.d. independent and identically distributed
pdf probability density function
pmf probability mass function
r.v. random variable
w.p. with probability; wp1 with probability 1
i.o. infinitely often, i.e. ${A_{n} i.o.} = ⋂_{N} ⋃_{n \geq N} A_{n}$
ult. ultimately, i.e. ${A_{n} ult.} = ⋃_{N} ⋂_{n \geq N} A_{n}$