# Back of the Envelope

Observations on the Theory and Empirics of Mathematical Finance

## [PDS] Probability in Finance: Key Ideas – I

Need for a Mathematical Theory of Probability

One of the reasons we need a mathematical (read measure-theoretic) foundations of probability is that when dealing with infinite sample spaces (choosing a number at random in $[0, 1]$ for example), there is no immediately obvious way of assigning probabilities to ‘not very likely events’. Let me elaborate.

While it is not difficult to understand that when selecting a number at random from $[0, 1]$, the probability that it lies between $[0, 0.5]$ would be $0.5$, it is not immediately obvious what would be the probability that the number selected would be one of $\{0.001, 0.002, ... , 0.999\}$ or, say,  a rational number $(\mathbb{Q})$.

Both the sets $\{0.001, 0.002, ... , 0.999\}$ and $\mathbb{Q}$ are countable and, while seemingly big, are yet ‘too small’ compared to all the points in $[0, 1]$. Also, the set of rational numbers $\mathbb{Q}$ is not is not even an interval in the way, say, $[0, 0.5]$ is.

The fact that there are subsets on the real line $\mathbb{R}$ which are not intervals or not ‘nice’ subsets (e.g. the Cantor Set) means that the notion of length as distance between two points is not enough. We need a ‘better scale’ if-you-will that allows us to measure the length of sets like $\mathbb{Q}$ and identify the ‘smallness’ of countable sets.

This ‘better scale’ that we are looking for is known as the Lebesgue Measure. (Of course, there are many other advantages of using the idea of measure than to just assign probabilities, but that needn’t concern us for now.)

But before we lay out the properties of this scale, we need some new machinery.

Null Sets

Just like the development of numbers begins with defining the number $0$, development of a theory of measure begins with defining sets which are negligible, or alternatively, null sets.

Null sets are those which have a measure $0$ according to our new scale. Knowing sets that have a measure $0$ then allows us to identify sets that have a ‘finite length’.

Null Sets: Definition

A set $A \subseteq \mathbb{R}$ is null if it can be covered by a sequence of intervals $\{I_n: n \ge 1\}$ of near-zero total length, i.e. $A$ is null if given an arbitrary small $\epsilon > 0$:

$A \subseteq \displaystyle\bigcup_{i=1}^{\infty} I_n$

$\displaystyle\sum_{i=1}^{\infty} l(I_n) < \epsilon$

The definition says that an arbitrary set is null if the total length of sets which cover it is ‘very small’. This implies that any countable set $A$ is null as:

$A = \{1, 2, \cdots N\} \subseteq [1, 1] \cup [2, 2] \cdots \cup [N, N] = \displaystyle\bigcup_{i=1}^{N} [i, i]$

$\displaystyle\sum_{i=1}^{\infty} l([k, k]) = 0< \epsilon$

Since length of the closed interval is $l([k, k]) = 0$, the length of set $A$ must also be zero (it is a countable sum of zero-length closed intervals). This suggests that given the way we have defined null sets, all countable sets, including the set of rational numbers $\mathbb{Q}$, are null sets – that is have a measure $0$.

Outer Measure

The notion of covering/approximating a set by a sequence of intervals turns out to be an important and very useful step in constructing a theory of measures. Continuing with the notion of covers, first we define what is called the Outer Measure.

Outer Measure: Definition

The Outer Measure of a set $A \subseteq \mathbb{R}$ is denoted by $m^*(A)$ and is given by:

$m^*(A) = \inf \{\displaystyle\sum_{i=1}^{\infty} l(I_n): A \subseteq \displaystyle\bigcup_{i=1}^{\infty} I_n\}$

Intuitively this definition says the length (Outer Measure) of a set is the smallest total length of all intervals that cover the set.

The reason for developing a ‘new scale’ (measure) was so that we could measure all kinds of arbitrary subsets on $\mathbb{R}$. But having done so the least we would expect is that for intervals measure gives us the same answer as its ‘length’. Indeed – all the expected/intuitive properties of length are preserved by Outer Measure. Below we list its important properties (taken directly from Capinski and Kopp, including the notation):

Outer Measure: Properties

1. $A \subseteq \mathbb{R}$ is null iff $m^*(A) =0$

2. $A, B \subseteq \mathbb{R}$ and $A \subset B$ $\Rightarrow m^*(A) \le m^*(B)$

3. Outer Measure of an interval equals its length, i.e. $m^*([a, b]) = b - a$

4. Outer Measure is countably sub-additive, i.e. for a sequence of sets $\{A_n: i \ge 1\}$ (not necessarily disjoint)

$\displaystyle m^*\Big( \bigcup_{n=1}^{\infty} A_n \Big) \le \sum_{n=1}^{\infty} m^*(A_n)$

While this is great, the fourth property of Outer Measure gives us some cause for concern. The problem is that it doesn’t guarantee that measure of a union of disjoint sets does not necessarily add-up to the sum of measure of individual sets. And what do we mean by that?

Consider two intervals $A = [a, b]$ with measure $b - a$ and $B=\left(b, c \right]$ with measure $c - b$ which are disjoint, then we should expect that length of the set $A \cup B = [a, c]$ should have a length $c - a=(b-a)+(c-b)$. While for this example value of Outer Measure respects our intuition of length, in general this additivity property is not true for Outer Measure for all subsets of $\mathbb{R}$. And we do not like that!

Ok, admittedly, this is only the case for really nasty sets, but for now (and in probability) we do not want to work with such nasty sets. So we are looking for those subsets of $\mathbb{R}$ for which the Outer Measure is additive for disjoint sets, i.e. we only want to work with those subsets of $\mathbb{R}$ for which if $A_i \cap A_j = \varnothing$ $\forall i, j$ then:

$\displaystyle m^*\Big( \bigcup_{n=1}^{\infty} A_n \Big)=\sum_{n=1}^{\infty} m^*(A_n)$

Collection of all subsets of $\mathbb{R}$ for which this additivty property holds (and all other properties of Outer Measure) are called Lebesgue-Measurable Sets and is denoted by $\mathbb{M}$.

Lebesgue Measure

The collection of subsets $\mathbb{M}$ is important enough to warrant a different notion of measure for which the additivty property holds. Outer Measure with additivity property for sets in $\mathbb{M}$ is called the Lebesgue Measure on $\mathbb{R}$. We use the notation $m$ for Lebesgue measures and write:

$\displaystyle m\Big( \bigcup_{n=1}^{\infty} A_n \Big)=\sum_{n=1}^{\infty} m(A_n)$

Technical definition of Lebesgue Measure is more than what we need at this stage and can be found, for example, in Capinski and Kopp. For us, thinking of Lebesgue measure as an Outer Measure with additivity property is about enough. Needless to say, all the other properties of Outer Measure carry through to the Lebesgue Measure.

Lebesgue Measurable Sets

The collection $\mathbb{M}$ is clearly a ‘nice’ subset of $\mathbb{R}$. Other than the additivity property of the Lebesgue measure for subsets of $\mathbb{M}$, its subsets have some other ‘nice’ notable properties:

1. $\mathbb{R} \in \mathbb{M}$

2. $A \in \mathbb{M}$ $\Rightarrow A^c \in \mathbb{M}$

3. $A_i \in \mathbb{M}$ $\forall i \ge 1$ $\displaystyle \Rightarrow \cup_{i=1}^{\infty}A_i \in \mathbb{M}$

The above properties just say that when one does simple operations (taking ‘complements’ and ‘unions’) on sets belonging to $\mathbb{M}$ we remain in $\mathbb{M}$, i.e. doing simple operations on sets in $\mathbb{M}$ does not take us ‘out of’ $\mathbb{M}$ – we remain in the ‘nice’ world of $\mathbb{M}$.

(Those familiar with the notion of $\sigma$-fields would recognize that $\mathbb{M}$ forms a $\sigma$-field on $\mathbb{R}$. )

Measure Space

We began with the set $\mathbb{R}$. We considered ‘nice’ subsets of $\mathbb{R}$ which were Lebesgue measurable (in the sense above) and called the collection of all such subsets as the Lebesgue-measurable sets $\mathbb{M}$. So we have three things now: the underlying set $\mathbb{R}$, collection of Lebesgue-measurable subsets $\mathbb{M}$, and the Lebesgue measure $m$. For the sake of brevity we often call this ‘triple’:

$\big(\mathbb{R}, \mathbb{M}, m\big)$

as the measure space.

At this stage it is also useful to explicitly identify the Lebesgue measure $m$ as a ‘scale’ that assigns each subset in $\mathbb{M}$ a ‘length’ (measure). Given our understanding of a function, then this is what a Lebesgue measure is. $m$ is a function that assigns to each subset in $\mathbb{M}$ a number between $0$ (for null sets) and $\infty$ (for uncountable sets like $\mathbb{R}$), i.e.:

$m:\mathbb{M} \rightarrow \left[0, \infty\right)$

Next we extend the idea of measure space to abstract spaces replacing $\mathbb{R}$ by $\Omega$, $\mathbb{M}$ by $\mathbb{F}$ ($\sigma$-field on $\Omega$) and the Lebesgue measure $m$ by a measure $\mathbb{P}$ called the probability measure. The resulting space $(\Omega,\mathbb{F}, \mathbb{P})$ is then called a probability space.

[PS: Much of the discussion in this post summarizes the treatment of measure-theoretic ideas as in Capinski and Kopp]

Written by Vineet

February 13, 2013 at 11:30 pm

Posted in Teaching: PDS

Tagged with , ,