# Characteristic Function and Convergence

Reivew & Summary
$$\begin{cases} e^{iux}=\cos(ux) + i\sin(ux) \\ e^z = \sum\limits_{n=0}^{ \infty} \frac{z^n}{n!} \end{cases}$$;
$$\begin{cases} \hat\mu(x) := \displaystyle\int_{ \mathbb R^d } e^{i \langle x,y \rangle }d\mu(y)\\ \hat f(x):= \mathbb{ E} e^{i \langle x,f \rangle } = \displaystyle\int_{ \Omega } e^{i \langle x,f(\omega) \rangle }d \mathbb{ P}(\omega) = \displaystyle\int_{ \mathbb R^d } e^{i \langle x,y \rangle } d\mu_f (y) \end{cases}$$ ;
Properties of $$\hat\mu \begin{cases} \hat\mu: \mathbb{R}^d \to \mathbb{C} \text{ is uniformly continuous} \\ \hat\mu(0) = 1\\ |\hat\mu(x)| \leq \hat\mu(0) \\ \sum\limits_{k,l=1}^{ n} \lambda_k \bar\lambda_l \hat\mu(x_k-x_l) \geq 0 \\ \hat\mu+\hat\nu = \widehat{\mu+\nu} \\ \widehat{\mu*\nu} = \hat\mu \hat\nu \\ \hat\mu = \hat\nu\ ⇔\ \mu=\mu \end{cases}$$ ;
Convolution $$\begin{cases} (\mu_1 * \cdots * \mu_n) (B) = (\mu_1 \times \cdots \times \mu_n)(\{(x_1,\dots,x_n): x_1 +\cdots+x_n \in B\}) \\ (f_1*f_2)(x):= \displaystyle\int_{ \mathbb R^d } f_1(x-y)f_2(y)d \lambda_d(y)\end{cases}$$;
Polya Theorem: $$\varphi$$ is Fourier transform if $$\varphi: \mathbb{R} \to [0,\infty) \begin{cases} \text{continuous} \\ \text{even } \varphi(x)=\varphi(-x) \\ \text{convex on } [0,\infty) \\ \varphi(0)=1,\ \lim\limits_{x \to \infty} \varphi(x)=0\end{cases}$$;
Fourier transformation: $$\begin{cases} \widehat\gamma _{m,\sigma^2} (u)=e^{imx}e^{-\frac{1}{2} \sigma^2 x^2}\\ \widehat{\text{pois} }_\lambda (u) = e^{\lambda(e^{iu}-1)}\\ \widehat{\text{u} }_{[-a,a]}(u) = \begin{cases} \frac{\sin(au)}{au} & u\neq0 \\ 1 &u=0 \end{cases} \end{cases}$$ ;
Bochner & Chinčin: $$\varphi$$ is Fourier Transformation ⇔ $$\begin{cases} \varphi: \mathbb{R} ^d \to \mathbb{C} \text{ is continuous}\\ \varphi(0)=1 \\ \varphi \text{ is positive semidefinite} \end{cases}$$ ;
$$\exists \mathbf{ R} \text{ (covariance matrix)},\ \mathbf{m} \text{(expectation vector)}:\ \hat\mu(x) = e^{i \langle \mathbf{ x} , \mathbf{ m} \rangle – \frac{1}{2} \langle \mathbf{ Rx}, \mathbf{ x} \rangle }$$;
A Gaussian measure is degenerated if $$rank(R)\lt d$$;
$$f ⫫ g \ ⇒ \widehat{(f,g)}=\hat f \hat g \ ⇒ cov(f,g) = 0$$ ;
Moment of $$(l_1,\dots,l_d):\ \displaystyle\int_{ \mathbb R^d } y_1^{l_1} \cdots y_d^{l_d} d\mu (y) = \frac{1} {i^{l_1 + \cdots + l_d}} \frac{\partial ^{l_1 + \cdots + l_d}}{\partial x_1^{l_1}\cdots x_d^{l_d}} \hat\mu(0)$$ ;
Central limit theorem: $$\mathbb{P} \bigg( \frac{(f_1-m)+\cdots+(f_k-m)}{\sqrt{k\sigma^2}} \leq x \bigg) \overset{k}{\longrightarrow} \frac{1}{\sqrt{2\pi}} \displaystyle\int_{ -\infty } ^x e^ {- \frac{\xi^2}{2} }d\xi$$ ;
$$\begin{cases} \mu_n \overset{ \text{w} }{\longrightarrow} \mu: & \hat\mu_n \overset{n}{\longrightarrow} \mu \ ⇔\ F_n(x) \overset{n}{\longrightarrow} F(x)\ \forall x\\ f_n \overset{a.s.}{\longrightarrow} f: & \lim\limits_{ n \to \infty} \mathbb{P} (\sup \limits_{ k\geq n}|f_k-f|\gt\varepsilon) = 0\ ⇔\ \mathbb{P} (\{|f_n-f| \overset{n}{\longrightarrow} 0\})=1\\ f_n \overset{\mathbb{P} }{\longrightarrow} f: & \lim\limits_{ n \to \infty} \mathbb{P} (|f_n-f|\gt \varepsilon) =0 \ ⇔\ \lim\limits_{ n \to \infty} d(f_n,f) = 0 \\ f_n \overset{L_p}{\longrightarrow} f: & \lim\limits_{ n \to \infty} \displaystyle\int_{ \Omega } |f_n-f|^p d \mathbb{P} =0 \end{cases}$$ .

# Complex Numbers

Määr Complex number space
▻▻ $$\begin{cases} \mathbb{ C} \cong \mathbb{ R}^2 = \{(x,y): x,y\in \mathbb{ R} \} \\ z=x+iy \in \mathbb{ C} \end{cases}$$ .
Lause Let $$z_1=(x_1,y_1),\ z_2=(x_2,y_2)$$ :
▻▻ $$\begin{cases} \text{Addition} &z_1+z_2 = (x_1+x_2) + i(y_1+y_2) \\ \text{Multiplication} &z_1 z_2 = (x_1 x_2 – y_1 y_2) + i (x_1 y_2 + x_2 y_1) \\ \text{Length} & |z_1| = \sqrt{x_1^2 + y_1^2} \\ \text{Conjugate} & \bar z_1 = x_1 – iy_1 \\ \text{Polar coorinates} &\begin{cases} x= r \cos\varphi \\ y = r \sin \varphi \\r = |z| \end{cases} \end{cases}$$ .
Määr $$e^z := \sum\limits_{ n=1}^{ \infty} \frac{ z^n}{ n!}$$ .
Lause Euler's formula $$e^{ix} = \cos x + i \sin x$$ .

## Complex Valued Random Variables

Määr A map $$f:\Omega\to \mathbb{ C}$$ is called  measurable provided that $$f:\Omega \to \mathbb{ R}^2$$ is measurable.
Määr A map $$f:\Omega\to \mathbb{ C}$$ is called  integrable provided that $$\displaystyle\int_{ \Omega} |f(\omega)|d \mathbb{ P}(\omega) \lt \infty$$ , where
▻▻ $$\displaystyle\int_{ \Omega} f(\omega)d \mathbb{ P}(\omega) = \displaystyle\int_{ \Omega} \mathcal{ Re} (f(\omega))d \mathbb{ P}(\omega) + i \displaystyle\int_{ \Omega} \mathcal{ Im} (f(\omega))d \mathbb{ P}(\omega)$$ .

# Basics of Characteristic Functions

## Definitions

Määr $$\mathcal{ M}_1^+ (\mathbb{ R}^d )$$ is the set of all probability measure $$\mu$$ on $$\mathcal{ B}(\mathbb{ R}^d)$$ .
▻▻ The “1” stands for $$\mu (\mathbb{ R}^d ) =1$$ and the “+” for $$\mu (A) \geq 0$$ .
Määr Fourier transformation of $$\mu$$ is $$\hat \mu (x):= \displaystyle\int_{ \mathbb{ R}^d } e^{i \langle x,y \rangle }\ d\mu(y)$$ .
Määr characteristic function of $$f$$ is $$\hat f(x) := \mathbb{ E}e^{i \langle x,f \rangle } = \displaystyle\int_{ \Omega} e^{i \langle x,f(\omega) \rangle }\ d \mathbb{ P} (\omega)$$ .
Esim For Dirac-measure: $$\hat \delta_a(x) = \displaystyle\int_{ \mathbb{ R} } e^{i \langle x,y \rangle }\ d\delta_a(y) = e^{i \langle x,a \rangle }$$ .
Määr Fourier transformation of $$\varphi: \hat\varphi (x):= \displaystyle\int_{ \mathbb{ R}^d } e^{i \langle x,y \rangle } \varphi (y)\ d\lambda (y)$$ ( $$\varphi$$ must be lebesgue-integrable)
Määr Finite signed measure: $$\mu = \mu^+ – \mu^-,\ \ \mu^± : \mathcal{ f}\to [0,\infty)$$ and the space is denoted by $$\mathcal{ M}(\Omega, \mathcal{ F} )$$ .

## Properties of $$\mathcal{ M}_1^+(\mathbb{ R}^d )$$

Lause Let $$\mu \in \mathcal{ M}_1^+(\mathbb{ R}^d )$$ :
(1) $$\hat\mu : \mathbb{ R}^d \to \mathbb{ C}$$ is uniformly continuous:
▻▻ $$\forall \varepsilon \gt 0, \ \exists\delta\gt 0: |x-y|\leq \delta\ ⇒ \ |\hat\mu(x) -\hat\mu(y)|\leq\varepsilon$$ ;
(2) $$\forall x \in \mathbb{ R}^d: |\hat\mu(x)| \leq \hat\mu(0)=1$$ ;
(3) $$\hat \mu$$ is always positive semi-definite, that is:
▻▻ $$\forall x_i\in \mathbb{ R}^d, \ \lambda_i\in \mathbb{ C}: \sum\limits_{ i=1}^{ n} \lambda_k \bar\lambda_l \hat\mu(x_k-x_l) \geq 0$$ .

# Convolutins

Määr Convolution of measures Let $$\mu_1,\dots,\mu_n \in \mathcal{ M}_1^+(\mathbb{ R}^d )$$ Then, $$\mu_1 * \cdots *\mu_n \in \mathcal{ M}_1^+(\mathbb{ R}^d )$$ is the law of $$f:\Omega\to \mathbb{ R}^d$$ defined by:
▻▻ $$\begin{cases} (\Omega, \mathcal{ F}, \mathbb{ P} ):= (\mathbb{ R}^d\times\cdots\times \mathbb{ R}^d,\ \mathcal{ B}(\mathbb{ R}^d) \otimes \cdots \otimes \mathcal{ B}(\mathbb{ R}^d),\ \mu_1\times\cdots\times\mu_n ) \\ f(x_1,\dots,x_n):= x_1 + \cdots + x_n \end{cases}$$ ,
⇔ $$(\mu_1 * \cdots *\mu_n)(B) = (\mu_1 \times \cdots \times \mu_n) (\{(x_1,\dots,x_n):x_1+\cdots+x_n \in B\})$$ .
Lause For $$\mu_1,\mu_2,\mu_3 \in \mathcal{ M}_1^+(\mathbb{ R}^d )$$ :
(1) $$\mu_1 * \mu_2 = \mu_2 * \mu_1$$ ;
(2) $$\mu_1 * (\mu_2 * \mu_3) = (\mu_1 * \mu_2) * \mu_3 = \mu_1 * \mu_2 * \mu_3$$ .
Esim Convolutions of functions For integrable $$f_1,f_2:\mathbb{ R}^d \to \mathbb{ R}$$ :
▻▻ If $$f_1(x)\gt 0, f_2(x) \gt 0$$ . Then $$(f_1 *f_2)(x) := \displaystyle\int_{ \mathbb{ R}^d } f_1(x-y)f_2(y)\ d\lambda_d (y)$$ .
▻▻ For all functions: $$(f_1 * f_2)(x) = (f_1^+ * f_2^+)(x) -( f_1^+ * f_2^-)(x) – (f_1^- * f_2^+)(x) +( f_1^- * f_2^-)(x)$$ .
Lause Properties of convolutions: let $$f_1,f_2: \mathbb{ R}^d \to \mathbb{ R}$$ :
(1) $$f_1*f_2: \mathbb{ R}^d \to [0,\infty]$$ is a nextended measruable function;
(2) $$\lambda_d(\{x: (f_1*f_2)(x)=\infty\})=0$$ ;
(3) $$\displaystyle\int_{ \mathbb{ R}^d } (f_1*f_2)\ d\lambda_d = \displaystyle\int_{ \mathbb{ R}^d } f_1\ d\lambda_d \displaystyle\int_{ \mathbb{ R}^d } f_2\ d\lambda_d$$ .
Lause (connecting two convolutions)
Let $$\mu_1,\mu_2\in \mathcal{ M}_1^+(\mathbb{ R}^d )$$ with $$\mu_i := \displaystyle\int_{ \mathbb{ R}^d } \large\chi\normalsize_{ B}(x)p_i(x)\ d\lambda_d(x)$$ and $$\displaystyle\int_{ \mathbb{ R}^d } p_i(x)\ d\lambda_d (x)=1$$ .
▻▻ Then, $$(\mu_1*\mu_2)(B) = \displaystyle\int_{ \mathbb{ R}^d } (p_1*p_2)(x) \large\chi\normalsize_{ B}(x) \ d\lambda_d(x)$$ .

# Important Properties of Measure Space

Lause For $$\mu,\nu \in \mathcal{ M}_1^+(\mathbb{ R}^d )$$ :
(1) $$\hat\mu + \hat\nu = \widehat{\mu+\nu}$$ ;
(2) $$\widehat{\mu*\nu} = \hat\mu \hat\nu$$ ;
(3) $$\widehat{A(\mu)} (x)= \hat\mu(A^T x) \text{ , where } A = (a_{ij})^d_{i,j=1}$$ a linear trnasformation;
(4) $$\widehat{S_a(\mu)}= \hat\delta_a \hat\mu \text{ , where } S_a(x) = x+a$$ a sihft operator.
Määr π-system  $$\forall A,B\in\Pi:A\cap B\in \Pi$$ .
Lause $$\mu_i \in \mathcal{ M}_1^+(\mathbb{ R}^d )$$ are the laws of $$f_i$$ , i.e. $$\mathbb{ P} (f_k\in B) = \mu_k(B),\ \forall B$$ ⇒ $$\mu_1*\dots*\mu_n$$ is the laws of $$S:=f_1+\cdots+f_n$$ .
Lause $$\mu=\nu \ ⇔ \ \hat\mu=\hat\nu$$ .
Määr continuous function space and mode.
▻▻ $$\begin{cases} C_0(\mathbb{ R}^d; \mathbb{ C} )= \big\{ g: \mathbb{ R}^d \to \mathbb{ C} \text{ continuous and } \lim\limits_{ |x| \to \infty} |g(x)|=0 \big\} \\ ||g||_{C_0} := \sup \limits_{ x\in \mathbb{ R}^d } |g(x)| \end{cases}$$ .
Lause Riemann & Lebesgue $$f \in L_1(\mathbb{ R}^d; \mathbb{ C} ) \ ⇒ \ \hat f\in C_0(\mathbb{ R}^d; \mathbb{ C} )$$ .
Lause $$A:= \big\{ \hat f: \mathbb{ R}^d \to \mathbb{ C}:f\in L_1 (\mathbb{ R}^d; \mathbb{ C} ) \big\} \subseteq C_0(\mathbb{ R}^d; \mathbb{ C} )$$ is dense.
Lause Bochner & Chinein Assume $$\varphi: \mathbb{ R}^d \to \mathbb{ C}$$ is continuous with $$\varphi(0)=1$$, then the following are equivalent:
▻▻ $$\varphi$$ is the Fourier Transformation of some $$\mu \in \mathcal{ M}_1^+ (\mathbb{ R}^d )$$,
⇔ $$\varphi$$ is positive semi-definite, i.e. $$\sum\limits_{ k,l=1}^{ n} \lambda_k \bar\lambda_l \varphi(x_k-x_l)\geq 0$$ always.
Lause Explicit inversion formula. Let $$\mu \in \mathcal{ M}_1^+ (\mathbb{ R}^d ),\ F(b)=\mu((-\infty, b])$$, a density function, then:
(1) $$F(b)-F(a) = \lim\limits_{ c \to \infty} \frac{ 1}{ 2\pi} \displaystyle\int_{ -c}^c \frac{ e^{-iya}-e^{-iyb}}{ iy} \hat\mu(y) \ d\lambda(y)$$ ;
(2) if $$\displaystyle\int_{ \mathbb{ R} } |\hat\mu(x)| \ d\lambda(x) \lt \infty$$, then $$\mu$$ has a continuous density, i.e.
▻▻ $$\begin{cases} \mu(B) = \displaystyle\int_{ B} f(x)\ d\lambda(x) \\ f(x) =\frac{ 1}{ 2\pi} \displaystyle\int_{ \mathbb{ R} } e^{-ixy} \hat\mu(y) \ d\lambda(y) \end{cases}$$ .
Lause $$\mu(B)=\mu(-B),\ \forall B \in \mathcal{ B}(\mathbb{ R}) \ ⇔ \ \hat\mu(x)\in \mathbb{ R},\ \forall x \in \mathbb{ R}$$.
Lause Polya Let $$\varphi: \mathbb{ R}\to [0,\infty)$$,
▻▻ $$\varphi \text{ has } \begin{Bmatrix} \text{continuous}\\ \varphi(x)=\varphi(-x)\\ \text{convex on } [0,\infty) \\ \varphi(0)=1\\\lim\limits_{ x \to \infty}\varphi(x)=1 \end{Bmatrix}$$ ⇒ $$\exists\ \mu \in \mathcal{ M}_1^+(\mathbb{ R} ): \hat\mu(x)=\varphi(x)$$.

# Examples of Characteristic Functions

Lause For standard normal distribution $$\gamma(B):= \displaystyle\int_{ B} \frac{ 1}{ \sqrt{2\pi}} e^{- \frac{ x^2}{ 2} }\ d\lambda_1(x)$$ has $$\hat\gamma(x) = e^{- \frac{ x^2}{ 2} }$$.
▻▻ For $$Normal(m,\sigma^2) \ ⇒ \begin{cases} \gamma_{m,\sigma^2} \in \mathcal{ M}_1^+(\mathbb{ R} )\\ \displaystyle\int_{ \mathbb{ R} } x\ d\gamma_{m,\sigma^2}(x) = m\\ \displaystyle\int_{ \mathbb{ R} } (x-m)^2\ d\gamma_{m,\sigma^2}(x) = \sigma^2 \\ \hat\gamma_{m,\sigma^2}(x) = e^{imx}e^{-\frac{ 1}{ 2}\sigma^2 x^2 } \end{cases}$$ .
Määr Gaussian measure $$\mu = \delta_{\{m\}},\ m \in \mathbb{ R}$$ or $$\mu = \gamma_{m,\sigma^2}$$.
Määr Standard Gaussian Distribution on $$\mathbb{ R}^d:\ \gamma(B) := \displaystyle\int_{ B} e^{-\frac{ \langle x,x \rangle }{ 2} } \frac{ dx}{ \sqrt{2\pi}^d}$$.
Määr A matrix $$R = (r_{ij})^d_{i,j=1}$$ is called positive semi-definite provided that $$\langle Rx,x \rangle = \sum\limits_{ k,l=1}^{ n} r_{kl}x_k x_l \geq 0, \ \ \ \forall x \in \mathbb{ R}^d$$.
Lause Let $$\mu \in \mathcal{ M}^+_1 (\mathbb{ R}^d )$$. Then the following assertions are equivalent:
(1) There exist $$A = (a_{kl})^d_{k,l=1} \text{ and } m \in \mathbb{ R}^d$$, such that $$\mu(B)= \gamma^d \big( \big\{ x\in \mathbb{ R}^d: \Gamma x +m \in B \big\} \big)$$, i.e. $$\mu$$ is the image measure of $$\gamma ^ {(d)}$$;
(2) There exist a positive semi-definite and symmetric matrix $$R= (r_{kl})^d_{k,l=1} \text{ and } m’ \in \mathbb{ R}^d$$, such that $$\hat\mu (x) = e ^{i \langle x,m’ \rangle – \frac{ 1}{ 2} \langle Rx,x \rangle }$$;
(3) $$\forall b \in \mathbb{ R}^d$$ the law of $$\varphi_b: \mathbb{ R}^d \to \mathbb{ R},\ \varphi_b(x):= \langle x,b \rangle$$ is a Gaussian meausre on the real line .
▻▻ In particular, we have $$m, R, m’$$ are unique and
▻▻ $$\begin{cases} m=m’,\ R=AA^T \\ \displaystyle\int_{ \mathbb{ R}^d } x_k\ d\mu(x) = m_k \\ \displaystyle\int_{ \mathbb{ R}^d } (x_k-m_k)(x_l-m_l)\ d\mu(x) = r_{kl} \end{cases}$$.
Määr A Gaussian measure with mean $$m$$ and co-variance $$R$$ is  degenerated if $$rank(R)\lt d$$ or non-degenerated if $$rank(R)=d$$.
Lause For non-degenerate Gaussian measure $$\mu(B) = \displaystyle\int_{ B} e^{\frac{ 1}{ 2} \big\langle R^{-1} \langle x-m \rangle, x-m \big\rangle} \frac{ dx}{ (2\pi)^ \frac{ d}{ 2} |det R|^ \frac{ 1}{ 2} }$$.
Määr Cauchy distribution with parameter $$\alpha \gt 0:\ d\mu_\alpha (x) = \frac{ \alpha}{ \pi} \frac{ 1}{ \alpha^2 + x^2}dx \in \mathcal{ M}_1^+(\mathbb{ R}^d )$$.
Lause Cauchy distribution with parameter $$\alpha$$ has $$\widehat\mu_\alpha(x) = e ^{- \alpha |x|}$$.

# Independent Random Variables

Lause Basice properties of independence
(1) if $$f,g$$ are independent and $$\mathbb{ E}|f| \lt \infty, \mathbb{E} |g|\lt \infty \ ⇒ \mathbb{ E} fg = \mathbb{ E}f \mathbb{ E}g \text{ and } \mathbb{ E}|fg|\lt \infty$$;
(2) if $$f_i(\omega_1, \dots, \omega_n) := g_i(\omega_i)$$, then $$f_1, \dots, f_n$$ are independent and $$\text{law}(f_i) = \text{law}(g_i)$$.
Lause $$f ⫫ g \ ⇔\ \hat h(x,y) = \hat f(x) \hat g(y),\ \ h(\omega):=(f(\omega),g(\omega))$$ .
Esim $$f,g:\Omega \to \mathbb{ R}$$ are uncorrelated provided that $$\mathbb{ E}(f-\mathbb{ E}f )(g- \mathbb{ E}g ) =0$$.
Lause independent ⇒ uncorrelated.
Lause If $$f,g$$ joint distribution is Gaussian, then independent ⇔ uncorrelated.

# Moments of Measures

Määr Three types of moments of a measure:
▻▻ moment of order $$(l_1,\dots,l_d):\ \displaystyle\int_{ \mathbb{ R}^d } x_1^{l_1} \cdots x_d^{l_d}\ d\mu(x_1,\dots,x_d)$$;
▻▻ absolute moment of order $$(l_1,\dots,l_d):\ \displaystyle\int_{ \mathbb{ R}^d } |x_1^{l_1} \cdots x_d^{l_d}|\ d\mu(x_1,\dots,x_d)$$;
▻▻ centered absolute p-th moment $$\ \displaystyle\int_{ \mathbb{ R}^d } \bigg| x – \displaystyle\int_{ \mathbb{ R}^d } x\ d\mu(x) \bigg|^p \ d\mu(x)$$.
Lause Assume $$\forall l:\ \displaystyle\int_{ \mathbb{ R}^d } |x_1^{l_1} \cdots x_d^{l_d}|\ d\mu(x_1,\dots,x_d) \lt\infty$$, then:
(1) $$\frac{ \partial ^ {l_1+\cdots+l_d}}{ \partial x_1^{l_1} \cdots \partial x_d^{l_d}} \hat\mu\in C_b (\mathbb{ R}^d )$$ ;
(2) $$\frac{ \partial ^ {l_1+\cdots+l_d}}{ \partial x_1^{l_1} \cdots \partial x_d^{l_d}} \hat\mu(x) = i^{l_1+\cdots+l_d} \displaystyle\int_{ \mathbb{ R}^d } e^{i \langle x,y \rangle } y_1 ^{l_1} \cdots y_d^{l_d}\ d\mu(y)$$;
(3) $$\frac{ \partial ^ {l_1+\cdots+l_d}}{ \partial x_1^{l_1} \cdots \partial x_d^{l_d}} \hat\mu(0) = i^{l_1+\cdots+l_d} \displaystyle\int_{ \mathbb{ R}^d } y_1 ^{l_1} \cdots y_d^{l_d}\ d\mu(y)$$;
(4) the partial derivatives of $$\hat\mu$$ are uniformly continuous.
Lause $$\forall \alpha \gt 0:\ \displaystyle\int_{ \mathbb{ R}} |x|^k\ d\mu_\alpha (x) = \infty$$ .
Lause Let $$f: \mathbb{ R}\times \Omega \to \mathbb{ C}$$:
If $$f \text{ has } \begin{Bmatrix} \frac{ \partial f}{ \partial x}( \cdot,\omega) \text{ is continuous } \forall\omega \\ \frac{ \partial f}{ \partial x}( x,\cdot),\ f(x,\cdot) \text{ are random variables} \\ \exists g: |\frac{ \partial f}{ \partial x}( x,\omega)| \leq g(\omega),\ \forall \omega, x,\ \mathbb{ E}g\lt\infty \\ \displaystyle\int_{ \Omega} |f(x,\omega)|\ d \mathbb{ P}(\omega)\lt\infty,\ \forall x \end{Bmatrix}$$. Then,
▻▻ $$\displaystyle\frac{ \partial }{ \partial x} \displaystyle\int_{ \Omega} f ( x,\omega)\ d \mathbb{P}(\omega) = \displaystyle\int_{ \Omega} \frac{ \partial f}{ \partial x} ( x,\omega)\ d \mathbb{P}(\omega)$$.

# Weak Convergence

Lause Let $$\mu_n,\mu\in \mathcal{ M}_1^+(\mathbb{ R}^d )$$, the following assertions are equivalent:
\begin{align} \ &\forall\text{ continuous and bounded }\varphi: \mathbb{ R}^d\to \mathbb{ R}:\ \displaystyle\int \varphi(x)d\mu_n (x) \overset{n}{\to } \displaystyle\int_{ } \varphi(x)\ d\mu(x) \\ ⇔ \ & \forall \text{ closed sets } A\in \mathcal{ \mathbb{ R}^d}: \limsup\limits_n \mu_n (A) \leq \mu(A) \\⇔ \ & \forall \text{ open sets } A\in \mathcal{ \mathbb{ R}^d}: \liminf\limits_n \mu_n (A) \geq \mu(A) \\⇔ \ & F_n(x) \overset{n}{\to } F(x), \ \ F_i (x):=\mu_i ((-\infty,x])\\⇔ \ & \hat\mu_n(x) \overset{n}{\to } \hat\mu(x) \end{align}.
Määr The above is weak convergence.
▻▻ $$f_n$$ converges to $$f$$ weakly, i.e. $$f_n \overset{d}\to f$$ , if the corresponding laws $$\mu_n(B)= \mathbb{ P}_n(f_n\in B)$$ are converging weakly.
Lause $$f_n \overset{d}\to f \ ⇒ \ f_n \overset{ \mathbb{ P} }\to f$$.
Lause Central limit theorem Let $$f_1,\dots,f_n,\dots$$ sequence of independent random variables, such that $$\mathbb{ E}(f_k – \mathbb{ E}f_k )^2 = \sigma^2\lt\infty,\ \ \ \mathbb{ E}f_k=m$$. Then,
▻▻ $$\mathbb{ P} \bigg( \frac{ 1}{ \sqrt{k\sigma^2}} ((f_1-m) + \cdots + (f_k-m))\leq x \bigg) \overset{k}{\to } \frac{ 1}{ \sqrt{2\pi}} \displaystyle\int_{-\infty}^x e^{-\frac{ \xi^2}{ 2} }\ d\xi$$ .
Lause Convergence to Poisson: Let independent rvs with $$\mathbb{ P}(f_{n,k}=1) = p_{nk}, \ \mathbb{ P}(f_{n,k}=0) = q_{nk},\ \ p+q=1$$.
Assume that $$\max \limits_{ 1\leq k \leq n} p_{nk} \overset{n}{\to }0 \text{ and } \sum\limits_{ k=1}^{ n} p_{nk} \overset n \to \lambda$$. Then
▻▻ the law of sum $$\mu_n:= \text{law}(f_{n,1} + \cdots + f_{n,n}) \overset n\to \pi_\lambda (B) = \sum\limits_{ k=0}^{ \infty} e^{-\lambda} \frac{ \lambda^k}{ k!} \delta_{\{ k\}}(B)$$.
Lause Let indepdent rvs s.t. $$\mathbb{ E}f_{nk}=0,\ \sum\limits_{ k=1}^{ n} \mathbb{ E}f_{nk}^2 =1$$.
Assuume Lindeberg condition , i.e. $$\sum\limits_{ k=1}^{ n} \displaystyle\int_{ |x|\gt\varepsilon} x^2\ d\mu_{nk}(x) \overset n\to 0$$. Then,
▻▻ The sum $$S_n \overset d\to N(0,1)$$.

# An Ergodic Theorem

Määr  Measure preserving and ergodic map:
▻▻ $$\begin{cases} \text{measure preserving, if } \mathbb{ P}(T^{-1}(A)) = \mathbb{ P}(A),\ \ \forall A \in \mathcal{ F} \\ \text{ergodic measure preserving, if } T^{-1}(A)=A\ ⇒ \ \mathbb{ P}(A) \in \{0,1\},\ \ A\in \mathcal{ F}\end{cases}$$ .
Lause Ergodic Theorem of Birkhoff & Chincin Let $$T:\Omega\to \Omega$$ ergodic, and $$f:\Omega\to \mathbb{ R}$$ be a random variable with $$\mathbb{ E}f \lt\infty$$. Then:
▻▻ $$\lim\limits_{ n} \frac{ 1}{ n} \sum\limits_{ k=0}^{ n-1}f(T^k \omega) = \mathbb{ E}f$$ a.s.
Lause Let $$T:G\to G$$ a measure preserving map and $$f$$ integrable random variable.
▻▻ Let $$S_n:= f+ \cdots +f(T^{n-1}),\ M_n:= \max\{0,S_1,\dots,S_n\}$$. Then $$\mathbb{ E}(f_{\large\chi\normalsize\{M_n\gt 0\} })\geq 0$$.

# Almost Sure Convergence

Määr Let $$f_n, f:\Omega\to \mathbb{ R}$$ random variables. $$f_n$$  converges almost surely to $$f,\ \ \text{ i.e. } f_n \underset{a.s.}\longrightarrow f$$, if:
▻▻ $$\mathbb{ P}(\{\omega\in\Omega: |f_n(\omega)-f(\omega)|\overset n\to 0\}) = 1$$.
Lause $$f_n \underset{a.s.}\longrightarrow f\ \ ⇔ \ \lim\limits_{ n} \mathbb{ P}(\{ \omega\in\Omega: \sup\limits_{ k\geq n} |f_k(\omega)-f(\omega)|\gt\varepsilon\})=0,\ \forall \varepsilon\gt 0$$.
Lause Let $$f_1,f_2,\dots :\Omega\to \mathbb{ R}$$ a sequence of random variables. Then the following are equivalent:
\begin{align} & \mathbb{ P}(\{\omega\in\Omega: (f(\omega))_{n=1 }^\infty \text{ is a Cauchy sequence} \})=1 \\ \Leftrightarrow\ &\forall \varepsilon\gt0 : \lim\limits_{ n \to \infty} \mathbb{ P} \bigg( \sup\limits_{ k,l\geq n} |f_k-f_l| \geq\varepsilon \bigg) =0 \\ \Leftrightarrow\ &\forall \varepsilon\gt0 : \lim\limits_{ n \to \infty} \mathbb{ P} \bigg( \sup\limits_{ k\geq n} |f_k-f_n| \geq\varepsilon \bigg) =0 \end{align}.

# Convergence in Probability

Määr Ky Fan-metric $$d(f,g):=\inf\{\varepsilon\gt0: \mathbb{ P}(|f-g|\gt\varepsilon)\leq\varepsilon \}$$.
Huom $$\varepsilon\mapsto \mathbb{ P}(|f-g|\gt\varepsilon)$$ is right continuous. Therefore $$d(f,g)=d_0\ ⇒ \ \mathbb{ P}(|f-g|\gt d_0) \leq d_0$$ .
Lause $$d(f_n,f)\overset{n\to\infty}\longrightarrow 0 \ \ ⇔ \ \ \lim\limits_{ n } \mathbb{ P}(|f_n-f|\gt\varepsilon)=0,\ \forall \varepsilon\gt0$$.
Määr Let $$f_n,f\in \mathcal{ L}_0(\Omega, \mathcal{ f}, \mathbb{ P} )$$:
▻▻ Convergence in Probability $$f_n \underset{\mathbb{P}}\to f \text{ , if } \lim\limits_{ n}d(f_n,f)=0\\$$;
▻▻ $$(f_n)_{n=0}^\infty$$ Cauchy Sequence in Probability, provided that $$\forall \varepsilon\gt0,\ \exists n(\varepsilon)\geq 1:\ d(f_k,f_l)\leq\varepsilon,\ \forall k,l\geq n(\varepsilon)$$.
Lause For $$f,g,h \in \mathcal{ L}_0(\Omega, \mathcal{ f}, \mathbb{ P} )$$, the following holds:
(1) $$d(f,g)=0\ \ ⇔\ \ \mathbb{ P}(f=g)=1$$;
(2) $$d(f,h)\leq d(f,g)+d(g,h)$$;
(3) $$d(f,g)=d(g,f)$$ .
Lause Basic properties of the convergence in probability:
(1) $$f_n \underset {\mathbb{ P} }\to f\ \ ⇒ \ \ (f_n)_{n=0}^\infty$$ is Cauchy sequence in probability ;
(2) Uniqueness: $$f_n \underset {\mathbb{ P} }\to f \text{ and }f_n \underset {\mathbb{ P} }\to g \ \ ⇒ \ \ \mathbb{ P}(f=g)=1$$ ;
(3) Algebraic operation $$f_n \underset {\mathbb{ P} }\to f \text{ and }g_n \underset {\mathbb{ P} }\to g \ \ ⇒ \ \ \lambda f_n + \mu g_n \underset {\mathbb{ P} }\to \lambda f+ \mu g,\ f_n g_n \underset {\mathbb{ P} }\to fg$$ ;
(4) Completeness a Cauchy sequence in probability has a subsequence with $$f_n \underset {\mathbb{ P} }\to h,\ f_{n_k}\underset {\mathbb{ P} }\to h$$ .
Lause Almost surely convergence and convergence in probability
(1) $$f_n \underset {a.s. }\to f \ \ ⇒ \ \ f_n \underset {\mathbb{ P} }\to f$$ ;
(2) $$f_n \underset {\mathbb{ P} }\to f \ \ ⇒ \ \ \exists1\leq n_1 \leq n_2 \leq \cdots: f_{n_k} \underset {a.s. }\to f$$ ;
(3) $$f_n \underset {\mathbb{ P} }\to f \ \ ⇔ \ \ \forall \text{ subsequence } 1\leq n_1 \lt n_2\lt\cdots,\ \exists \text{ subsubsequence} 1\leq n_{n_1}\lt n_{n_2} \lt\cdots:\ f_{n_{k_l}} \underset {a.s.}\to f,\ l\to\infty$$ .

# Convergence in Mean

Määr  convergence to p-th mean, $$f_n \underset{L_p}\to f$$, provided that
▻▻ $$\lim\limits_{ n \to \infty} \displaystyle\int_{ \Omega} |f_n-f|^p \ d\mathbb{P}=0$$ .
Lause Let $$0\lt p\lt q\lt \infty$$:
(1) $$f_n \underset{L_p}\to f \ \ ⇒ \ \ f_n \underset{ \mathbb{ P} }\to f$$;
(2) $$f_n \underset{L_p}\to f,\ g_n \underset{L_p}\to g \ \ ⇒ \ \ f_n+g_n \underset{L_p}\to f+g$$ ;
(3) $$f_n \underset{L_q}\to f \ \ ⇒ \ \ f_n \underset{L_p}\to f$$ ;
(4) If $$f_n \underset{\mathbb{P} }\to f,\ \ \mathbb{ E}\sup |f_n|^p \lt\infty$$, then $$f\in \mathcal{ L}_p (\Omega, \mathcal{ F}, \mathbb{ P} ),\ \ \ \lim\limits_{ n } \mathbb{ E} |f_n-f|^p =0$$