离散分布
Bernoulli-伯努利分布
\[ \begin{align} X&\sim Bernoulli(p)\\ &P(x)=p^x(1-p)^{1-x},\quad x=0,1\\ &\mu_x=p,\quad \sigma_x^2=p(1-p)\\ &M_x(t)=pe^t+1-p\\ &\phi_x(t)=pe^{it}+1-p \end{align} \]
Binomial-二项分布
\[ \begin{align} X&\sim Bin(n,p)\\ &P(x)=\bigl( \begin{smallmatrix}n \\ x \end{smallmatrix}\bigr) p^x(1-p)^{n-x},\quad x=0,1,...,n\\ &\mu_x=np,\quad \sigma_x^2=np(1-p)\\ &M_x(t)=(pe^t+1-p)^n\\ &\phi_x(t)=(pe^{it}+1-p)^n \end{align} \]
Poisson-泊松分布
\[ \begin{align} X&\sim Poisson (\lambda)\\ &P(x)=\frac{e^{-\lambda}\lambda^x}{x!},\quad x=0,1,2,...\\ &\mu_x=\lambda,\quad \sigma_x^2=\lambda\\ &M_x(t)=e^{\lambda(e^t-1)}\\ &\phi_x(t)=e^{\lambda(e^{it}-1)} \end{align} \]
一段时间内出现的次数
Negative Binomial-负二项分布
\[ \begin{align} X&\sim NB(r,p)\\ &P(x)=\bigl( \begin{smallmatrix}x+r-1 \\ r-1 \end{smallmatrix}\bigr) p^r(1-p)^x,\quad x=0,1,2,...\\ &\mu_x=\frac{r(1-p)}{p},\quad \sigma_x^2=\frac{r(1-p)}{p^2}\\ &M_x(t)=(\frac{p}{1-(1-p)e^t})^r\\ \end{align} \]
成功概率是\(p\)，\(r\)次成功会出现多少次失败
Geometry-几何分布
\[ \begin{align} X&\sim Geometry(p)\\ &P(x)=p(1-p)^{x-1},\quad x=1,2,...\\ &\mu_x=\frac1{p},\quad \sigma_x^2=\frac{1-p}{p^2}\\ &M_x(t)=\frac{pe^t}{1-(1-p)e^t},\quad t < -\ln(1-p)\\ \end{align} \]
成功概率是\(p\)，出现一次成功总共要多少次
Hypergeometric-超几何分布
\[ \begin{align} X&\sim Hyper(M,N,n)\\ &P(x)=\frac{ \bigl( \begin{smallmatrix}M \\ x \end{smallmatrix}\bigr) \bigl( \begin{smallmatrix}N-M \\ n-x \end{smallmatrix}\bigr) } {\bigl( \begin{smallmatrix}N \\ n \end{smallmatrix}\bigr)}\\ &\mu_x=\frac{nM}{N},\quad \sigma_x^2=\frac{nM}{N}\frac{N-M}{N}\frac{N-n}{N-1} \end{align} \]
总共有\(N\)条鱼，\(M\)条带标记，某一次捞\(n\)条，带标记的有几条
Multinomial-多项分布
\[ \begin{align} X_1,&X_2,...,X_k\sim MultiNomial(p_1,...,p_k,n)\\ &P(x)=\frac{n!}{x_1!x_2!...x_k!}p_1^{x_1}p_2^{x_2}...p_k^{x_k}\\ &x_i\in\{0,...,n\},i\in\{1,...,k\},\sum x_i=n\\ &E(X_i)=np_i,\quad Var(X_i)=np_i(1-p_i) \end{align} \]
\(k\)种取值，掷n次，\(x_i\)表示每种发生几次
\(n=1\)时，\(x\)可以取\(1,...,k\)这几个值，\(P(x)\)表示取这些值的概率
\(n=1\)时,\(P(x)=p_1^{1\{x=1\}}p_2^{1\{x=2\}}...p_k^{1\{x=k\}}\)
连续分布
Uniform分布
\[ \begin{align} X&\sim Uniform(a, b)\\ &f_x(x)=\frac1{b-a},\quad x\in[a,b]\\ &\mu_x=\frac{a+b}{2},\quad \sigma_x^2=\frac{(b-a)^2}{12}\\ &M_x(t)=\frac{e^{tb}-e^{ta}}{t(b-a)}\\ &\phi_x(t)=\frac{e^{itb}-e^{ita}}{it(b-a)}\\ U(0,1)&\rightarrow a=0,b=1 \end{align} \]
Normal-正态分布
\[ \begin{align} X&\sim N(\mu, \sigma^2)\\ &f_x(x)=\frac1{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^2}{2\sigma^2}},\quad -\infty< x<+\infty\\ &\mu_x=\mu,\quad \sigma_x^2=\sigma^2\\ &M_x(t)=e^{\mu t+\frac12\sigma^2t^2}\\ &\phi_x(t)=e^{i\mu t-\frac12\sigma^2t^2}\\ \int_{-\infty}^{\infty}&e^{-\frac{x^2}{2}}dx=\sqrt{2\pi} \end{align} \]
Gamma分布
\[ \begin{align} X&\sim Gamma(\alpha, \beta)\\ &f_x(x)=\frac{x^{\alpha-1}e^{-\frac{x}{\beta}}} {\Gamma(\alpha)\beta^\alpha} ,\quad x>0\\ &\mu_x=\alpha\beta,\quad \sigma_x^2=\alpha\beta^2\\ &M_x(t)=(1-\beta t)^{-\alpha}, \quad t<\frac1\beta\\ &\phi_x(t)=(1-\beta it)^{-\alpha}\\ \Gamma(\alpha)&=\int_0^\infty t^{\alpha-1}e^{-t}dt=(\alpha-1)\Gamma(\alpha-1)=(\alpha-1)!\\ \Gamma(\frac12)&=\sqrt{\pi} \end{align} \]
再出现\(\alpha\)个要等多久。
Chi-square-卡方分布
\[ \begin{align} X&\sim \chi^2_v\\ &f_x(x)=\frac{x^{\frac v2-1}e^{-\frac{x}{2}}} {\Gamma(\frac v2)\sqrt{2^v}} ,\quad x>0\\ &\mu_x=v,\quad \sigma_x^2=2v\\ &M_x(t)=(1-2 t)^{-\frac v2}, \quad t<\frac12\\ &\phi_x(t)=(1-2 it)^{-\frac v2}\\ Gamma&特例\rightarrow \alpha=\frac v2,\beta=2 \end{align} \]
Exponential-指数分布
\[ \begin{align} X&\sim exp(\beta)\\ &f_x(x)=\frac1\beta e^{-\frac x\beta},\quad x>0\\ &\mu_x=\beta,\quad \sigma_x^2=\beta^2\\ &M_x(t)=\frac1{1-\beta t},t<\frac1\beta\\ &\phi_x(t)=\frac1{1-\beta it}\\ Gamma&特例\rightarrow \alpha=1\\ Poisson&\rightarrow \lambda=\frac1\beta \end{align} \]
出现下一个要等多久。
Beta分布
\[ \begin{align} X&\sim Beta(\alpha,\beta)\\ &f_x(x)=\frac1{B(\alpha,\beta)}x^{\alpha-1}(1-x)^{\beta-1} ,\quad 0\leq x\leq 1,\alpha>0,\beta>0\\ &\mu_x=\frac{\alpha}{\alpha+\beta},\quad \sigma_x^2=\frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)}\\ &M_x(t)=1+\sum_{j=1}^n(\prod_{i=0}^{j-1}\frac{\alpha+i}{\alpha+\beta+i})\frac{t^j}{j!}, \quad t<\frac12\\ B&(\alpha, \beta)=\frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha+\beta)}\\ U&(0,1)\rightarrow \alpha=1,\beta=1\\ \int_0^1&x^{\alpha-1}(1-x)^{\beta-1}=B(\alpha, \beta) \end{align} \]
\(X_1, X_2, ..., X_n\)是均匀分布\(U(0,1)\)产生的随机数，其中第\(k\)小的数\(X^{(k)}\)服从\(Beta(k, n-k+1)\)的Beta分布。
Log-normal分布
\[ \begin{align} X&\sim LogN(\mu,\sigma^2)\\ &f_x(x)=\frac1{\sqrt{2\pi}\sigma}\frac1x exp\{-\frac1{2\sigma^2}(\log x-\mu)^2\} ,\quad x>0\\ Normal&\rightarrow Y\sim N(\mu,\sigma^2),\quad X=e^Y\sim LogN(\mu,\sigma^2)\\ E(X^k)=&E(e^{kY})=M_Y(k)=exp\{\frac{\sigma^2k^2}{2}+\mu k\} \end{align} \]
Double/Laplace exponential-拉普拉斯分布
\[ \begin{align} X&\sim Laplace(\alpha,\beta)\\ &f_x(x)=\frac1{2\beta}exp\{-\frac{|x-\alpha|}{\beta}\} ,\quad -\infty < x < +\infty\\ &\mu_x=\alpha,\quad \sigma_x^2=2\beta^2\\ &M_x(t)=\frac{e^{\alpha t}}{1-\beta^2 t^2}, \quad |t|<\frac1\beta\\ 厚尾&，关于\alpha对称 \end{align} \]
Cauchy-柯西分布
\[ \begin{align} X&\sim Cauchy(\mu,\sigma)\\ &f_x(x)=\frac1{\pi\sigma}\frac1{1+(\frac{x-\mu}{\sigma})^2} ,\quad -\infty < x < +\infty\\ E(x^k)&不存在,k\geq 1，MGF不存在\\ \phi_x(t)&=e^{iut-\sigma|t|},对t不可导 \end{align} \]
IGamma-逆Gamma分布
\[ \begin{align} X&\sim IGamma(\alpha, \beta)\\ &f_x(x)=\frac{\beta^\alpha}{\Gamma(\alpha)}(\frac1y)^{\alpha+1}e^{-\frac\beta y},\quad x>0\\ Gamma&\rightarrow Y\sim Gamma(\alpha, \beta),\quad X=Y^{-1}\sim IGamma(\alpha, \beta)\\ \end{align} \]
其他
exponential family-指数族
\[ p(y;\eta)=b(y)exp(\eta^TT(y)-a(\eta)) \]
共轭先验分布
- \(Poisson(\lambda)\rightarrow\lambda\Rightarrow Gamma(\alpha, \beta)\rightarrow Gamma(\alpha+n\bar X, \beta+n)\)
- \(exp(\beta)\rightarrow\lambda\Rightarrow IGamma(\alpha, \beta)\rightarrow Gamma(\alpha+n, \beta+n\bar X)\)
- \(Bin(n,p)\rightarrow p\Rightarrow Beta\)
- \(Normal\)方差已知时\(\mu\Rightarrow Normal\)，均值已知时\(\sigma^2\Rightarrow IGamma\)，都未知\(\mu,\sigma^2\Rightarrow N-IGamma\)