Distributions
James Totterdell
2021-03-24
Source:vignettes/web_only/99-distributions.Rmd
99-distributions.Rmd
Introduction
This document provides a reference for relevant terms for useful distributions and their relationships in the context of variational inference.
Gamma and Inverse-Gamma
Gamma
y∼Gamma(α,β)f(y|α,β)=βαΓ(α)e−yβyα−1,y,α,β∈R+lnf(y|α,β)=αlnβ−lnΓ(α)−yβ+(α−1)lnyE[y]=αβV[y]=αβ2H[y]=α−lnβ+lnΓ(a)+(1−α)ψ(α)E[lny]=ψ(α)−ln(β)
Chi-squared, Inverse-Chi-squared, and Scaled-Chi-squared
Chi-squared
y∼Chi-squared(ν)f(y|ν)=12n/2Γ(n/2)e−ν/2yn/2−1,y,ν∈R+lnf(y|ν)=−(n/2)ln(2)−lnΓ(n/2)−ν/2+(n/2−1)ln(y)E[y]=νV[y]=2νH[y]=ψ(n/2)+ln(2)
Inverse-Chi-squared
y∼Inverse-Chi-squared(ν)f(y|ν)=2−ν/2Γ(ν/2)y−ν/2−1e−1/(2y),y,ν∈R+lnf(y|ν)=−(ν/2)ln(2)−lnΓ(ν/2)−(ν/2+1)ln(y)−1/(2y)E[y]=1ν−2,ν>2V[y]=2(ν−2)2(ν−4),ν>4H[y]=ν/2+ln(ν2Γ(ν/2))−(ν/2+1)ψ(ν/2)
Wishart, Inverse-Wishart, G-Wishart
Wishart
Σ∼Wishartd(ξ,Σ)f(Σ|ξ,λ)=12ξd/2|Λ|ξ/2Γd(ξ/2)|Σ|(ξ−d−1)/2e−tr(Λ−1Σ)/2,ξ>d−1,Λ>0lnf(Σ|ξ,λ)=−(ξd/2)−(ξ/2)ln|Λ|−lnΓd(ξ/2)+(ξ−d−1)/2ln|Σ|−tr(Λ−1Σ)/2E[Σ]=ξΛV[Σ]ij=H[Σ]=d+12ln|Λ|+d(d+1)2ln(2)+lnΓd(ξ/2)−ξ−d−12ψd(ξ/2)+ξd2E[ln|Σ|]=ψd(ξ/2)+dln(2)+ln|Λ|
Inverse-Wishart
If Σ∼Wishartd(ξ,Λ) then Σ−1∼Inverse-Wishartd(ξ,Λ−1)
Σ∼Inverse-Wishartd(ξ,Σ)f(Σ|ξ,Λ)=|Λ|ξ/22ξd/2Γd(ξ/2)|Σ|−(ξ+d+1)/2e−tr(ΛΣ−1)/2,ξ>d−1,Λ>0lnf(Σ|ξ,Λ)=ξ/2ln|Λ|−(ξd)/2ln(2)−lnΓd(ξ/2)−(ξ+d+1)/2ln|Σ|−tr(ΛΣ−1)E[Σ]=Λξ−d−1E[Σ−1]=ξΛ−1V[Σ]ij=(ξ−d+1)λ2ij+(ξ−d−1)λiiλjj(ξ−d)(ξ−d−1)2(ξ−d−3)H[Σ]=−ξ2ln|Λ|+(ξ+d+1)E[ln|Σ|]+ξd2ln(2)+lnΓd(ξ/2)+ξd2E[ln|Σ|]=ln|12Λ|−ψd(ξ−d+1)