Loading [MathJax]/jax/output/HTML-CSS/jax.js
Skip to contents

Introduction

This document provides a reference for relevant terms for useful distributions and their relationships in the context of variational inference.

Gamma and Inverse-Gamma

Gamma

yGamma(α,β)f(y|α,β)=βαΓ(α)eyβyα1,y,α,βR+lnf(y|α,β)=αlnβlnΓ(α)yβ+(α1)lnyE[y]=αβV[y]=αβ2H[y]=αlnβ+lnΓ(a)+(1α)ψ(α)E[lny]=ψ(α)ln(β)

Inverse-Gamma

If then y1Inverse-Gamma(α,β).

yInverse-Gamma(α,β)f(y|α,β)=βαΓ(α)ey/βyα1,y,α,βR+lnf(y|α,β)=αlnβlnΓ(α)y/β(α+1)ln(y)E[y]=βα1,α>1E[y1]=αβV[y]=β2(α1)2(α2),α>2H[y]=α+lnβ+lnΓ(a)(1+α)ψ(α)E[lny]=lnβψ(α)

Relations

YGamma(α,β)YInv-Gamma(α,β)YGamma(ν/2,1/2)YChi-square(ν)YInv-Gamma(α,1/2)YInv-Chi-square(2α)YInv-Gamma(α,β)YInv-Wishart(2α,2β)

Y2|XInv-Gamma(ν/2,ν/X)XInv-Gamma(1/2,1/A2)YHalf-t(ν,A)Y2|XInv-Gamma(1/2,1/X)XInv-Gamma(1/2,1/A2)YHalf-Cauchy(A)

Chi-squared, Inverse-Chi-squared, and Scaled-Chi-squared

Chi-squared

yChi-squared(ν)f(y|ν)=12n/2Γ(n/2)eν/2yn/21,y,νR+lnf(y|ν)=(n/2)ln(2)lnΓ(n/2)ν/2+(n/21)ln(y)E[y]=νV[y]=2νH[y]=ψ(n/2)+ln(2)

Inverse-Chi-squared

yInverse-Chi-squared(ν)f(y|ν)=2ν/2Γ(ν/2)yν/21e1/(2y),y,νR+lnf(y|ν)=(ν/2)ln(2)lnΓ(ν/2)(ν/2+1)ln(y)1/(2y)E[y]=1ν2,ν>2V[y]=2(ν2)2(ν4),ν>4H[y]=ν/2+ln(ν2Γ(ν/2))(ν/2+1)ψ(ν/2)

Scale-inverse-Chi-squared

yScaled-inverse-Chi-squared(ν,τ2)f(y|ν,τ2)=(τ2ν/2)ν/2Γ(ν/2)exp(ντ22y)y1+ν/2lnf(y,τ2)=(ν/2)ln(ντ2/2)lnΓ(ν/2)ντ22y(1+ν/2)ln(y)E[y]=ντ2ν2,ν>2V[y]=2ν2τ4(ν2)2(ν4),ν>4H[y]=ν/2+ln(ντ22Γ(ν/2))(1+ν/2)ψ(ν/2)

Relations

Chi-square(ν)Gamma(ν/2,2)Inverse-Chi-Squared(ν)Scale-Inverse-Chi-Squared(ν,1/ν)Scale-Inverse-Chi-Squared(ν,τ2)Inverse-Gamma(ν/2,ντ2/2)

Wishart, Inverse-Wishart, G-Wishart

Wishart

ΣWishartd(ξ,Σ)f(Σ|ξ,λ)=12ξd/2|Λ|ξ/2Γd(ξ/2)|Σ|(ξd1)/2etr(Λ1Σ)/2,ξ>d1,Λ>0lnf(Σ|ξ,λ)=(ξd/2)(ξ/2)ln|Λ|lnΓd(ξ/2)+(ξd1)/2ln|Σ|tr(Λ1Σ)/2E[Σ]=ξΛV[Σ]ij=H[Σ]=d+12ln|Λ|+d(d+1)2ln(2)+lnΓd(ξ/2)ξd12ψd(ξ/2)+ξd2E[ln|Σ|]=ψd(ξ/2)+dln(2)+ln|Λ|

Inverse-Wishart

If ΣWishartd(ξ,Λ) then Σ1Inverse-Wishartd(ξ,Λ1)

ΣInverse-Wishartd(ξ,Σ)f(Σ|ξ,Λ)=|Λ|ξ/22ξd/2Γd(ξ/2)|Σ|(ξ+d+1)/2etr(ΛΣ1)/2,ξ>d1,Λ>0lnf(Σ|ξ,Λ)=ξ/2ln|Λ|(ξd)/2ln(2)lnΓd(ξ/2)(ξ+d+1)/2ln|Σ|tr(ΛΣ1)E[Σ]=Λξd1E[Σ1]=ξΛ1V[Σ]ij=(ξd+1)λ2ij+(ξd1)λiiλjj(ξd)(ξd1)2(ξd3)H[Σ]=ξ2ln|Λ|+(ξ+d+1)E[ln|Σ|]+ξd2ln(2)+lnΓd(ξ/2)+ξd2E[ln|Σ|]=ln|12Λ|ψd(ξd+1)

Relations

Wishart1(ξ,Λ)Gamma(ξ/2,Λ/2)inv-Wishart1(ξ,Λ)Inv-Gamma(ξ/2,Λ/2)

Σ|X1,...,XpInv-Wishartp(ν+p1,2νdiag(1/X1,...,1/Xp))XjindInv-Gamma(1/2,1/A2j)σjHalf-t(ν,Aj)ρij(1ρ2ij)ν/21 where Σij=ρijσiσj

Identities and Definitions

Γd(x)=multivariate gamma function(x)=πd(d1)/4dj=1Γ[x+(1j)/2]ψd(x)=multivariate digamma function(x)=dj=1ψ[x+(1j)/2]

(bdiag(A1,...,Ap))1=bdiag(A11,...,A1p)

E[tr(AX)]=tr(AE[X])