In probability and statistics, a natural exponential family (NEF) is a class of probability distributions that is a special case of an exponential family (EF).
The natural exponential families (NEF) are a subset of the exponential families. A NEF is an exponential family in which the natural parameter η and the natural statistic T(x) are both the identity. A distribution in an exponential family with parameter θ can be written with probability density function (PDF) where and are known functions. A distribution in a natural exponential family with parameter θ can thus be written with PDF [Note that slightly different notation is used by the originator of the NEF, Carl Morris. [1] Morris uses ω instead of η and ψ instead of A.]
Suppose that , then a natural exponential family of order p has density or mass function of the form: where in this case the parameter
A member of a natural exponential family has moment generating function (MGF) of the form
The cumulant generating function is by definition the logarithm of the MGF, so it is
The five most important univariate cases are:
These five examples – Poisson, binomial, negative binomial, normal, and gamma – are a special subset of NEF, called NEF with quadratic variance function (NEF-QVF) because the variance can be written as a quadratic function of the mean. NEF-QVF are discussed below.
Distributions such as the exponential, Bernoulli, and geometric distributions are special cases of the above five distributions. For example, the Bernoulli distribution is a binomial distribution with n = 1 trial, the exponential distribution is a gamma distribution with shape parameter α = 1 (or k = 1 ), and the geometric distribution is a special case of the negative binomial distribution.
Some exponential family distributions are not NEF. The lognormal and Beta distribution are in the exponential family, but not the natural exponential family. The gamma distribution with two parameters is an exponential family but not a NEF and the chi-squared distribution is a special case of the gamma distribution with fixed scale parameter, and thus is also an exponential family but not a NEF (note that only a gamma distribution with fixed shape parameter is a NEF).
The inverse Gaussian distribution is a NEF with a cubic variance function.
The parameterization of most of the above distributions has been written differently from the parameterization commonly used in textbooks and the above linked pages. For example, the above parameterization differs from the parameterization in the linked article in the Poisson case. The two parameterizations are related by , where λ is the mean parameter, and so that the density may be written as for , so
This alternative parameterization can greatly simplify calculations in mathematical statistics. For example, in Bayesian inference, a posterior probability distribution is calculated as the product of two distributions. Normally this calculation requires writing out the probability distribution functions (PDF) and integrating; with the above parameterization, however, that calculation can be avoided. Instead, relationships between distributions can be abstracted due to the properties of the NEF described below.
An example of the multivariate case is the multinomial distribution with known number of trials.
The properties of the natural exponential family can be used to simplify calculations involving these distributions.
In the multivariate case, the mean vector and covariance matrix are[ citation needed] where is the gradient and is the Hessian matrix.
A special case of the natural exponential families are those with quadratic variance functions. Six NEFs have quadratic variance functions (QVF) in which the variance of the distribution can be written as a quadratic function of the mean. These are called NEF-QVF. The properties of these distributions were first described by Carl Morris. [3]
The six NEF-QVF are written here in increasing complexity of the relationship between variance and mean.
The properties of NEF-QVF can simplify calculations that use these distributions.
Given independent identically distributed (iid) with distribution from a NEF-QVF. A convolution of a linear transformation of an NEF-QVF is also an NEF-QVF.
Let be the convolution of a linear transformation of X. The mean of Y is . The variance of Y can be written in terms of the variance function of the original NEF-QVF. If the original NEF-QVF had variance function then the new NEF-QVF has variance function where
This article relies largely or entirely on a
single source. (June 2012) |
This article needs additional citations for
verification. (June 2012) |
In probability and statistics, a natural exponential family (NEF) is a class of probability distributions that is a special case of an exponential family (EF).
The natural exponential families (NEF) are a subset of the exponential families. A NEF is an exponential family in which the natural parameter η and the natural statistic T(x) are both the identity. A distribution in an exponential family with parameter θ can be written with probability density function (PDF) where and are known functions. A distribution in a natural exponential family with parameter θ can thus be written with PDF [Note that slightly different notation is used by the originator of the NEF, Carl Morris. [1] Morris uses ω instead of η and ψ instead of A.]
Suppose that , then a natural exponential family of order p has density or mass function of the form: where in this case the parameter
A member of a natural exponential family has moment generating function (MGF) of the form
The cumulant generating function is by definition the logarithm of the MGF, so it is
The five most important univariate cases are:
These five examples – Poisson, binomial, negative binomial, normal, and gamma – are a special subset of NEF, called NEF with quadratic variance function (NEF-QVF) because the variance can be written as a quadratic function of the mean. NEF-QVF are discussed below.
Distributions such as the exponential, Bernoulli, and geometric distributions are special cases of the above five distributions. For example, the Bernoulli distribution is a binomial distribution with n = 1 trial, the exponential distribution is a gamma distribution with shape parameter α = 1 (or k = 1 ), and the geometric distribution is a special case of the negative binomial distribution.
Some exponential family distributions are not NEF. The lognormal and Beta distribution are in the exponential family, but not the natural exponential family. The gamma distribution with two parameters is an exponential family but not a NEF and the chi-squared distribution is a special case of the gamma distribution with fixed scale parameter, and thus is also an exponential family but not a NEF (note that only a gamma distribution with fixed shape parameter is a NEF).
The inverse Gaussian distribution is a NEF with a cubic variance function.
The parameterization of most of the above distributions has been written differently from the parameterization commonly used in textbooks and the above linked pages. For example, the above parameterization differs from the parameterization in the linked article in the Poisson case. The two parameterizations are related by , where λ is the mean parameter, and so that the density may be written as for , so
This alternative parameterization can greatly simplify calculations in mathematical statistics. For example, in Bayesian inference, a posterior probability distribution is calculated as the product of two distributions. Normally this calculation requires writing out the probability distribution functions (PDF) and integrating; with the above parameterization, however, that calculation can be avoided. Instead, relationships between distributions can be abstracted due to the properties of the NEF described below.
An example of the multivariate case is the multinomial distribution with known number of trials.
The properties of the natural exponential family can be used to simplify calculations involving these distributions.
In the multivariate case, the mean vector and covariance matrix are[ citation needed] where is the gradient and is the Hessian matrix.
A special case of the natural exponential families are those with quadratic variance functions. Six NEFs have quadratic variance functions (QVF) in which the variance of the distribution can be written as a quadratic function of the mean. These are called NEF-QVF. The properties of these distributions were first described by Carl Morris. [3]
The six NEF-QVF are written here in increasing complexity of the relationship between variance and mean.
The properties of NEF-QVF can simplify calculations that use these distributions.
Given independent identically distributed (iid) with distribution from a NEF-QVF. A convolution of a linear transformation of an NEF-QVF is also an NEF-QVF.
Let be the convolution of a linear transformation of X. The mean of Y is . The variance of Y can be written in terms of the variance function of the original NEF-QVF. If the original NEF-QVF had variance function then the new NEF-QVF has variance function where
This article relies largely or entirely on a
single source. (June 2012) |
This article needs additional citations for
verification. (June 2012) |