Is an approximation analogous to Fisher's argument necessary to solve the BehrensâFisher problem?
In statistics, the BehrensâFisher problem, named after Walter-Ulrich Behrens and Ronald Fisher, is the problem of interval estimation and hypothesis testing concerning the difference between the means of two normally distributed populations when the variances of the two populations are not assumed to be equal, based on two independent samples.
One difficulty with discussing the BehrensâFisher problem and proposed solutions, is that there are many different interpretations of what is meant by "the BehrensâFisher problem". These differences involve not only what is counted as being a relevant solution, but even the basic statement of the context being considered.
Let X1, ..., Xn and Y1, ..., Ym be i.i.d. samples from two populations which both come from the same locationâscale family of distributions. The scale parameters are assumed to be unknown and not necessarily equal, and the problem is to assess whether the location parameters can reasonably be treated as equal. Lehmann [1] states that "the BehrensâFisher problem" is used both for this general form of model when the family of distributions is arbitrary, and for when the restriction to a normal distribution is made. While Lehmann discusses a number of approaches to the more general problem, mainly based on nonparametrics, [2] most other sources appear to use "the BehrensâFisher problem" to refer only to the case where the distribution is assumed to be normal: most of this article makes this assumption.
Solutions to the BehrensâFisher problem have been presented that make use of either a classical or a Bayesian inference point of view and either solution would be notionally invalid judged from the other point of view. If consideration is restricted to classical statistical inference only, it is possible to seek solutions to the inference problem that are simple to apply in a practical sense, giving preference to this simplicity over any inaccuracy in the corresponding probability statements. Where exactness of the significance levels of statistical tests is required, there may be an additional requirement that the procedure should make maximum use of the statistical information in the dataset. It is well known that an exact test can be gained by randomly discarding data from the larger dataset until the sample sizes are equal, assembling data in pairs and taking differences, and then using an ordinary t-test to test for the mean-difference being zero: clearly this would not be "optimal" in any sense.
The task of specifying interval estimates for this problem is one where a frequentist approach fails to provide an exact solution, although some approximations are available. Standard Bayesian approaches also fail to provide an answer that can be expressed as straightforward simple formulae, but modern computational methods of Bayesian analysis do allow essentially exact solutions to be found.[ citation needed] Thus study of the problem can be used to elucidate the differences between the frequentist and Bayesian approaches to interval estimation.
Ronald Fisher in 1935 introduced fiducial inference [3] [4] in order to apply it to this problem. He referred to an earlier paper by Walter-Ulrich Behrens from 1929. Behrens and Fisher proposed to find the probability distribution of
where and are the two sample means, and s1 and s2 are their standard deviations. See BehrensâFisher distribution. Fisher approximated the distribution of this by ignoring the random variation of the relative sizes of the standard deviations,
Fisher's solution provoked controversy because it did not have the property that the hypothesis of equal means would be rejected with probability Îą if the means were in fact equal. Many other methods of treating the problem have been proposed since, and the effect on the resulting confidence intervals have been investigated. [5]
A widely used method is that of B. L. Welch, [6] who, like Fisher, was at University College London. The variance of the mean difference
results in
Welch (1938) approximated the distribution of by the Type III Pearson distribution (a scaled chi-squared distribution) whose first two moments agree with that of . This applies to the following number of degrees of freedom (d.f.), which is generally non-integer:
Under the null hypothesis of equal expectations, Îź1 = Îź2, the distribution of the BehrensâFisher statistic T, which also depends on the variance ratio Ď12/Ď22, could now be approximated by Student's t distribution with these ν degrees of freedom. But this ν contains the population variances Ďi2, and these are unknown. The following estimate only replaces the population variances by the sample variances:
This is a random variable. A t distribution with a random number of degrees of freedom does not exist. Nevertheless, the BehrensâFisher T can be compared with a corresponding quantile of Student's t distribution with these estimated numbers of degrees of freedom, , which is generally non-integer. In this way, the boundary between acceptance and rejection region of the test statistic T is calculated based on the empirical variances si2, in a way that is a smooth function of these.
This method also does not give exactly the nominal rate, but is generally not too far off.[ citation needed] However, if the population variances are equal, or if the samples are rather small and the population variances can be assumed to be approximately equal, it is more accurate to use Student's t-test.[ citation needed]
A number of different approaches to the general problem have been proposed, some of which claim to "solve" some version of the problem. Among these are, [7]
In Dudewiczâs comparison of selected methods, [7] it was found that the DudewiczâAhmed procedure is recommended for practical use.
For several decades, it was commonly believed that no exact solution to the common BehrensâFisher problem existed.[ citation needed] However, it was proved in 1966 that it has an exact solution. [12] In 2018 the probability density function of a generalized BehrensâFisher distribution of m means and m distinct standard errors from m samples of distinct sizes from independent normal distributions with distinct means and variances was proved and the paper also examined its asymptotic approximations. [13] A follow-up paper showed that the classic paired t-test is a central BehrensâFisher problem with a non-zero population correlation coefficient and derived its corresponding probability density function by solving its associated non-central BehrensâFisher problem with a nonzero population correlation coefficient. [14] It also solved a more general non-central BehrensâFisher problem with a non-zero population correlation coefficient in the appendix. [14]
A minor variant of the BehrensâFisher problem has been studied. [15] In this instance the problem is, assuming that the two population-means are in fact the same, to make inferences about the common mean: for example, one could require a confidence interval for the common mean.
One generalisation of the problem involves multivariate normal distributions with unknown covariance matrices, and is known as the multivariate BehrensâFisher problem. [16]
The nonparametric BehrensâFisher problem does not assume that the distributions are normal. [17] [18] Tests include the Cucconi test of 1968 and the Lepage test of 1971.
![]() | This article includes a list of general
references, but it lacks sufficient corresponding
inline citations. (February 2010) |
Is an approximation analogous to Fisher's argument necessary to solve the BehrensâFisher problem?
In statistics, the BehrensâFisher problem, named after Walter-Ulrich Behrens and Ronald Fisher, is the problem of interval estimation and hypothesis testing concerning the difference between the means of two normally distributed populations when the variances of the two populations are not assumed to be equal, based on two independent samples.
One difficulty with discussing the BehrensâFisher problem and proposed solutions, is that there are many different interpretations of what is meant by "the BehrensâFisher problem". These differences involve not only what is counted as being a relevant solution, but even the basic statement of the context being considered.
Let X1, ..., Xn and Y1, ..., Ym be i.i.d. samples from two populations which both come from the same locationâscale family of distributions. The scale parameters are assumed to be unknown and not necessarily equal, and the problem is to assess whether the location parameters can reasonably be treated as equal. Lehmann [1] states that "the BehrensâFisher problem" is used both for this general form of model when the family of distributions is arbitrary, and for when the restriction to a normal distribution is made. While Lehmann discusses a number of approaches to the more general problem, mainly based on nonparametrics, [2] most other sources appear to use "the BehrensâFisher problem" to refer only to the case where the distribution is assumed to be normal: most of this article makes this assumption.
Solutions to the BehrensâFisher problem have been presented that make use of either a classical or a Bayesian inference point of view and either solution would be notionally invalid judged from the other point of view. If consideration is restricted to classical statistical inference only, it is possible to seek solutions to the inference problem that are simple to apply in a practical sense, giving preference to this simplicity over any inaccuracy in the corresponding probability statements. Where exactness of the significance levels of statistical tests is required, there may be an additional requirement that the procedure should make maximum use of the statistical information in the dataset. It is well known that an exact test can be gained by randomly discarding data from the larger dataset until the sample sizes are equal, assembling data in pairs and taking differences, and then using an ordinary t-test to test for the mean-difference being zero: clearly this would not be "optimal" in any sense.
The task of specifying interval estimates for this problem is one where a frequentist approach fails to provide an exact solution, although some approximations are available. Standard Bayesian approaches also fail to provide an answer that can be expressed as straightforward simple formulae, but modern computational methods of Bayesian analysis do allow essentially exact solutions to be found.[ citation needed] Thus study of the problem can be used to elucidate the differences between the frequentist and Bayesian approaches to interval estimation.
Ronald Fisher in 1935 introduced fiducial inference [3] [4] in order to apply it to this problem. He referred to an earlier paper by Walter-Ulrich Behrens from 1929. Behrens and Fisher proposed to find the probability distribution of
where and are the two sample means, and s1 and s2 are their standard deviations. See BehrensâFisher distribution. Fisher approximated the distribution of this by ignoring the random variation of the relative sizes of the standard deviations,
Fisher's solution provoked controversy because it did not have the property that the hypothesis of equal means would be rejected with probability Îą if the means were in fact equal. Many other methods of treating the problem have been proposed since, and the effect on the resulting confidence intervals have been investigated. [5]
A widely used method is that of B. L. Welch, [6] who, like Fisher, was at University College London. The variance of the mean difference
results in
Welch (1938) approximated the distribution of by the Type III Pearson distribution (a scaled chi-squared distribution) whose first two moments agree with that of . This applies to the following number of degrees of freedom (d.f.), which is generally non-integer:
Under the null hypothesis of equal expectations, Îź1 = Îź2, the distribution of the BehrensâFisher statistic T, which also depends on the variance ratio Ď12/Ď22, could now be approximated by Student's t distribution with these ν degrees of freedom. But this ν contains the population variances Ďi2, and these are unknown. The following estimate only replaces the population variances by the sample variances:
This is a random variable. A t distribution with a random number of degrees of freedom does not exist. Nevertheless, the BehrensâFisher T can be compared with a corresponding quantile of Student's t distribution with these estimated numbers of degrees of freedom, , which is generally non-integer. In this way, the boundary between acceptance and rejection region of the test statistic T is calculated based on the empirical variances si2, in a way that is a smooth function of these.
This method also does not give exactly the nominal rate, but is generally not too far off.[ citation needed] However, if the population variances are equal, or if the samples are rather small and the population variances can be assumed to be approximately equal, it is more accurate to use Student's t-test.[ citation needed]
A number of different approaches to the general problem have been proposed, some of which claim to "solve" some version of the problem. Among these are, [7]
In Dudewiczâs comparison of selected methods, [7] it was found that the DudewiczâAhmed procedure is recommended for practical use.
For several decades, it was commonly believed that no exact solution to the common BehrensâFisher problem existed.[ citation needed] However, it was proved in 1966 that it has an exact solution. [12] In 2018 the probability density function of a generalized BehrensâFisher distribution of m means and m distinct standard errors from m samples of distinct sizes from independent normal distributions with distinct means and variances was proved and the paper also examined its asymptotic approximations. [13] A follow-up paper showed that the classic paired t-test is a central BehrensâFisher problem with a non-zero population correlation coefficient and derived its corresponding probability density function by solving its associated non-central BehrensâFisher problem with a nonzero population correlation coefficient. [14] It also solved a more general non-central BehrensâFisher problem with a non-zero population correlation coefficient in the appendix. [14]
A minor variant of the BehrensâFisher problem has been studied. [15] In this instance the problem is, assuming that the two population-means are in fact the same, to make inferences about the common mean: for example, one could require a confidence interval for the common mean.
One generalisation of the problem involves multivariate normal distributions with unknown covariance matrices, and is known as the multivariate BehrensâFisher problem. [16]
The nonparametric BehrensâFisher problem does not assume that the distributions are normal. [17] [18] Tests include the Cucconi test of 1968 and the Lepage test of 1971.
![]() | This article includes a list of general
references, but it lacks sufficient corresponding
inline citations. (February 2010) |