Boschloo's test is a statistical hypothesis test for analysing 2x2 contingency tables. It examines the association of two Bernoulli distributed random variables and is a uniformly more powerful alternative to Fisher's exact test. It was proposed in 1970 by R. D. Boschloo.
Setting
A 2x2 contingency table visualizes independent observations of two binary variables and :
The probability distribution of such tables can be classified into three distinct cases. The row sums and column sums are fixed in advance and not random. Then all are determined by . If and are independent, follows a hypergeometric distribution with parameters : . The row sums are fixed in advance but the column sums are not. Then all random parameters are determined by and and follow a binomial distribution with probabilities : Only the total number is fixed but the row sums and the column sums are not. Then the random vector follows a multinomial distribution with probability vector .
Fisher's exact test is designed for the first case and therefore an exact conditional test (because it conditions on the column sums). The typical example of such a case is the Lady tasting tea: A lady tastes 8 cups of tea with milk. In 4 of those cups the milk is poured in before the tea. In the other 4 cups the tea is poured in first. The lady tries to assign the cups to the two categories. Following our notation, the random variable represents the used method (1 = milk first, 0 = milk last) and represents the lady's guesses (1 = milk first guessed, 0 = milk last guessed). Then the row sums are the fixed numbers of cups prepared with each method: . The lady knows that there are 4 cups in each category, so will assign 4 cups to each method. Thus, the column sums are also fixed in advance: . If she is not able to tell the difference, and are independent and the number of correctly classified cups with milk first follows the hypergeometric distribution .
Boschloo's test is designed for the second case and therefore an exact unconditional test. Examples of such a case are often found in medical research, where a binary endpoint is compared between two patient groups. Following our notation, represents the first group that receives some medication of interest. represents the second group that receives a placebo. indicates the cure of a patient (1 = cure, 0 = no cure). Then the row sums equal the group sizes and are usually fixed in advance. The column sums are the total number of cures respectively disease continuations and not fixed in advance.
An example for the third case can be constructed as follows: Simultaneously flip two distinguishable coins and and do this times. If we count the number of results in our 2x2 table (1 = head, 0 = tail), we neither know in advance how often coin shows head or tail (row sums random), nor do we know how often coin shows head or tail (column sums random).
Test hypothesis
The null hypothesis of Boschloo's one-tailed test (high values of favor the alternative hypothesis) is:
The null hypothesis of the one-tailed test can also be formulated in the other direction (small values of favor the alternative hypothesis):
The null hypothesis of the two-tailed test is:
There is no universal definition of the two-tailed version of Fisher's exact test. Since Boschloo's test is based on Fisher's exact test, a universal two-tailed version of Boschloo's test also doesn't exist. In the following we deal with the one-tailed test and .
Boschloo's idea
We denote the desired significance level by . Fisher's exact test is a conditional test and appropriate for the first of the above mentioned cases. But if we treat the observed column sum as fixed in advance, Fisher's exact test can also be applied to the second case. The true size of the test then depends on the nuisance parameters and . It can be shown that the size maximum is taken for equal proportions and is still controlled by . However, Boschloo stated that for small sample sizes, the maximal size is often considerably smaller than . This leads to an undesirable loss of power.
Boschloo proposed to use Fisher's exact test with a greater nominal level . Here, should be chosen as large as possible such that the maximal size is still controlled by : . This method was especially advantageous at the time of Boschloo's publication because could be looked up for common values of and . This made performing Boschloo's test computationally easy.