[Jain, Ch. 21]
We now study experiment designs with two factors, each of which taking an arbitrary number of levels.
The model is:
Compared to the one-factor experiment, we add term 
.
is the mean of all 
is the effect of factor A at level j, satisfying
is the effect of factor B at level i, satisfying
is an error term, satisfying
and 
model, for k=2:
model gives one value (i.e., 
)
for both factor levels.
Click here for a pdf image of Table 21.2 to see an example table for analysis of observations.
The column and row effects are 's and 's,
respectively.
Derivation of effects and 
:
In the last equation, 
is the column mean. So in the
table computation we add each column, then divide by b to obtain
. Similarly, we obtain for all i.
Results:
How much of the variation is due to:
As in earlier designs, square both sides of model equation:
Crossterms are zero. Thus:
We desire SSA/SST, SSB/SST, and SSE/SST. We compute SST and SSE
without computing the 's as follows:
and
For our example,
Thus number of caches is an important in processor design.
To statistically test factor significance, compute mean squared values of Y, 0, A, B, and E.
Recall:
and similarly for MS0, MSA, MSB, and MSE. Degrees of freedom are:
SSE has (a-1)(b-1) degrees of freedom because the rows and columns of the table must sum to zero.
Thus:
Finally, computed F statistics are:
See Table 21.4
See Table 21.5
Thus number of caches has significant effect on performance, but choice of workload does not.
Check that
See Fig. 21.1
See Fig. 21.2
See Fig. 21.2
(3) is violated because error variance is higher at higher values of the response.
CI formula for a quantify 
is the
mean plus/minus product of t distribution and standard
deviation of x:
where variance 
is
and, as usual, 
.
For our example, 90% CI for mean is:
where
Thus CI for is
All CI's are summarized below:
Table 1: From Jain Table 21.7. Asterisk denotes ``not significant.''
Sometimes a few values of 
are missing.
This may be due to:
We next consider analyzing two factor designs with missing observations.
Certain cells in the table of 's will be missing.
Jain lists alternative (and controversial) analysis methods.
Example: comparison of six processors on 11 workloads:
See Table 21.20
Should we use an additive model?
Evidence against additive model (A+B):
is several orders of
magnitude. (If 
and 
, then
, and factor B is irrelevant!)
In our data, 
: additive model is inappropriate.
We take the log of all observations to use a multiplicative model:
See Table 21.21
The grand mean is computed with 60 as the denominator, because
six observations are missing from the table of 
cells.
We will compute a confidence interval. The easiest way is just to compute SSE (not SST, SSA, etc.):
See Table 21.22
The degrees of freedom (DF) of errors is:
The term 60 reflects six missing observations!
Modify formulas expressing standard deviation of effects in terms of
to account for number of observations at each factor
level.
Let
Thus
and similarly for 
.
Confidence interval, using earlier formula ( 
), is:
See Figure 21.6
The significantly different pairs are RISC & Z8002 as well as 68000 with any other processor.
Plot of residuals versus response shows that largest positive error is on same order of magnitude as predicted response. These points correspond to 68000.
See Fig. 21.7
A normal quantile-quantile plot of residuals shows that errors are not normally distributed. The 68000 causes deviation from normal.
See Fig. 21.8
Problem: The 68000 has too many missing observations!
Without 68000, both visual tests pass:
See Fig. 21.9
See Fig. 21.10
Moral: If a row or column has many missing observations, exclude it!