Confidence Intervals for Effects

Each response variable has some true value, which we will never know.

From r replicas of an experiment (holding factor levels constant), we can estimate response variable's true value by statistical inference.

How far is our estimate from the true value?

Suppose our data satisfies three conditions:

1. Independence of error components: each experiment outcome is in no way related to the outcome of any other experiment
2. Normal distribution of errors ( ) with zero mean
3. Homogeneity of error variance: the variance of the errors for all i must be equal

Then t-distribution yields confidence interval for some significance level ( ).

Computing confidence intervals

Review of CI

Recall CI formula from Ch 13.2 in Jain for n samples:

For factorial design, . But what is sample standard deviation s?

Computing Sample Standard Deviation

Sample variance of errors ( 's) is:

where .

Denominator is degrees of freedom. It's less than number of experiments performed (i.e., ) because errors for all replicas for each combination of factor levels sums to zero (i.e., ).

Let q denote any effect or interaction:

Therefore the desired sample standard deviation of q is

Computing CI for Effects

CI for an effect (for ) is

Example

Continuing the memory-cache study, the sample standard deviation of errors is:

The sample standard deviation of each effect or interaction q is:

The t-value at degrees of freedom and is . Thus CI for each effect is:

Therefore:

How To Interpret Confidence Interval

Note that all CI's exclude zero; therefore all are effects and interactions significant!

Suppose we choose and perform many factorial experiments. Then for 90% of them the true mean value of each effect or interaction lies in the confidence interval calculated for that experiment.

Thus in 10% of the experiments, the true mean will not lie in the confidence interval!