Page 1
Lenth's Analysis of Unreplicated Factorial Experiments
When analysing 2k factorial experiments with replicated measurements at each treatment combination, an estimate of may be calculated from the replicates at each treatment combination and these may be combined using the "root mean square" (RMS) formula to provide an overall estimate.
When there is no replication, that is, there is just one measurement of the response at each treatment combination, this is not possible.
The traditional solution to this problem was to construct Normal plots of the estimated effects and judge as "significant" those that departed from linearity by a substantial distance. Following such a determination of the "significant" effects, an estimate of could be found by fitting a model corresponding to the "significant" effects, on the basis that the remaining effects reflected chance variation. A variation on this method may be used in the case that one or more factors appears inactive, that is, no main effect of interaction involving those factors appears "significant". In that case, the observations at the different levels of those inactive factors may be regarded as replicates at each combination of levels of the active factors and a full factorial analysis conducted.
In 1989, an alternative approach was proposed by Russell Lenth[1] and that approach has gained popularity and is included as an option in several software packages including Minitab. The method is explained here in the context of an example taken from BHH, Chapter 5, Sections 5.13, 5.14, pp. 199-208.
Illustrative example
As part of a chemical process development exercise, the effects of changing four process input factors on process yield (%) were studied in a 2^4 experiment. The factors included in the study, along with their experimental levels, were
A: Catalyst Charge (lb), 10 and 15,
B: Temperature (°C), 220, 230,
C: Reactant Concentration (%), 10, 12,
D: Pressure (psi), 50, 80.
A response was measured at each of the 16 reaction conditions, as listed below. The run order was randomised, as also shown.
DesignPoint / Catalyst
Charge (lb) / Temperature
°C / Concentration % / Pressure
psi / Run
Order / Yield
%
1 / 10 / 220 / 10 / 50 / 8 / 70
2 / 15 / 220 / 10 / 50 / 2 / 60
3 / 10 / 240 / 10 / 50 / 10 / 89
4 / 15 / 240 / 10 / 50 / 4 / 81
5 / 10 / 220 / 12 / 50 / 16 / 60
6 / 15 / 220 / 12 / 50 / 5 / 49
7 / 10 / 240 / 12 / 50 / 11 / 88
8 / 15 / 240 / 12 / 50 / 14 / 82
9 / 10 / 220 / 10 / 80 / 15 / 69
10 / 15 / 220 / 10 / 80 / 9 / 62
11 / 10 / 240 / 10 / 80 / 1 / 88
12 / 15 / 240 / 10 / 80 / 13 / 81
13 / 10 / 220 / 12 / 80 / 3 / 60
14 / 15 / 220 / 12 / 80 / 12 / 52
15 / 10 / 240 / 12 / 80 / 6 / 86
16 / 15 / 240 / 12 / 80 / 7 / 79
Normal plot of effects
Minitab command DOE may be used to calculate estimates of all 15 main effects and interactions. The results follow.
Term Effect
A -8.00
B 24.00
C -5.50
D -0.25
A*B 1.00
A*C -0.00
A*D 0.75
B*C 4.50
B*D -1.25
C*D -0.25
A*B*C 0.50
A*B*D -0.75
A*C*D -0.25
B*C*D -0.75
A*B*C*D -0.25
If all effects were null, these 15 effect estimates would constitute a simple random sample from a Normal distribution with constant standard deviation. To assess this, a Normal plot may be drawn, as follows.
This suggests one highly significant effect and three others that also appear to deviate substantially from the rest. From the list of effects above, these may be identified as the A, B and C main effects and the BC interaction.
Lenth's analysis
Using the Lenth's analysis option in the Minitab DOE command results in the following plots.
The first is an enhanced version of the Normal effects plot, with effects significant according to Lenth's analysis identified.
The second is a Pareto chart, designed to show the separation of the "vital few" from the "trivial many", in accordance with the "Pareto Principle" popularised by the quality guru Juran. In this regard there does appear to be a sharp distinction between the four biggest (in magnitude) effects and the rest. The placement of Lenth's "margin of error" at 1.93 corresponds with this. The "margin of error" is the product of Lenth's estimated standard error, PSE = 0.75, by a critical t-value.
Lenth's analysis explained
The basis for Lenth's analysis is that, given several Normal values with mean 0 and common standard deviation, and given their absolute values (magnitudes, or values without signs), then it may be shown that
SD(Normal values) ≈ 1.5 × median(Absolute values).
If effects were null, then this could be applied directly to the estimated effects. The list shown above may be adjusted to show just the absolute values of the effects,
Term Effect
A 8.00
B 24.00
C 5.50
D 0.25
A*B 1.00
A*C 0.00
A*D 0.75
B*C 4.50
B*D 1.25
C*D 0.25
A*B*C 0.50
A*B*D 0.75
A*C*D 0.25
B*C*D 0.75
A*B*C*D 0.25
and then sorted in increasing order of the absolute values,
Term Effect
A*C 0.00
D 0.25
C*D 0.25
A*C*D 0.25
A*B*C*D 0.25
A*B*C 0.50
A*D 0.75
A*B*D 0.75
B*C*D 0.75
A*B 1.00
B*D 1.25
B*C 4.50
C 5.50
A 8.00
B 24.00
In a sorted list of 15, the 8th is the median, here 0.75, so that Lenth's estimate of effect standard error would be
1.5 × 0.75 = 1.125.
Since it is unlikely that all effects are null, Lenth refines this estimate by excluding all effects that exceed 2.5 times this estimate (on the basis that the chances of a null estimate exceeding this value are negligible) and repeating the exercise on the remaining effects.
Since
2.5 × 1.125 = 2.8125,
this entails excluding the last four effects in the ordered list, determining the median of the remaining 11, 0.5, and calculating
PSE = 1.5 × 0.5 = 0.75.
Here, PSE stands for "Pseudo Standard Error".
To convert this into a critical value for determining "significant effects, Lenth multiplies by a critical t-value with degrees of freedom equal to total number of effects divided by 3. This formula was arrived at by a combination of probabilistic simulation, trial and error and judgement.
In this case, it leads to
df = 15 / 3 = 5,
and
t0.05, 5 = 2.57,
so that the critical value for effects is
2.57 × 0.75 = 1.9275,
rounded to 1.93 by Minitab.
In summary
Lenth's PSE, or pseudo standard error, is used in calculating a critical value for the effects when there are no replicates. It is based on the fact that the standard deviation of a sample from a N(0,) distribution may be estimated as 1.5×median(absolute values). In case some effects are non null, a refinement is to delete effects that exceed 2.5 times this estimate and recompute. To find the critical value for the effects, the PSE is multiplied by the appropriate critical value for t with m/3 degrees of freedom, where m is the number of effects being assessed.
[1]Russell V. Lenth, (1989), "Quick and Easy Analysis of Unreplicated Factorials", Technometrics, 31, 4, 469-473.