In 1969, the renowned management guru Peter Drucker predicted: "To make knowledge-work productive will be the great management task of this century." Who knows more about their work than engineers and scientists? Yet, by traditional scientific methods, even the world's foremost subject-matter experts generally do their experiments in a most unproductive wayone factor at a time (OFAT). This so-called "scientific method" reached its apogee in the work of Thomas Edison, who doggedly applied trial and error OFAT to invent the light bulb, among other things. His approach was "1% inspiration and 99% perspiration."
Around 1926, a British statistician, Ronald Fisher, while working in the field of agriculture, developed a new form of experimentation called two-level factorial design. Two-level factorial designs are characterized mathematically as "2k," where k represents the number of experimental factors. Fisher's innovative 2k design of experiments (DOE) overcame the year-long agricultural growing cycle by studying many factors in parallel via sophisticated, matrix-based test plans.
The parallel 2k DOE approach offers tremendous efficiencies over the serial OFAT approach. Consider a three-factor (23) design (see figure 1). In this scenario, 2k DOE (left) offers four experimental runs at the high level for factor A and the same for the low level (right versus left faces of cubical experimental region). Similarly, factors B and C also benefit from four runs at both high and low levels (top versus bottom of cube and back versus front, respectively). Accomplishing all of this testing requires only eight experimental scenarios because the factors are varied in parallel.
Figure 1. Graphic comparison of an experiment design for a 23 DOE (left) versus a comparable OFAT approach shows the parallel processing afforded by DOE.
To provide similar power of replication for effect estimationto compete datawisethe serial OFAT approach must provide four runs each at the high levels of each factor and four each at the base line (all low levels at origin). This necessitates a total of 16 experimental scenarios for OFAT. As a result of its far-more-efficient parallel processing, 2k DOE trumps the serial OFAT scheme, and its efficiency advantage only becomes more pronounced as the number of factors (k) increases.
Figure 2. Using OFAT, we separately vary factor A (above), then factor B (below), to find a maximum response of 82. In an RSM approach, we develop a response surface (left) to determine a maximum of 94.
Let's really put OFAT to the fire by making use of an even more powerful form of DOE called response-surface methods (RSM). Assume that the OFAT experimenter arbitrarily starts with factor A and varies it systematically over nine levels from low to high (-2 to +2) while holding factor B at mid-level (0). The resulting response curve shows a maximum response for A at about 0.63 (see figure 2). The next step is to vary B while holding factor A fixed at this "optimal" point.
The response now rises from the previous figure of 80 to 82 by adjusting factor B to a level of 0.82 based on this second response plot. The OFAT experimenter announces the optimum (A, B) combination of (0.63, 0.82), which produced a response above 80 in only 18 runs. The real optimum of about 94, produced by RSM, is far higher.
This is the first part of our DOE discussion. Part II, appearing in the September 2005 issue, will offer an example of how this approach is actually applied.
1. M. Anderson and P. Whitcomb, DOE Simplified: Practical Tools for Effective Experimentation, Productivity Inc., New York, NY (2000).
2. D. Montgomery, Design and Analysis of Experiments (6th ed.), John Wiley, New York, NY (2005).
Mark Anderson is principal with Stat-Ease Inc., Minneapolis, MN.