Are Classical Design of Experiments Models Too Limited for Modern Needs?

[DrDu]

I need to build up knowledge about Design of Experiments and have a fundamental question about the real goal of DoE. Most classical texts start with discussing in considerable detail full factorial design plans where each factor only has two levels. The underlying statistical model is (multi-)linear. However I wonder whether these models are really that important in practice. I.e., I would rather expect that most factors are continuous rather than dichotomic, like e.g. temperature in the design of a reactor. Then, a linear model would not allow to find an optimum but at best the direction in which to look for an optimum.

 

[micromass]

Design of experiments is only really important if you have a limited sample size. Clearly, if you can easily generate hundreds to thousands of samples, then there is no need for the entire design for experiments business.

But let's say you can only really afford to generate 10 samples. Then clearly you cannot seriously expect to find an optimum. But you might find some directions that work better or not. For example, you can try high temperature in a reactor vs low temperature, and high pressure vs low pressure. If you have 1000 samples available, you can very very easily try every temperature and pressure. But if samples are expensive, then this is not possible. Design of experiments gives you a very efficient way to get good information of where the optimum is.

So yes, a design of experiment will only tell you the direction in which to look for an optimum. So if you want to find the optimum, then this will not be sufficient for you. But sometimes this is all you can afford with respect to money or time.

 

[DrDu]

Thanks for your reply, micromass!
You are clearly right in that DoE is driven by optimal use of limited resources. But what I wonder: In general, I would not only want to try low and high pressure or low and high temperatures but maybe several levels. The classical DoE seems to overemphasize the importance of interactions as compared to nonlinearities which I would rather consider on an equal footing.

 

[Stephen Tashi]

The underlying statistical model is (multi-)linear.

The classical DoE seems to overemphasize the importance of interactions as compared to nonlinearities which I would rather consider on an equal footing.

What distinguishes classical DoE from "response surface methodology"https://en.wikipedia.org/wiki/Response_surface_methodology ? Is classical DoE just just the specialization of a multinomial objective function to a multi-linear function ?

I recall encountering Design of Experiments in an amusing example of bureaucracy in action. A study was planned to consider alternatives A,B,C,D,E using a Monte-Carlo simulation that took months to set-up and several days to run just a single "rep". A smart guy with influence had recently gotten his Phd with a dissertation in DoE. He and his advisor gave presentations arguing that it would be better to define the alternatives using DoE. There was resistance to this from other people because the each of the alternatives A,B,C,D,E had been agreed upon as acceptable by groups with diverse interests (not identical to the "objective" function). If a best alternative was estimated by DoE, it might be different that A,B,C,D,E and hence diplomatically controversial.

 

[FactChecker]

Thanks for your reply, micromass!
You are clearly right in that DoE is driven by optimal use of limited resources. But what I wonder: In general, I would not only want to try low and high pressure or low and high temperatures but maybe several levels. The classical DoE seems to overemphasize the importance of interactions as compared to nonlinearities which I would rather consider on an equal footing.

The goal of Design of Experiments is to determine the most efficient set of experiments that will adequately handle interactions of variables. Even in cases where a large number of experiments can be done, it is surprising how often they have been designed wrong. They end up with a lot of data that can not be used to separate the effects of interacting variables.