Locked lesson.
About this lesson
Exercise files
Download this lesson’s related exercise files.
DOE Studies.docx61 KB DOE Studies - Solution.docx
60.1 KB
Quick reference
DOE Studies
Design of Experiments (DOE) Studies are a process that generate test results used in a statistical analysis that supports a clear business objective.
When to use
DOE Studies are used to determine optimal settings of product or process design factors and to understand the interaction effects between those factors. Studies are normally done during product or process development prior to design freeze or during improvement projects to understand the significance and interaction effects between factors.
Instructions
DOE studies must be carefully planned and managed so that the test results can be statistically analyzed. This statistical analysis relies on precise configurations for each experimental test and in some cases the statistical analysis requires that the tests be done in a specific order. If test configurations are incorrect or out of order, the analysis can become meaningless and invalid.
The study starts with a clear objective for the study. This should clarify the purpose of the study and the factor(s) that is measured in each of the experimental runs. This objective will determine both the type of DOE study that should be used and the factors that should be included in the study design.
This DOE study is supporting a business goal. When this goal is associated with design of new products or processes, the study should be completed before design freeze, so the results can be used to create the design. When the study is used to investigate a problem or issue, the study results are often combined with other business factors to make the business decision with regards to the problem or issue.
Hints & tips
- Carefully plan the DOE study and monitor what others are doing. People who don’t understand how a DOE study works or its goals will often change what is tested or the test sequence, thinking they are “helping” to get the right answer. This is because they do not understand the statistical analysis associated with the study and the need for precise test configurations.
- The study objective should have a measurable (variable) factor as its goal, not just a pass/fail factor.
- Ambiguous DOE study objectives make the study design step very difficult. When there is confusion at that step, go back to the stakeholders and update the study objective with more clarity.
- 00:04 Hello, I'm Ray Sheen.
- 00:05 There are many different variations of DOE Studies.
- 00:08 However, there are a few common questions and guidelines.
- 00:12 And before we look at the different types of DOE studies,
- 00:15 let's address these common points.
- 00:18 I'll start with some basic ground rules for conducting a study.
- 00:23 First, a DOE is a process, not an event.
- 00:26 It will take time.
- 00:27 It has many steps that must be completed in the correct order.
- 00:31 In fact, one of the reasons we teach this course is to help you to
- 00:35 learn the process, so you can apply it and conduct a successful DOE study.
- 00:39 Because it is a process, it must be planned and
- 00:42 monitored to ensure that the process is followed correctly.
- 00:46 The DOE study results will be based upon a statistical analysis.
- 00:50 If the study process is not managed well, the data points used in the statistics
- 00:55 will not be valid and the statistical analysis will be meaningless.
- 01:00 You will see as we go through the process of different DOE types, that they often
- 01:04 will consist of several studies or sets of experiments, each one narrowing the area
- 01:10 of focus, to improve the understanding and validity of the statistical results.
- 01:14 Again, this process must be managed well so that the results are meaningful.
- 01:19 Another reason we need careful planning is that the study often involves multiple
- 01:24 functions or organizations.
- 01:25 You may need special materials from suppliers, or
- 01:28 specially designed and built test samples.
- 01:31 You may want operations to conduct experiments to remove
- 01:34 one source of variation.
- 01:36 All this will require a coordination and that means project planning and
- 01:40 project management.
- 01:41 One other point is the timing of the study.
- 01:44 If this study is supporting product development of a new product and
- 01:48 not process improvement for an existing product or
- 01:50 process, then the study needs to be done before the design freeze.
- 01:54 The study will determine which parameters or factors are critical to product
- 01:58 performance and the best settings for those parameters.
- 02:01 Learning these after the design work is done and the design is frozen,
- 02:05 will delay the development and create extra costs.
- 02:08 Determining it before the design is frozen, will enhance the ability of
- 02:12 the development team to create a successful new product.
- 02:15 That brings us to a critical aspect of any DOE study and that is the study objective.
- 02:21 A DOE study should have a clear, measurable objective.
- 02:24 This will lead to the selection of the type of DOE, the factors to be studied,
- 02:29 and the communication and application of the study results.
- 02:32 DOE studies can have many different purposes.
- 02:35 They may be used to understand the implication of design trade-offs,
- 02:39 to characterize the design space of new technologies or new products, or
- 02:44 to determine the limits of reliable performance.
- 02:47 The structure of the study will depend upon the objective, so
- 02:50 make sure that that is clear before you start.
- 02:53 Let me give you a few examples of what I mean by an objective.
- 02:56 These are all paraphrases of study objectives I have used in the past:
- 03:01 Improve system accuracy and efficiency by 10%.
- 03:04 Notice that you can have multiple measurements in the objective.
- 03:08 Improve the yield of soldering process by 20%.
- 03:11 Notice, the objective focuses the aspects of the study,
- 03:15 which will be helpful for factor selection.
- 03:18 Achieve 99.5% first pass yield on final acceptance testing.
- 03:23 Notice, the objective sets a target for performance that will be useful for
- 03:27 describing the trade-off among factors, when translating the statistical
- 03:32 results into a practical process and product design and performance parameters.
- 03:37 In each case, the objective is used to help design study, so that it can answer
- 03:42 questions about the impacts of various factors on product or process performance.
- 03:47 And as with any study that requires time and money,
- 03:50 you need to tie your objective to a clear business goal.
- 03:53 That explains why you are expanding your resources.
- 03:55 If there is no business goal that is affected, then there is no reason for
- 03:59 the study.
- 04:00 Let me foreshadow some of the questions that must be answered based upon
- 04:04 the objective.
- 04:05 These questions will impact the study design, which means the time and
- 04:09 resources needed to complete the project.
- 04:11 With a clear objective, these questions are easy to answer.
- 04:14 When the objective is vague or contradictory, the answers are unclear.
- 04:18 First, what is the goal?
- 04:20 This will help to decide the type of DOE study and the factor selection.
- 04:25 Next, how many factors?
- 04:27 There are really two questions here, how many potential factors and
- 04:31 how many can we afford to study in depth.
- 04:33 Again, the answer will determine the type of study and the factor selection.
- 04:37 What is the response factor to be measured?
- 04:40 This is usually the way that the study goal is described.
- 04:43 It must be directly related to the objective, for the study to be successful.
- 04:48 This is the performance measure or measures that are recorded after each
- 04:52 experimental run and used in the statistical analysis.
- 04:55 Now, the remaining questions will get into the study design and
- 04:59 we'll understand the answers to these,
- 05:02 as we explain the details of planning conducting a study in upcoming lessons.
- 05:06 I mention them now because the answers to these will fundamentally change
- 05:11 the structure, planning and monitoring of the study.
- 05:14 The first deals with the type of study.
- 05:17 Full or fractional factorial.
- 05:19 The next is replicates, this means to double or
- 05:22 triple the number of test runs in the study.
- 05:25 Obviously it cost more, but it improves the accuracy of the statistics.
- 05:29 The use of center points.
- 05:30 This is a DOE study calibration technique.
- 05:33 If the study will be conducted over a long time period, the importance of these
- 05:37 points is increased, to ensure that the test conditions are not drifting.
- 05:41 Blocks, again this is a study design question that takes into account the real
- 05:46 world limitations and constraints.
- 05:48 We will discuss this more in detail in another lesson and
- 05:51 provide some ground rules for when to use blocking.
- 05:54 The answers to these last few questions will have a major
- 05:57 effect on the DOE study design and therefore on cost and schedule.
- 06:01 The study objective and real world constraints will dictate the answers.
- 06:06 A foundation for successful DOE study is a clear objective,
- 06:11 up front planning and effective project management of the plan.
- 06:16 These success criteria apply to any DOE study,
- 06:19 regardless of the type of study or breadth of analysis.
Lesson notes are only available for subscribers.
PMI, PMP, CAPM and PMBOK are registered marks of the Project Management Institute, Inc.