Designing experiments for SPC
Design of
Experiments
in SPC
Design of Experiments (DOE) is used a catch-all term to describe a set of statistical methods and tools that ensure effective and efficient conduct of experiments. The steps that constitute DOE include designing and conducting the experiment, data analysis, and interpretation of the results.
* People often conflate design of experiments (DOE) with designing an experiment, which are two separate concepts. DOE incorporates the designing of an experiment as an important step in the process but it also includes other steps that are just as vital.
Traditional approach (w/o DOE and statistics)
In an attempt to save time and money, while creating the least amount of disruption and causing the minimum hassle, most companies tend to approach the experimentation part of the process with the goal of performing a few all encompassing large-scale experiments.
This approach treats all factors discovered during the process of creating the cause-and-effect diagram as being relevant and try to develop operating procedures to monitor all of them. This quickly becomes time consuming, costly and inefficient. It also create frustration amongst the equipment operator as they are asked to collect data, that in most cases is never studied or followed up on in any manner.
This often coupled with relying on experience and expertise of employees who often try tweaking the process at several different places, often lacking proper coordination or knowledge of all the changes being instituted by other departments.
Another common trait of this approach is the reliance on a small number of results, done in an effort to save time and money. As a whole such an experiment, a large-scale attempt to monitor several factors, while tweaking the process independent of it, using a small number of results, leads to hard to interpret results.
Even if the results came out as hoped, how can it inspire any confidence that the results will be the same over time when the changes are made permanent. Such large unstructured experiments also make it really hard to perform any statistical analysis on the results, which further invalidates any confidence that could be gained from the results.
* It is perfectly reasonable for the reader to assume, from the above passage, that the best approach would be to perform multiple smaller experiments where all factors are tested individually but it is not so.
When you test factors one-at-a-time it becomes impossible to ascertain how factors interact with one another, which might the the root cause of the problem rather than the the factors themselves.
Challenges in setting up experiments:
When in an industrial setting, even the simplest of experiments can be difficult to setup, execute and analyze the results for. This complexity contributes to the high costs associated with experiments, both in terms of time and money, especially if the sample are analyzed destructively.
Because of the aforementioned factors, using statistically designed, conducted, and analyzed experiments become imperative. With proper design of experiments (DOE) you will get the most value for your investment.
How can
Design of Experiments
(DOE) help?
DOE, and the statistics associated with it, help answer some of the most pertinent questions when a company is looking to conduct experiments in search of the root cause of a problem:
- Which variable or combination of variables affected the results?
- Are the results significant (i.e., likely to be the same if the experiments were conducted again)?
Steps in Design of Experiments (DOE)
Step 1: Objectives
At the beginning of the experiment two questions need to be answered,
- Why are we conducting the experiment?
- What does the company hope to learn from it?
Step 2: Determine the ‘Response variables’
Response variables are the determined by the outcomes of interest of any experiment. They must be selected carefully, it serves to keep in mind that response variables must match the problem that is needed to be solved.
Next, a way of measuring these variables needs to be established, this includes how, when and where in the process they will be measured.
Step 3: Determine the ‘Process variables/factors’
It can be hard to know which factors to control and which ones to vary, especially on the first attempt. A common strategy is to rely on the experience of your employees and iteration.
This step also involves deciding the correct granularity for process variables of the experiment. Once the granularity is decided, methods for measuring each continuous factor and labeling scheme for discrete factor also needed to be decided upon.
* Granularity can be a tricky concept to deal with, especially for continuous factors. You must ask yourself how much coverage do you want? For example, assume a factory making cast iron skillets gets iron ranging from 80%-95% purity, what percentages of iron should you study? Levels should be realistic but with enough range, so that, if the factor is indeed important, then, real differences are likely to occur.
Step 4: Number of replicates
Based on the process variables, there are a bunch of combinations of factors that need to be tested. This raises the question, how many samples do you plan to collect for each combination of factors? There are statistical formulae that provide guidance on the number of replicates needed, depending on the variability of the measure of interest.
The number of replicates needed are also determined by the level of difference you want to be able to detect and your desired certainty in the outcome.
* Usually, the ability to detect significant results increases with the number of replicates. However, If experience has shown that a factor is consistent, has low variability, fewer samples may be needed and if the factor has high variability in the results, a larger sample will be needed to detect significant differences.
If the variability is extensive, then it might make more sense to improve the stability of the process before conducting an experiment.
Step 5: Detailed experimental plan
Create a plan that details, step-by-step. Who will do what, when, and where they’ll do it? Specify the materials, procedures, operators, measurement tools, testing date and time, etc. It is also important to consider how you will analyze the data, and how you will interpret and use the results. Make sure to analyze in a way that answers critical questions. Perform ‘Dry run’ to figure out what might go wrong, how things are going to move, special fixtures required, and try to analyze the preliminary data.
* Establish a budget and set deadlines at this step. Experience would suggest that no more than 1/4th of the full budget should be spent on the first experiment. Many times, a first trial will reveal more questions and provide suggestions for what to study next.
Step 6: Decide what factors to be held constant
In step 3, factors that were to be varied intentionally were identified, however, other factors that are held constant still need to be taken into account. Besides the potential causes that are being studied, every other cause in the cause-and-effect diagram should be held constant. The goal is to eliminate, as much as possible, all other factors that may be the cause of variability, thus providing assurance that any differences in results are due to the selected experimental factors.
Step 7: Post-Experiment Plans
This step consists of finding answers to the following questions, How will you use the results? If the results suggest a potential solution, how, by whom, and when will it be implemented? Will you conduct confirmation trials? How will you monitor the process to be sure that the changes remain in place and continue to effectively reduce the problem? If the tests are not successful, what course of action will you take next?
Conclusion
Often at the end of an experiment there is a good chance that there will be no definite solutions but rather recommendations that need to be implemented and followed up on. Still there is great value in designing experiments using DOE and statistical techniques because they can help you ensure you maximize the return on your investment, especially given the high costs and general complexity of conducting experiments in an industrial environment.
Want to learn more?
Factory System’s FSWorks is a world-class data acquisition and quality management software
Still not sold?
Try our 30-day trial and witness the transformational ability of FSWorks for yourself
