Life Science, Engineering, Design Enschede

1. Life Science, Engineering, Design Enschede Design of Experiments DoE Introduction www.saxion.nl/led

2. Design of Experiments DOE • Essential tool in Six-Sigma strategy • Black-Box approach of the problem • Experimental, empirical way of finding out: • What factors significantly affect the CTQ • If there are any interactions that affect the CTQ • What is the precise effect on the CTQ • Characteristic for DOE: • Reducing number of tests by smart design of the experiments and using statistics

3. Design of Experiments: DOE Noise factors Input: Adjustable factors Gauge Response Process CTQ: critical to quality!! Measurable! Essence: there is always VARIATION!!!!!

4. Model for response to inputCTQ = f(x1, x2, x3,.., xn) • Scientific approach: Theory (concepts, principles, laws) gives a model for the relation between input and response. • When this is sufficient DoE is not needed! • DOE: Black-Box approach, NOT using scientific theory at all simply because this is not available or the problem is too complex

5. Goal? • What do we want to achieve with such an experiment: • Best setting of factors that lead to • Optimal response (stable, low variation) • The sensitivity for noise-factors • The variation in the process in view of stability and robustness

6. Sequential strategy • Screening: with many controllable factors, which factors are most important? • Refining: wich adjustments of the critical factors are required? • Optimization: what is the optimal setting of the critical factors?

7. All experiments are experiments! • Often poorly design: valuable resources are used inefficient and ineffective (subjective conclusions): poor ROI • Statistically design and analysis permits efficiency in the process and leads to scientific objectivity: high ROI

8. Interactions • Suppose, there are only 2 input factors, x1 and x2 • Then x1 and x2 can have an effect on the response of their own, but there can also exist a so-called interaction-factor x1x2 • The linear model y=ax1 + bx2 + cx1x2+ constant • Where: • a is the effect of the main factor x1ony • b is the effect of the main factor x2ony • c is the effect of the interactionx1x2ony

9. Interactions Suppose we have 2 input factors x1 and x2 that could have an effect on response y Which tests could be run to determine the effects of x1, x2 and x1x2 on y?? Suppose you can run 5 tests. What values for x1 and x2 would you choose and why? y-axis y-axis centerpoint x2-axis x2-axis x1-axis x1-axis

10. Interactions • 2 input factors: two main factors and one 2nd order interaction exists • 3 input factors: three main factors, three 2nd order interactions and one 3rd order interaction exists • 4 input factors: four main factors, six 2nd order interactions, four 3rd order interactions and one 4th order interaction exists

11. Interactions • The higher the order of an interaction is the lower the likelihood of significant impact • Rule of thumb: • 2nd order interactions occur regularly • 3nd order interactions occur incidentally • The occurrence of 4th and higher order is usually negligible!

12. Example Velocity v Process in industry: coating copper wire 3 adjustable factors: Temperature in pipe-oven, T; Wire-velocity, v; Viscosity of the solvent,  CTQ: number of short-circuits in a coil Regular setting: T=60C;  =1,010-3 Pas; v=7,2m/s Problem: 31% defects!!!!! Cause: variation in coating thickness Pipe oven temp T Coating supply 

13. Intermezzo: Sample mean & variance Sample mean Sample mean Sum of Squares SS Sample variance Degrees of freedom  In general: Mean Squares = Sum of Squares divided by Degrees of freedom: MS = SS/ Note: s2 can be determined only if n  2

14. Variance • Information of variance is needed for • Determining stability and robustness • Evaluation measuring method • Only one way to obtain information about variance: Repeat measurements!!!!

15. Trend / Drift in noise factors • Trend or drift that frequently occur: • Chance in temperature, pressure or humidity • Wear (tools or gauge) • Fatigue, habits or practice of operators • Only one way to prevent that these trends are attributed to the influence of one or more factors: Randomize measurements!!!!

16. A procedure to evaluate change in variation is needed! Statistical F-test

17. Intermezzo: F-Test Is the difference in variance between two samples significant or not? Data samples  0, s02, 1, s12 Issue: difference in variance • Zero hypothesis H0: 12/02 = 1; H1: 12/02 > 1 • Choose the level of significance  (mostly 0,05) • Generally: The F-test is one-sided • Determine the values of F=s12/s02 and F0 (, 1, 2) • Critical region: F > F0 ; Acceptance region: F < F0 • Draw conclusion

18. =5% F0 Acceptation region Critical region

19. Intermezzo: F-Test If F < F0, F is in the acceptance region. This means: s12not significantly exceeds s02 If F > F0, F is in the critical region. This means: s12indeed significantly exceeds s02

20. Intermezzo: F-TestHypothesis Testing: two types of errors

21. Intermezzo: F-TestSignificance level and Power Type I-error: Rejection the null hypothesis when in fact it is true P(type I error) = a a = significance level or size of the test Type II-error: Failing to reject the null hypothesis when in fact it is false P(type II error) = b (1-b) = power (sensitivity) of the test

22. Intermezzo: F-TestSignificance level and Power We want the probabilities that either of the errors occur both to be as small as possible, however: •  can be set on a desired value up front •  strongly depends on the true (not known) value of the parameter that is tested and can only be estimated after the test • Decreasing  generally increases  • To obtain a smaller value of  one can either increase  or the sample-size n. • Increasing sample-size is obviously the best choice in most cases.

23. Relation Tolerance and Gauge Noise factors Input: Adjustable factors Gauge Response CTQ: critical to quality!! Measurable! Process Tolerance: USL - LSL What part of the tolerance is already consumed by the gauge???

24. Gauge-capability (Cp) Tolerance compared with variation due to gauge LSL USL 6gauge

25. Relation Tolerance and Gauge Each gauge and measurement method contributes to the variance in the data!! Rule of thumb: gauge << 0,1(USL-LSL)

26. Always first: Gauge R&R Before proceeding with DoE an evaluation of the measurement method, system and procedure (m, s & p) is necessary. Why? Is the measurement (m, s & p) adequate with respect to the defined specifications? When not: it’s useless to proceed with DoE!!!

27. DOE: essential step = Gauge R&R Each gauge and measurement method contributes to the variance in the data!! Repeatability = degree of variance (s2repeat) in the measurement data when the measurement is repeated under exact the same conditions Reproducibility = degree of variance (s2repro) in the measurement data when the measurement is repeated under different conditions

28. Gauge R&R Evaluation measurement method and procedure Measurements in which all factors are held constant! To determine Repeatability (2repeat): repeat the measurement several times under exact the same conditions AND ALSO To determine Reproducibility (2repro): repeat the measurement several times under different conditions (operator, instrument, …..) Perform ANOVA on the sample data

29. Gauge R&R Gauge R&R is like a one-factor experiment with the ‘measurement-condition’ as factor Hypothesis: Different conditions have no effect! This means that all data come from a certain normal distribution (,) Number of different conditions: a Number of repeats: n a·n tests; total= (a·n-1); repro= (a-1); repeat= a·(n-1)

30. Gauge R&R Hypothesis: Different conditions have no effect! This means that all data come from a certain normal distr. (,)

31. Gauge R&R It can be shown that MSerror , MSrepeat and MSrepro are independent. Also: MSrepeat is an unbiased estimator of 2repat

32. Gauge R&R; 4 conditions and 5 repeats

33. ANOVA-table

34. Gauge R&R

35. Gauge R&R: drawing conclusions Compare variance due to gauge (2gauge) with the specifications (USL-LSL) and the required accuracy! When adequate  proceed with DoE When not adequate  improve measurement method, system and procedure (m, s & p) first! When variance due to reproducibility is significant larger then variance due to repeatability 2repro> 2repeat the measurement procedure should be improved!!

36. Gauge R&R; parts (4), operators (3), repeats (2)

37. Gauge R&R; parts (a), operators (b), repeats (n)Calculation totals and averages

38. Gauge R&R; parts (a), operators (b), repeats (n)Sum of Squares

39. Gauge R&R; parts (a), operators (b), repeats (n)Degrees of freedom

40. Gauge R&R; parts (a), operators (b), repeats (n)Mean Squares and F-values

41. Gauge R&R; parts (a), operators (b), repeats (n)Unbiased estimators of variation It can be shown that: E(MSparts)=2+bn2parts+n2part·operator E(MSoperator)=2+an2operator+ n2part·operator E(MSpart·operator)=2+n2part·operator E(MSerror)=2 2T=2parts+ 2operators+ 2part·operator +2error 2repro 2repeat 2gauge =2repro+ 2repeat

42. Gauge R&R; parts (a), operators (b), repeats (n)Unbiased estimators of variation

43. Gauge R&R; parts (a), operators (b), repeats (n)ANOVA-table

44. Errors • Noise: mean value is E = 0, variance E2 • Constant systematic error combined with noise: mean value is E ≠ 0, variance E2 • Drifting systematic error combined with noise: mean value is E ≠ f(t), variance E2

45. Sources of Variance • In each process variance is inevitable!! • Noise factors and gauge always cause variation in the response • This is variation that is not due to the adjustable factors and can be noted as e2 (error)

46. Sources of Variance • DoE: vary factors  • find out whether factors are significant influence on the response or not! ANOVA • And in the case of significant influence: • Find out what is the effect of the factor on the response

47. General procedure of ANOVA • Calculate Sum of Squares for the total data, factors, interactions and error • Determine the degrees of freedom • Calculate the Mean Squares (MS) for each factor, interaction and error • Determine the boundaries of the acceptance and critical regions (F0-values) • Calculate F-values for factors and interactions • Determine which factors and interactions have a significant effect on the response • And in the case of significant influence: • Find out what is the effect of the factor on the response

48. Intermezzo: F-Test If F < F0, F is in the acceptance region. This means: sfactor2not significantly exceeds serror2 If F > F0, F is in the critical region. This means: sfactor2indeed significantly exceeds serror2 In this case the factor significantly influences the process investigated. The question is: How strong is the effect of this factor? Test results  ‘empirical model’

49. Doe in 16 steps Determine the respons (CTR) that really matters Define specification limits and required accuracy in the respons and thus in the the sample data Choose measurement system (instrument, procedure) Perform Gauge R&R Determine factors that should be investigated Determine range and levels of the factors Design the experiment (type of design, repeats, randomization, blocking) Perform the experiment

50. Doe in 16 steps (cont’d) Analyze the sample data Determine significant factors and interactions Determine the effect of the significant factors and interactions Propose a model for the respons Determine and evaluate the residuals Accept or reject the proposed model Perform a confirming test of the model Refine or optimize the results