Gage R&R Study Design Considerations

Locked lesson.

Upgrade

About this lesson

When designing the study, there are several critical considerations based upon the type of test (destructive or non-destructive), the discrimination of the test system, and whether attribute tests can designate an item into multiple categories beyond Pass/Fail. These considerations impact the accuracy and performance of the analysis.

Quick reference

Gage R&R Study Design Considerations

When designing a Gage R&R study there can be issues that create unusual of special considerations.  These are often caused by the test equipment or the nature of the inspection or test.  This lesson will review some of these considerations and explore approaches for addressing the issue.

When to use

Each of the considerations are based upon special circumstances.  Nested studies are done when it is impossible for all operators and all equipment to test all items multiple times.  If the discrimination of the test is inadequate, the study should either be redesigned, or the test method rejected.  Finally, when attribute testing has multiple categories (more than just pass/fail) an additional test needs to be done to ensure the attribute data Gage R&R will be effective.

Instructions

The special conditions addressed in this lesson require a design modification or additional calculations in order to ensure the Gage R&R results are trustworthy.

Crossed and Nested Designs

The typical Gage R&R study is a crossed study.  That means that all appraisers will test all items multiple times and if there are multiple pieces of equipment, they will all test all items.  However, in some cases this can not be done.  Some types of tests are destructive or consume the test item.  This is often the case in chemical analysis or in some cases it is impossible to move the item between test centers. In these cases, a nested design is used.

The nested design is based upon a critical assumption.  The assumption is that items can be batched and that items from the same batch are identical.  This is often a valid assumption for chemicals.  An advantage of the nested design is that you don’t need all operators to test all parts multiple times.  The problem with a nested design is that it loses some accuracy.  Interaction effects cannot be easily modelled, and the level of uncertainty is higher.  If you need to do a nested design, I recommend that you refer to an advanced statistical book or rely on a statistical software program such as Minitab to assist in the design and analysis.

Number of Distinct Categories (NDC)

Another concern is the accuracy and the discrimination of the test equipment used within the study. Be sure to check the calibration status of your measuring equipment and if appropriate do a calibration to limit bias errors. Also, be sure the equipment is accurate in the region of the parts being measured.  Check the linearity study to determine this.  The NDC value is a number that is calculated after a study is completed to inform you whether the study can be trusted or if you either need to redesign the study or reject the measurement system.

NDC looks at the discrimination of the measurement system in the region of the items that were being measured.  The higher the NDC the better.  If NDC is less than 2 then the measurement system is unable to discriminate the value of one item from another.  If the value is between 2 and 5, the system can divide the items into two or three categories, such small/large or low/medium/high.  This may be acceptable for an attribute data Gage R&R study but is definitely not for a variable data Gage R&R study.  If the NDC is between 5 and 14 it is considered marginal.  If the item being measured is a critical parameter, you should consider improving the test method.  A value over 14 is good and the Gage R&R results can be relied upon.

 

The NDC calculation is:The standard deviation of the part is based upon the parts that are used in the Gage R&R study.  The standard deviation of the gage is calculated in the Gage R&R study.  A common mistake that is made in variable data Gage R&R studies is that the parts used do represent the full range of allowable parts.  If the parts are almost identical, the standard deviation for the parts will be very small and the NDC will also be small.  For this reason, be sure that you are using the full tolerance range when selecting items for the Gage R&R study.  If the full range is used and the value is still low, that is an indication that the measurement system should be rejected.

Fleis’ Kappa

The third special consideration is associated with attribute data Gage R&R studies.  A typical attribute study uses two categories for analysis – pass/fail.  However, in some cases that attribute analysis is categories, such as Likert scale good – better – best.  In this case, there are multiple selection categories, so a different measure is used known as Fleis’ Kappa.

Fleis’ Kappa will have a value ranging form plus one to minus one.  Plus one means that the appraisers perfectly agree.  Minus one means that the appraisers perfectly disagree.  A value of zero indicates that the likelihood of their agreement is based upon random chance.  The Fleis’ Kappa value must be greater than 0.7 for the Gage R&R study to provide reliable results. A value of 0.9 or greater is ideal. 

The Fleis’ Kappa ratio is:

In this formula the P is the percentage of time that the appraisers agreed.  Pe is the expected percentage of agreement if the it was based upon random chance effects.  So, if there are four choices and all are equally likely, the Pe is 25%.  

The Fleis’ Kappa ratio has several beneficial attributes.  One is that it can be used with multiple appraisers, you just need to do the calculation for each pair.  Second, it does not require that all appraisers evaluate all the items.  The percentages will be based upon those items they have in common.

Hints & tips

  • When structuring you Gage R&R seek to avoid the special conditions mentioned.  They add complications which increases uncertainty in the results.
  • Don’t hesitate to use Minitab or other statistical support material when facing the special considerations.
Login to download
  • 00:04 Hi, I'm Ray Sheen.
  • 00:06 Now we've talked about how to organize a Gage R&R study.
  • 00:10 But there's some special considerations and cases that you need to be aware of
  • 00:15 these considerations could alter the nature of your Gage R&R design.
  • 00:21 I want to first discuss the differences between crossed and
  • 00:25 nested studies and when you would use each.
  • 00:28 Crossed studies are the most common, and in this case,
  • 00:31 every operator involved in the study tests every part that's used in the study.
  • 00:36 Everybody does everything.
  • 00:38 That means that the testing can't cause any damage or
  • 00:41 destruction to the test samples since they must be reused many times for many tests.
  • 00:48 With this type of test, we can separate out the different types of variation.
  • 00:53 Part to part variation can be separated from equipment variation.
  • 00:58 In addition, we can test for the human effect of repeatability and
  • 01:02 reproduceability.
  • 01:03 That means that we can separate out the part variation from the operative
  • 01:06 variation Each of these different factors can be isolated in the analysis.
  • 01:13 Now we'll contrast that with nested studies.
  • 01:16 In this case, the operators and
  • 01:18 the machines cannot all test all the same items.
  • 01:21 Typically this is because each test requires a unique test item such as with
  • 01:26 destructive testing and I don't mean that it blows up or burns up the test specimen.
  • 01:31 But if the nature of the test changes the test sample,
  • 01:35 then the sample cannot be tested multiple times.
  • 01:38 Another reason that may constrain the testing is if the physical size or
  • 01:42 some other attribute limits the ability to send the item to multiple locations.
  • 01:47 I don't just mean across country, you may have an automated process with
  • 01:52 multiple lines that each have inline inspection and test.
  • 01:56 In that case, you may not be able to route the same piece to multiple lines
  • 02:01 because the automation cannot accommodate it.
  • 02:04 One important consideration in this type of test
  • 02:06 is that parts being tested can be treated as homogeneous batch.
  • 02:11 So if it's a chemical process,
  • 02:13 the sample that is pull can be from any part of the batch that's mixed.
  • 02:16 There is an assumption that everything in the batch is identical but
  • 02:20 one thing we see here is that we don't have the situation
  • 02:24 where every operator and every machine tests every item.
  • 02:29 We strongly recommend that you use cross designs not nested.
  • 02:34 If you have to do a nested design, then I recommend you refer to Minitab or
  • 02:38 statistical book, to be certain that you've modified the equations accordingly.
  • 02:43 For what it's worth, I haven't had to do a nested design in years.I want to also
  • 02:48 take a few moments to discuss gage calibration and discrimination.
  • 02:52 We have another lesson coming up soon to address the type one gateway.
  • 02:55 Gage study, it can be done to do an independent analysis of these.
  • 02:59 But we can also do a quick check to help us in this area.
  • 03:03 It should go without saying, that you want the equipment to give accurate results.
  • 03:08 So an obvious element of preparing for
  • 03:09 a Gage R&R study is to make sure your equipment is calibrated.
  • 03:14 Also when the gage is initially put in service,
  • 03:17 a linearity study should have been done.
  • 03:19 Make sure that if you're using some parts out of spec in your study, those parts
  • 03:23 are not so far out of spec that they are beyond the region of linearity.
  • 03:28 Otherwise it will affect the accuracy of your analysis.
  • 03:32 Another quick check you can do is to calculate
  • 03:34 the number of distinct categories or NDC.
  • 03:38 Make sure your NDC will be appropriate for the type of study you are doing.
  • 03:43 NDC looks at the normal variability within the gage.
  • 03:47 If NDC is less than two, the gage cannot be used.
  • 03:50 If the NDC is between 2 and 5, the gage could be used for
  • 03:55 some attribute studies that just pick a pass fail or a low high value.
  • 04:00 If the NDC is between 5 and 14, then the gage is marginally acceptable
  • 04:05 depending upon the criticality of the parameter being measured.
  • 04:08 Finally, if the NDC is greater than 14, then the gage can be used for
  • 04:13 variable Gage R&R studies.
  • 04:15 The NDC calculation is very simple.
  • 04:18 It's the square root of 2 times the standard deviation apart measurements
  • 04:23 that are used in this study, divided by the standard deviation for the gage study.
  • 04:29 So when you complete your Gage R&R, you can check the NDC.
  • 04:32 If it's not greater than 5, you need to change your study parameters.
  • 04:37 Normally, the best way to do this is to use parts with a wider variation,
  • 04:42 use the full tolerance.
  • 04:45 The last special consideration is associated with attribute Gage R&R and
  • 04:49 the Feis' Kappa ratio.
  • 04:51 Typically, an attribute Gage R&R study we'll be using pass/fail data.
  • 04:56 The operators will separate the parts into two categories, good and bad.
  • 05:00 However, in some cases and inspection methods sorts the items into more than two
  • 05:05 categories, something like a good better best or think of rating something online,
  • 05:11 where you give it 1, 2, 3, 4 or 5 stars.
  • 05:15 That's also referred to as the Likert scale.
  • 05:18 The Fleis' Kappa will measure how well the various operators agree
  • 05:23 on which items belong in which category.
  • 05:25 The Kappa ratio can vary from + or- 1,
  • 05:29 +1 says that there's perfect agreement, -1 is perfect disagreement.
  • 05:35 And 0 means that the agreement is based upon random chance and
  • 05:39 the measurement system can't be trusted to measure anything.
  • 05:43 If your Fleis' Kappa ratio is not at least 0.7, don't even bother with a Gage R&R.
  • 05:48 The system is not capable.
  • 05:50 Ideally, you would want to Fleis' Kappa a 9 or higher.
  • 05:54 Incidentally, Fleis' Kappa can be used with multiple appraisers and
  • 05:58 they don't have to measure all the same items or valuate the same conditions.
  • 06:04 So let's look at this Fleis' Kappa ratio.
  • 06:07 It's pretty straightforward, you first need to calculate several percentages.
  • 06:12 One is to determine the percentage of time that the appraisers agree with each other.
  • 06:16 And the second is to determine the percentage of agreement you would expect
  • 06:20 if it was a random chance.
  • 06:22 So if for example, we were working with the five star scale,
  • 06:26 the random chance of two appraisers picking the same value one in five or 20%.
  • 06:32 If there were only two categories, the random percentage would be 50%.
  • 06:37 So once you have the two percentages,
  • 06:40 the ratio is the difference between the actual percentage minus the expected or
  • 06:45 chance percentage divided by one minus the expected percentage.
  • 06:51 As you prepare for
  • 06:52 your Gage R&R study, check to see if any of these special considerations apply.
  • 06:57 If so, you need to make sure that you've accounted for them.

Lesson notes are only available for subscribers.

PMI, PMP, CAPM and PMBOK are registered marks of the Project Management Institute, Inc.

Gift this course