Attribute Data Gage R&R Examples

Locked lesson.

Upgrade

About this lesson

This lesson walks through an example of an attribute data Gage R&R analysis. The example is demonstrated using manual data collection and equations with the Microsoft Excel spreadsheet application. The same example is then demonstrated using the statistical software program Minitab, which is often used for analyses with Lean Six Sigma projects.

Quick reference

Attribute Data Gage R&R Examples

An example of an attribute data Gage R&R analysis will illustrate the power of the technique. This example is demonstrated using comparison equations with a spreadsheet in Microsoft Excel. The same example is then demonstrated the statistical software program Minitab, which is often used to conduct analyses for Lean Six Sigma projects.

When to use

The example illustrates the technique. When doing your first few Gage R&R Studies, you will want to refer to the example to guide you through the process.

Instructions

This example will be based upon the data set shown below. It is for a study with three Appraisers, 25 items, and two trials.

Repeatability calculations are first conducted at the individual level first. This is done by determining the number of times an appraiser is consistent in their evaluation (which is the match column) divided by the number of items, and then converted to a percentage.

  • Appraiser #1 is 23/25 = 92%
  • Appraiser #2 is 22/25 = 88%
  • Appraiser #3 is 25/25 = 100%

The System Repeatability is the average of those three: (92% + 88% + 100%)/3 = 93.3%

 

Reproducibility is the calculation of the number of times all Appraiser #1 = all Appraiser #2, all Appraiser #1 = all Appraiser #3, and all Appraiser #2 = all Appraiser #3 divided by three times the number of items and then converted to a percentage.

Reproducibility = (21 + 23 + 22) / 75 = 88%

 

The accuracy measures will include the use of the Reference Standard column in the comparisons. First is the measure of accuracy, which is the number of times that each appraiser correctly assessed the item value divided by the total number of inspections and then converted to a percentage.

  • Appraiser #1 – first pass = 24 times, – second pass 22 times
  • Appraiser #2 – first pass = 22 times, – second pass 23 times
  • Appraiser #3 – first pass = 24 times, – second pass 24 times

Total Accuracy = 139 / 150 = 92.7%

 

The Miss Rate is based upon the Appraiser assessing a “Pass” for what should be a “Fail.” The Miss Rate is the number of times that occurred divided by the number of assessments of units whose true status was “Fail” and then converted to a percentage.

  • Appraiser #1 = 2 occurrences, Appraiser #2 = 4 occurrences, Appraiser #3 = 2 occurrences
  • Miss Rate = 8 / 66 = 12.12%

 

False Alarm Rate is based upon the Appraiser assessing a “Fail” for what should be a “Pass.” The False Alarm Rate is the number of times that occurred divided by the number of assessments of units whose true status was “Pass” and that converted to a percentage.

  • Appraiser #1 = 2 occurrences, Appraiser #2 = 1 occurrence, Appraiser #3 = 0 occurrences
  • False Alarm Rate = 3 / 84 = 3.57%

 

The system effectiveness is the number of times that all appraisers agreed on all trials with each other and the reference standard for an item divided by the number of items and that converted to a percentage.

System Effectiveness = 20/25 = 80%  

 

 

Minitab provides a wizard for measurement systems analysis that is found under the "Assistant" pull down menu. When selecting Measurement Systems Analysis the wizard panel shown below appears.

  • Attribute data analysis is on the right side of the first diamond and is labelled "appraisal". 
  • Whichever path you choose, you then have two options, either to set up the study which will create the data collection form for you, or the analysis side which will perform the calculations.
  • When setting up the study, you will need to provide the true status for each item in order for Minitab to create the Reference Standard column.

  • When selecting the analysis side for attribute data, a second panel is displayed. This panel will ask where information is found in your Minitab workbook.
  • The Minitab workbook operates similar to an Excel spreadsheet – in fact you can directly copy and paste data form one application to the other. 
  • One limitation within Minitab, is that it almost always requires the information to be in columns not rows. 
  • If you used Minitab to set up the study, the form it created will be organized for easy analysis. 
  • You will need to tell Minitab which columns are your appraiser names, your item numbers and of course your data. Be sure to select which of the entries is considered good in the “Value of good or acceptable items” window.

  • If you are not familiar with Minitab, the way to select a column for one of their entry fields is to first place your cursor in the field. This will cause all available columns that are not already assigned to be shown in the window on the left. 
  • Highlight the column you want to use for the selected field, then drop just below the window listing all columns and click on the Select button. That column name will now be in your field. 
  • Do this to select a column name for every open field on the panel. 
  • Then when ready, click the OK button at the bottom of the panel. Minitab will soon give you results.

The Minitab results are both graphical and numerical. It provides a summary of the analysis, but Minitab will break down the data into further detail in additional graphs. These show detail by operator, by trial and even by item.

Hints & tips

  • If planning on doing the analysis in Minitab, do the setup with it also and use the Minitab form. The analysis then takes about 15 seconds to select the columns and see the result.
Login to download
  • 00:04 Hi I'm Ray Sheen, it's now time for
  • 00:07 part two of lesson that we have to analysing attribute Data Gage R&R study.
  • 00:12 And this time we'll work through an example.
  • 00:15 So since we're working an example of how to do the analysis,
  • 00:19 let's start with the data.
  • 00:21 As the study progressed, you recorded the data and the study form or format that
  • 00:25 you created, in particular filling in that trial column on the spread sheet.
  • 00:30 Once a data was collected, I complete the match column.
  • 00:33 So if the appraiser agreed with themselves about an item,
  • 00:37 I enter a 1, if not enter 0.
  • 00:39 We use this column for repeatability and reproducibly analysis.
  • 00:43 The next column over was Match Standard, in this case I do a further check to see if
  • 00:48 the appraiser both agreed with themselves and matched the reference standard.
  • 00:53 We'll use that in some of the accuracy and effectiveness analysis.
  • 00:57 Once the analysis is done, add up the totals in each column.
  • 01:01 So let's run through the calculations now and see what we get.
  • 01:04 First, we calculate individual repeatability.
  • 01:06 Take the total from each appraiser's match column and divide by the number of items.
  • 01:12 We see that appraiser 1 is 92%, appraiser 2 is 88%, and appraiser 3 is 100%.
  • 01:19 If we average these three values, we get an overall system repeatability of 93.3%.
  • 01:25 Now for reproducability,
  • 01:27 we need to find how many times two appraisers agreed with each other and
  • 01:32 were consistent across all their trials that did not occur on four of the items.
  • 01:38 With item 1, appraiser did not agree with appraiser 2 or 3.
  • 01:41 With item 14, nobody agreed with anybody else.
  • 01:45 Item 18, appraiser 2 did not agree with appraisers 1 or 3, and
  • 01:48 the same situation occurred on item 20.
  • 01:51 So in the end,
  • 01:51 we had 66 pairs where the appraisers agreed with themselves and each other,
  • 01:57 divide that by 75 pair opportunities to give us a reproducibility of 88%.
  • 02:02 So now let's look at the accuracy calculations.
  • 02:06 First is to check to see if an appraiser's assessment of an item agrees with
  • 02:10 the standard.
  • 02:11 This is done for each individual assessment.
  • 02:14 This occurred on 139 occasions out of the possible 150, that is 92.7%.
  • 02:21 Next was the miss rate, based upon the reference standard columns,
  • 02:25 there were 11 items that were fail.
  • 02:27 Well since these appraisers each did two trials,
  • 02:31 there were 66 times that the correct answer would be fail.
  • 02:35 Unfortunately, this did not occur on eight occasions.
  • 02:38 Most notably, item nine where all appraisers passed it on all trials, but
  • 02:42 they should have failed it.
  • 02:44 So we add up the total of 8, divide by 66 and we get 12.12% for our miss rate.
  • 02:51 On to the false alarm rate,
  • 02:53 based upon the reference standard there were 14 items that were passed.
  • 02:57 Since three appraisers each did two trails there were 84 times
  • 03:01 the correct answer would be pass.
  • 03:04 On three of those occasions this did not occur, so
  • 03:06 the false alarm rate is 3 divided by 84 which is 3.57%.
  • 03:08 Finally, let's look at overall system effectiveness.
  • 03:15 I checked the match standard column for all items.
  • 03:18 If the match standard is one for all three appraisers, that means they all agree with
  • 03:22 themselves, each other and the standard and that is an effective item.
  • 03:27 When I do this, I find that there was a problem on items 1, 9, 14, 18, and 20.
  • 03:33 This means that there were 20 effective items out of 25 or
  • 03:37 an effectiveness of 80%.
  • 03:39 Okay, time to switch to Minitab.
  • 03:42 This starts the same way we did with variable data Gage R&R.
  • 03:46 Go to the Assistant pull down menu and select Measurement System Analysis.
  • 03:49 You should get this panel, you could choose to Set up or analysis path.
  • 03:53 And I will do wanna take a minute to talk about set up for this study.
  • 03:57 Within Minitab we need to do a bit more work to set up an attribute data Gage R&R
  • 04:02 than for a variable data Gage R&R.
  • 04:04 When you selected set up on the previous panel, you should see this panel up here.
  • 04:07 There are more fields on this panel than on the variable data Gage R&R panel.
  • 04:13 Start by entering the number of operators, trials, items, and
  • 04:16 if you want to personalize it even more, put in the operator names.
  • 04:20 Then define the term that you're going to use for
  • 04:22 an acceptable and unacceptable performance.
  • 04:25 It may be pass, fail, it may be green, red, make it easy for
  • 04:29 the person doing the recording of the data.
  • 04:31 Now for the part that takes a little more work.
  • 04:34 You have to set the value for the reference standard for each item.
  • 04:38 Make sure you have the right value with the correct item number.
  • 04:41 Finally, click OK and Minitab will generate your study input form.
  • 04:45 Now we're ready to talk Minitab analysis.
  • 04:48 So if you've collected all your data, then go back to the Assistant,
  • 04:52 select the Measure Systems Analysis.
  • 04:55 But this time, select the icon at the end of the Analyze Path on the appraisal side
  • 04:59 of the decision tree.
  • 05:00 And you should get a panel in Minitab that looks like this.
  • 05:02 Select the columns for your Appraisers, your Trials, your items,
  • 05:07 your Results, and your Reference.
  • 05:10 If you aren't familiar with using Minitab,
  • 05:12 recall we do this by placing our cursor in the parameter field.
  • 05:15 That brings up the columns list in the window on the left side of the panel.
  • 05:19 Highlight the correct column and
  • 05:21 click on the Select button that is below that window.
  • 05:23 This moves the column to the parameter field.
  • 05:26 Finally, when all the parameters are set, click on OK to see the results.
  • 05:31 So let's look at the results.
  • 05:32 Minitab will provide results in several format with lots of graphs and charts.
  • 05:37 I'm just showing the summary panel here.
  • 05:40 The Minitab results will focus more on accuracy than repeatability and
  • 05:44 reproducibility.
  • 05:45 Using the values in the upper right,
  • 05:47 the repeatability score is just one minus the mixed rating.
  • 05:52 Minitab does not provide a reproducibility score as a standalone value.
  • 05:56 However, it does calculate accuracy.
  • 05:59 That is what is shown on the sliding scale on the upper left.
  • 06:02 The miss rate is fail rated as pass and
  • 06:05 the false alarm rate is pass rated as fail.
  • 06:09 These values are the same we found with our equations in Excel.
  • 06:12 Well, as this example shows,
  • 06:15 Attribute Data Gage R&R calculations are simple and easy to perform.

Lesson notes are only available for subscribers.

PMI, PMP, CAPM and PMBOK are registered marks of the Project Management Institute, Inc.

Gift this course