Blogs
Sign-up to receive the latest articles related to the area of business excellence.
The Right Way to Report Measurements - With Uncertainty
View All Blogs
When we make any measurement, we can never be certain what the result exactly is. There is always an uncertainty associated with the measurement. If the uncertainty is small compared to the intended purpose, then we can confidently use our measurements to make good decisions. If the uncertainty is large, then we may end up making poor decisions. Hence, it is important to quantify the uncertainty when measurement values are being reported so that users can have a good idea how much to trust the measurement results. Often times in literature, measurements are reported without accompanying uncertainty numbers. For example, the speed of the car is 60 kmph, the length of a critical dimension is 2.54 mm etc. If we had incorporated uncertainty into our measurements, then we would report for example, the speed of the car is 60 ± 5 kmph, or the length is 2.54 ± 0.01 mm. In this article, we will talk about what causes uncertainty and how to calculate and report uncertainty along with our measurements.Any measurement that you make without the knowledge of its uncertainty is completely meaningless - Prof. Walter Lewin
What Causes Uncertainty
The measurement uncertainty can be caused by a number of factors.
- Instrument: Maybe the measuring instrument is not capable to begin with or deteriorates with age, wear, noise etc. This may be an issue if the instrument is not calibrated.
- Process: The measurement process may not be appropriate, especially if the standard way of measurement is not defined, each operator may perform the measurement differently contributing to the problem.
- Operator: The operator skill and/or knowledge may be a cause of the problem especially for difficult or complex measurements. This may be caused by poor training and lack of adherence to standards.
- Environment: Factors such as changes in temperature, humidity, pressure or other conditions which may affect the parts being measured and hence the measurement results.
- Sampling: If the sampling is not properly defined or adequate, the measurement errors may not be adequately captured.
The difference between the true value and the measurement is called measurement error. Many a times the true value is not really know, so we device various tests to determine the measurement errors. The measurement errors can be classified under the following 3 categories:
- Accuracy: Is the average of your measurements close to the true value. In order to check for accuracy of the measuring instrument, users may perform a calibration study where the results of the measurements are compared with a known standard. The results of such a study may result in determining the average bias (difference between the average measurements and the true value) and linearity. Such studies are usually called gage bias and linearity study.
- Repeatability & Reproducibility: Is there variation in your measurements due to instrument or person. In order to check for this, users may perform a Gage R&R study where the same measurement is repeated multiple times by one or more operators and the variation in the measurements may be reported. For example, there is a 10% variation in the measurement results. Such studies are usually called the Gage R&R study.
- Resolution: Is there an issue with the least count of your measurement instrument? The number of decimal places of the measuring instrument is an example of resolution. It is usually recommended that there are at least 10 divisions between the lower and upper specification limits in order for a measuring instrument to have sufficient resolution.
In several projects that I have encountered, users conduct these studies sort of independently. For example, they may perform a calibration of the measuring instrument every few years with a known standard and report the average bias or linearity, or they may perform a Gage R&R study and report the overall % GR&R value, or they may report the least count of the measuring instrument. These numbers are usually reported separately. What we are recommending is that you need to perform all these types of studies and report a single comprehensive number for measurement uncertainty for your measurements.
If you incorporate uncertainty is your measurements, you will express your answer as a range of possible values rather than as a single value. For example, we are 95% confident that the true measurement value for length lies between 2.54 ± 0.2 mm. Which means that the true measurement can lie anywhere from 2.52 to 2.56. We can also report the uncertainty as a %, such as the relative uncertainty is 0.2/2.54*100 = 7.8%.
How to Estimate Standard Deviation
The variation in the measurements can be quantified using the sample standard deviation. There are two methods in which we can estimate the standard deviation. In the Type A estimate, we take repeated measurements and then estimate the standard deviation of the average using the formula shown below. In the Type B estimate, we can estimate the standard deviation if we know the range of the measurement values and know the distribution the measurements fall into (for example uniform, normal etc.). The range (R) of measurements is nothing but the maximum minus the minimum value.
| Source | StdDev | Notes |
|---|---|---|
| Type A | s/√n | Take n repeated measurements and calculate the mean and standard deviation (s) |
| Type B | R/√12 | If we know the range of measurement values (R) and if the distribution is uniform. If the distribution was normal, then we would use a different constant to obtain the standard deviation. |
How to Calculate Uncertainty
Brainstorm with your team to determine all the possible contributors to the uncertainty in measurement. The uncertainty may come from the measuring instrument, people performing the measurement, environmental changes, resolution of the instrument etc. Each measurement situation is different, and you need to use the appropriate calculation for determining the uncertainty. For each source of uncertainty, determine the amount of variation or uncertainty in the measurement values. The standard deviation of the values is a measure of uncertainty of each source. It if is assumed that all possible sources of error are independent, the combined estimate of the standard deviation is given by the following formula:
The above formula is basically the root sum of squares when all the terms are additive. If there are more complex ways in which the final formula is derived, we would have to use the appropriate formula to combine the uncertainties. In order to use the above formula, make sure that all the measurements are expressed in the same units.
Once the combined standard deviation due to all uncertainty sources is known, the overall uncertainty of measurement can be obtained by using the expanded estimate based on the degree of confidence required in the measurement results. For example, if we want a 95% confidence in our measurements, then the scale factor = 2. If we want a 99.73% confidence, then the scale factor = 3 and so on. Hence, for a 95% level of confidence, the measurement uncertainty is reported as:
Once we compute the range for the measurements with uncertainty, we can report that we are 95% confident that the true value of measurement lies within this range. Note that we cannot precisely say what the true measurement is - this is the limitation of all measurements. Ideally, we want a small enough range of uncertainty for our measurements that we can live with.
We can also perform an uncertainty budget analysis to determine which are the major contributors for the uncertainty in our measurements. In order to perform this analysis, we use the variances (which are the square of the standard deviation) to perform the analysis. The uncertainty budget helps us identify which is the biggest contributor for the uncertainty in the measurements and we can take actions to reduce this value in order to make our measurements more precise.
Example
We want to measure the length of a critical dimension. The specification for this dimension is 10 ± 0.5 mm. This implies that the lower specification limit is 9.5 mm and the upper specification limit is 10.5 mm. We perform a measurement and determine that the mean value is 10.2 mm. We want to calculate the measurement uncertainty for this measurement.
We repeat the measurement multiple times by different operators and calculate the mean value as 10.2 mm and the measurement standard deviation of 0.05 mm. The calibration of this measurement device had provided an uncertainty of 0.5% with no significant average bias. Since, the nominal value is 10 mm, this translates to a standard uncertainty of 0.025 mm. The measuring instrument has a resolution of ± 0.01 mm. We assume this to be uniformly distributed, hence the standard deviation can be estimated by dividing this by √12. This gives a standard uncertainty of 0.0057 mm. Thus, the combined standard uncertainty (if no other factors are present and the errors are independent) is: √(〖0.05〗^2+〖0.025〗^2+〖0.0057〗^2 )=0.056 mm. The expanded uncertainty at a 95% confidence is given by 0.056 * 2 = 0.112 mm. Hence, we can write down the measurement result as, we are 95% confident that the measurement is: 10.2 ± 0.1 mm.
We can plot this result as shown in the figure below. The two dotted lines show the specification limits and the nominal value of the measurement is shown by the blue dot and the measurement uncertainty is shown by case C. For this measurement, since the uncertainty bounds are within the specification limits, we would conclude that the measurement is within the acceptable limits.
If the measurement values were as shown by case A or case E, then the measurement would not be acceptable since all values of the measurement lie outside the specification limits. For case B and case D, we cannot conclude if the measurement is acceptable because the confidence interval for the measurement straddles the specification limits. If we had not computed the uncertainties, we would have concluded that cases A, D, and E are unacceptable while cases B and C were acceptable.
The uncertainty budget for the above example is shown in the following table:
| Source | Std Dev | Variance | Budget |
|---|---|---|---|
| Calibration uncertainty | 0.025 | 0.000625 | 20% |
| Gage R&R | 0.05 | 0.0025 | 79% |
| Resolution | 0.0057 | 0.00003249 | 1% |
| 0.003157 | 100% |
From the uncertainty budget we can see that the biggest contributor to uncertainty is the Gage R&R value followed by accuracy. The resolution of the instrument is an insignificant contributor to the overall uncertainty.
In conclusion, we recommend that whenever we report measurement values, always determine the associated uncertainty in measurement and report the measured values as a range with a given level of confidence instead of as a single point measurement.
Follow us on LinkedIn to get the latest posts & updates.
