Measuring a measuring system. About MSA

Checking an artifact with measuring instruments that have been previously calibrated and certified does not necessarily mean they are still suitable for the intended use.

Calibration certifies the instruments were compliant at the time and place it was performed, not necessarily that they still are. More importantly, calibration attests that a measuring instrument is suitable for use within a specific context and for checking certain product characteristics.

Let's say you are a mechanical components manufacturer and the customer is checking the items you just delivered. They will do so with their own instruments – perhaps even of the same type and brand as yours – which have been most likely recently calibrated. But since the control occurs under different conditions, it is likely to produce different measurement values.

This article aims at providing a simple guide on how to know the behavior of our measuring instruments within the stated environmental conditions and measurement process.

We measure our products by following established procedures and methods, which are – by all intents and purposes – processes and, as such, can be measured likewise. It is reasonable to assume that if we always measure the same characteristic in the same way, any inconsistencies we might incur would be entirely attributable to such characteristics.

But is this always true?
No, not necessarily!

The readings obtained may or may not be correct. We cannot be entirely sure, because we don't know how much the instrument has affected these results. In other words, should the operator, place, time of day, state of preservation, use, and maintenance of the instrument be different, we might get readings that are seemingly correct but ultimately are not—or vice versa.

So what can we do?

In order to answer this and other similar questions in different application areas, Chrysler Group, Ford Motor Company, and General Motor Corporation established in 1982 a coordinating group called the Automotive Industry Action Group (AIAG). AIAG is a non-profit organization that brings together automotive companies and component suppliers. Its goal is to find common solutions and methods to increase efficiency and reduce costs while providing guidelines on their applications.

The activity of committees and study groups has led to some guidelines, adequately described in a series of manuals published by the same body. It's also worth noting that these are not outright regulations but rather sets of clear and detailed instructions on what to do and how to do it, within the context of several specific topics—including measuring instruments.

The first edition of the MSA (Measurement System Analysis) manual was released in October 1990, the second one in June 1998, the third one in March 2002, the fourth one in June 2010, followed by other updates and expansions in the years to follow. The MSA suggests the methods and rules to be followed to typify measuring instruments in the situation they are used in. The operators, the way they are maneuvered, the environment, and the context can significantly affect the measurements. Many methods have been recommended and applied by various manufacturers and/or users, each providing its own solution.

To ensure the reliability of measured values, AIAG suggests carrying out a statistical analysis of the measurement process taking into account all the elements involved, such as the parts, the operators, the procedures, as well as – of course– the gauges. The variables coming into play during a measurement process are similar to those of a production process, and as such must be handled. These numbers, managed in a structured and statistical way, will provide us with indicators that will identify the instrument and the measurement method.

But first, let’s clarify the terminology used.
The meaning of:

• Measurement: Eng. C. Eisenhart (1963) defines “measurement” as “the assignment of numbers to material things to represent the relationships among them with respect to particular properties.”
• Gauge, or gage: any entity used to record measurements; frequently used as a specific reference for devices used in manufacturing, including go/no-go gauges.
• Measurement system: the set of related operations, procedures, tools, and other equipment, software, and personnel used to assign a number (value) to measured characteristics; the complete process of obtaining measurements.

What are the characteristics that typify an instrument:

• Instrument resolution: is the smallest scale of measure or output that can be detected by the instrument.
• Reference value: the acceptable measurement for each artifact. It is typically used as the surrogate for the true value.
• True value: the actual value of an artifact with a 0% measurement error. By definition, it is unknown and unknowable.
• Accuracy: a generic concept of exactness related to the closeness of agreement between the average of one or more measured results and a reference value.
• Bias: the difference between the observed average measurement and a reference value.
• Stability: the variation in bias over time.
• Linearity: difference of bias throughout the expected operating (measurement) range of the equipment.
• Precision: it describes the net effect of discrimination, sensitivity, and repeatability over the operating range (size, range, and time) of the measurement system.
• Repeatability: the variation in measurement taken by a single operator with a single instrument on a single sample.
• Reproducibility: is the variation observed when different operators measure the same part with the same instrument.
• GRR (Gauge, R&R): is an estimate of the combined variation of repeatability and reproducibility.
• Measuring system capability: the measurement system (random) error over a short
period of time.
• Measuring system performance: the effect of all sources of variation over time.
• Sensitivity: is the smallest input that can generate a detectable output signal.
• Consistency: the difference in the variation of the measurements taken over time. It may be viewed as repeatability over time.
• Uniformity: the difference in variation throughout the operating range of the gauge.
• Capability: an estimate of the combined variation of measurement errors (random and systematic) based on a short term assessment.
• Performance: the net effect of all significant and determinable sources of variation over time.
• Uncertainty: estimated range of values about the measured value in which the true value is believed to be contained.
• Traceability: the ability to find the identification data and results of the studies performed and to repeat them under the same conditions.

All of these data combine to give a clear and complete identification of the measurement process.

In 1986, Deming stated that the “ideal” measurement system would produce only correct measurements each and every time a measurement is made. Each measurement would conform to a master reference standard. However, a measurement system capable of producing these results would have zero variance, zero bias, and ultimately zero probability of error. It is therefore clear that only controlling the production in toto would guarantee the condition of zero defects, but in the industrial world, complete control is hardly feasible.

In a normal production context, this goal is unattainable, as the time necessary for the controls would heavily impact the cost of the product. For this reason, we resort to repeated sampling, treating the detected values statistically.

Statistical process control according to pre-established criteria allows us to obtain representative indicators of the production stability, predicting how many parts per million might not be compliant to specifications, should the production continue unchanged.
Likewise, it is possible to measure the reliability of the measurement process and obtain the same type of indicators.

Measuring processes can analyze both variable characteristics (all those characteristics that can be measured) and attributive characteristics (all those characteristics that can be counted).

In this article, we will discuss only the measurement of variable characteristics.

The elements involved in the measurement process must be clear, defined, and repeatable, namely:

‣ Clear and complete identification of the part and feature to be measured and in which process
‣ Identification of the instrument and calibration status
‣ List of characteristics to be measured
‣ Choice of the type of study you want to perform (short or long)
‣ Date, time, and environmental conditions
‣ Name and number of the operator(s) who will carry out the study
‣ Number of measurements to be performed
‣ Control plans

In order to analyze the measurement system, measurements or sampling of an established size will have to be carried out a number of times, by a number of operators, and with a frequency as recommended by the AIAG manual.

From this analysis, we will have determined how much the examined instrument may have affected the observed values measured on our manufactured product.
Should the same part be measured in another place with another instrument by another operator, it is useful to know how much the instrument can influence the values measured, in the context in which it is used.

The artifacts could even "seem" wrong but not necessarily be so, because the differences detected could be attributable to the gauges or methods used.

Finally, it should not be underestimated that the state of preservation, management, and maintenance of the measuring instrument – as well as its conditions at the time of use – can heavily influence the quality of measurements.

Measuring a product with "unreliable" instruments and procedures is equivalent to not knowing what we are manufacturing.

As always, this topic is quite articulated and would require more space: in this article, we have only given some basic information. In any case, further insights are always available.

Giovanni Stagni
Fasteners Consultant

Ed. For comments and feedback on this article, please email us at [email protected]