Instrument calibration error: as-found checks in % of span
How to calculate as-found error in engineering units and percent of span, compare it with a tolerance, and record rising and falling test points.
What as-found error means
An as-found check compares what an instrument reads against a known applied reference before any adjustment is made. The difference is the as-found error. After adjustment or repair the check is repeated and the result is the as-left error. Keeping both values shows whether the instrument drifted since the last calibration and whether the adjustment was effective.
The instrument calibration error calculator takes the applied reference, the instrument reading, the lower and upper range values (LRV and URV) and an acceptance tolerance in percent of span. It returns the error in engineering units, as a percentage of span, as a percentage of the reference, the maximum permitted error and a within or outside tolerance result.
Formulas the calculator uses
- Error = reading − reference
- Span = URV − LRV
- Error (% of span) = error ÷ span × 100
- Permitted error = tolerance (% of span) × span ÷ 100
- Within tolerance if |error (% of span)| ≤ tolerance (a tiny rounding allowance of 1×10−9 is added)
- Error (% of reference) = error ÷ reference × 100, not defined when the reference is zero
A positive error means the instrument reads high. The reference, reading and range must all use the same unit. The tolerance is taken from the approved procedure or manufacturer specification. The tool handles only percent of span; a tolerance expressed as percent of reading or in engineering units must first be converted to a percentage of span or checked separately.
Worked example 1: a temperature transmitter within tolerance
A temperature transmitter is ranged 0 to 200 °C. A dry-block or calibrator applies 100 °C and the instrument reads 100.8 °C. The acceptance tolerance is ±0.5 % of span.
- Span = 200 − 0 = 200 °C
- Error = 100.8 − 100 = 0.8 °C
- Error (% of span) = 0.8 ÷ 200 × 100 = 0.4 %
- Permitted error = 0.5 ÷ 100 × 200 = 1.0 °C
- Error as % of reference = 0.8 ÷ 100 × 100 = 0.8 %
The calculator reports within tolerance, because 0.4 % is not more than 0.5 %. Note that the same 0.8 °C is 0.8 % of the reference but only 0.4 % of span, so the basis of the tolerance matters. If the specification were instead ±0.5 % of reading, the permitted error at 100 °C would be 0.5 ÷ 100 × 100 = 0.5 °C and the same 0.8 °C error would fail. The calculator would still report the percent-of-span answer, so convert the tolerance or check that basis by hand.
Worked example 2: a pressure test point outside tolerance
A pressure transmitter is ranged 0 to 100 kPa. At a 75 kPa applied reference it reads 75.6 kPa. The tolerance is ±0.5 % of span.
- Span = 100 kPa
- Error = 75.6 − 75 = 0.6 kPa
- Error (% of span) = 0.6 ÷ 100 × 100 = 0.6 %
- Permitted error = 0.5 kPa
The calculator reports outside tolerance, since 0.6 % exceeds 0.5 %. At 0 kPa with a reading of 0.3 kPa the error is 0.3 % of span and within tolerance, but the percent-of-reference line then shows that it is not defined because the reference is zero. This is one reason percent of span is the usual basis for a zero test point.
The same test can be done in the current domain. For a 4–20 mA output on a 4 to 20 mA “range”, an applied 12 mA reference with a 12.1 mA reading gives 0.1 mA error, 0.625 % of span and a permitted error of 0.04 mA for a 0.25 % tolerance, which is outside tolerance. Use the 4–20 mA to engineering units calculator or the engineering units to 4–20 mA calculator to convert between process value and expected current before comparing.
Test points and a simple record layout
One point does not describe an instrument. A typical check covers several points across the range, taken rising and then falling, so that hysteresis shows up. The calculator evaluates one point at a time. The table is an example layout using the two pressure points from the examples above; a real record would list every test point in both directions, for example at 0, 25, 50, 75 and 100 % of span rising and then falling.
| Applied (kPa) | Reading | Error (kPa) | Error (% span) | Result |
|---|---|---|---|---|
| 0 | 0.3 | 0.3 | 0.3 | Within 0.5 % |
| 75 | 75.6 | 0.6 | 0.6 | Outside 0.5 % |
The worst point decides the instrument result. Hysteresis is the difference between the rising and falling reading at the same applied value; recording both directions is the only way to see it. A zero or span adjustment normally shifts all points, while a linearity problem shows up as a pattern that changes along the range.
Common mistakes
- Using the wrong basis: a percent of reading tolerance is not the same as percent of span.
- Mixing units, such as a reference in bar and a reading in kPa.
- Entering an upper range value lower than or equal to the lower range value; the tool rejects it.
- Checking only one point and calling the instrument good.
- Adjusting the instrument before recording the as-found value.
- Using a reference standard that is not traceable or not clearly better than the instrument under test.
- Ignoring the sign of the error, which indicates the direction of drift.
Checks before you trust the result
- Confirm the range and the engineering unit match the instrument configuration, not only the nameplate.
- Take the tolerance from the approved procedure or manufacturer specification for the instrument.
- Allow the reference and instrument to stabilise before reading, and note the ambient and installation conditions.
- Compare rising and falling points and look at the worst case.
- Remember the result is an estimate. It is not a calibration certificate and does not include measurement uncertainty of the reference, readout or procedure.
Frequently asked questions
Why is the tolerance in percent of span?
It is a common way to state transmitter accuracy because the allowed error is the same across the range. The procedure or datasheet defines the basis actually applicable.
What should I do with an outside-tolerance result?
Follow the approved procedure for adjustment or repair, record the as-found and as-left values, and have the procedure owner assess the effect on the process data collected since the previous check.
Does a pass mean the instrument is accurate everywhere?
No. It only covers the tested point and condition.
Why can the percent-of-reference line show as not defined?
Dividing by a zero reference is not possible, so the tool states that the value is not defined at a zero test point and relies on percent of span.
Recording data
Record the tag, range, tolerance, each applied reference and reading, as-found and as-left errors, reference standard identifier, certificate number, date and person. The calibration record template holds these items together with the interval and a due date. Keeping the as-found result for every visit lets a recurring drift in one direction be seen across several calibrations, which can justify a shorter interval or a replacement decision under the site procedure.