ADC Resolution & Scaling Calculator

Formulas

• 1 LSB = V_REF ÷ 2^N (Microchip AN693)
• code = ⌊V_IN × 2^N ÷ V_REF⌋, maximum 2^N − 1 (ATmega328P §23.7, generalized to N bits)
• V = code × V_REF ÷ 2^N
• Current signal: V = I × R_shunt, voltage signal: V = Vin × divider ratio
• value = low + (code − code_low) ÷ (code_high − code_low) × (high − low)
• RMS quantization error = 1 LSB ÷ √12, max SNR = 6.02 × N + 1.76 dB, ENOB = (SINAD − 1.76) ÷ 6.02

Resolution is the number of codes you actually use

• A 12-bit ADC loses resolution when the signal does not span full scale. 4–20 mA into 150 Ω is 0.6–3.0 V, which uses about 2979 codes of a 3.3 V reference; for 0–10 bar one count is about 0.0034 bar.
• Codes below 4 mA (below 0.6 V) let you tell a broken loop (0 mA) or under-range apart, and the room above (3.0–3.3 V) takes readings slightly over 20 mA without clipping.
• With 250 Ω and a 5 V reference, 20 mA = 5 V is exactly full scale, so anything just above 20 mA sticks at the top code (2^N − 1). The calculator's «top clipped» warning is this case.

1 LSB by resolution

Resolution is not accuracy

• 1 LSB is the smallest step you can tell apart, not the accuracy. AN693 explains that even an ideal converter has up to ±½ LSB quantization error, and real converters add offset and gain errors plus DNL and INL (non-linearity).
• 6.02N + 1.76 dB is the theoretical maximum for a full-scale sine wave measured over 0 to fs/2. The ENOB computed from the datasheet SINAD is closer to the bits you can really use.
• Reference voltage error scales every code. Unless the measurement is ratiometric (sensor supply and reference from the same source), check the reference's accuracy and drift.

Input circuit

• The ATmega328P datasheet (§23.6.1) says its ADC is optimized for sources with an output impedance of about 10 kΩ or less, and advises removing components above the Nyquist frequency (f_ADC/2) with a low-pass filter. Pick the RC filter's cut-off with the RC time constant calculator.
• To read 0–10 V on a 3.3 V ADC, a divider ratio of 0.3 maps 10 V to 3.0 V with headroom. Choose standard resistor values in the voltage divider calculator.

FAQ

What is the resolution of a 12-bit ADC?

4096 steps. With a 3.3 V reference 1 LSB = 3.3 ÷ 4096 ≈ 0.806 mV; with 5 V about 1.22 mV.

How do I turn ADC counts into pressure or temperature?

Find the codes at the signal's low and high ends, then scale linearly: value = low + (code − code_low) ÷ (code_high − code_low) × (high − low). That is «Raw code → engineering value» above.

Does this work for PLC analog input values (0–27648, 0–32000)?

Those are already scaled by the module, so use that range instead of bits and reference. The 4–20 mA scaling calculator has the ranges we checked in S7, MELSEC and other manuals.

Which shunt resistor should I use for 4–20 mA?

Pick one so that 20 mA × R sits a little below the ADC reference (150 Ω → 3.0 V for 3.3 V). The shunt also takes voltage from the loop supply, so check the loop with the 4–20 mA loop load calculator.

For PLC counts (0–27648…) and NE 43 limits, use the 4–20 mA Scaling Calculator. For divider resistors, use the Voltage Divider Calculator. For the input filter, use the RC Time Constant & Cut-off Calculator. To amplify a small signal to the ADC range, use the Op Amp Gain Calculator.

Sources

• Microchip, AN693 Understanding A/D Converter Performance Specifications (DS00693A, 2000): code width and LSb, quantization error, eq. 2, 3, 5
• Microchip, ATmega328P datasheet (7810D, 01/2015): §23.6.1 Analog Input Circuitry, §23.7 ADC Conversion Result