HD6412340 HITACHI [Hitachi Semiconductor], HD6412340 Datasheet - Page 535

• Manufacturer Part Number: HD6412340
• Description: H8S/2345 F-ZTAT Hardware Manual
• Manufacturer: HITACHI [Hitachi Semiconductor]

522
Absolute precision
The deviation between the digital value and the analog input value. Includes the offset error,
full-scale error, quantization error, and nonlinearity error.
Figure 14.9 A/D Conversion Precision Definitions (1)
H'3FE
H'3FD
H'3FF
H'002
H'001
H'000
Digital output
1024
1
1024
2
Ideal A/D conversion
characteristic
Quantization error
1022
1024
1023
1024
Analog
input voltage
FS