]> git.baikalelectronics.ru Git - kernel.git/commit
staging: comedi: usbduxsigma: fixing rounding error in the range struct
authorBernd Porr <mail@berndporr.me.uk>
Tue, 1 Jul 2014 21:14:42 +0000 (22:14 +0100)
committerGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Tue, 8 Jul 2014 23:27:29 +0000 (16:27 -0700)
commit4121306aa1d5dd44d05872a25bff713f468b3a11
tree6ba7f207209cfb0d108e410b9a4a53059c0ec904
parentada95e9850f2988469ba121f9240aec8dfb20e06
staging: comedi: usbduxsigma: fixing rounding error in the range struct

The AD converter on the usbsigma turns vref (2.5V/2) into the
binary value 0x780000. However the full scale range goes up to
0x800000 so that vref/2 needs to be upscaled by 0x800000/0x780000
which is 1.06bar. In the driver it's been 1.06 without the infinite
6666. I've put the fraction in the code so that the compiler can
calculate the most precise scaling value by itself.

Signed-off-by: Bernd Porr <mail@berndporr.me.uk>
Reviewed-by: Ian Abbott <abbotti@mev.co.uk>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
drivers/staging/comedi/drivers/usbduxsigma.c