Hi again,
I fear the aoRecord is 'infected' by the same kind of disease, as one
can see by looking into the init_record() and convert() functions. In
the first case the solution is the same as in the aiRecord, replace
val = pai->rval + pai->roff;
in init_record() by
val = (double)pai->rval + (double)pai->roff;
In the second case (convert()) it goes the other way around: First val
is streched according to eslo as in
value = (value - pao->egul) / pao->eslo;
Afterwards roff is subtracted:
pao->rval -= pao->roff;
As in the aiRecord, if egul and eguf are assuming 32 bit raw values,
this can lead to an overflow. Proposed solution: replace the last 4
lines of convert():
if (value >= 0.0) pao->rval = value + 0.5;
else pao->rval = value - 0.5;
pao->rval -= pao->roff;
return;
by these three:
if (value >= 0.0) pao->rval = value - pao->roff + 0.5;
else pao->rval = value - pao->roff- 0.5;
return;
As in the aiRecord, this imposes a slight loss of efficiency (in case of
less than 32 bit raw values) because of the additional conversion of
pao->roff from long to double. Don't know how many nanoseconds (or, more
interesting, how many processor cycles) that takes. I assume one can
afford it in most cases.
Caution: I didn't test the code. Don't trust me blindly.
Ben
--
The Notorious Neb Nesknarf
// snail: BESSY II, Rudower Chaussee 5, D-12489 Berlin, Germany
// email: [email protected]
// phone/fax: +49(30)6392-4865 / 6392-4859
- Replies:
- Re: ai record support has an error Marty Kraimer
- References:
- ai record support has an error Benjamin Franksen
- Re: ai record support has an error Marty Kraimer
- Navigate by Date:
- Prev:
Subarray records with a minummum profile Hammonds, John
- Next:
dm byte widget Sean Prior
- Index:
1994
1995
1996
<1997>
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
Re: ai record support has an error Marty Kraimer
- Next:
Re: ai record support has an error Marty Kraimer
- Index:
1994
1995
1996
<1997>
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
|