Experimental Physics and
Industrial Control System

Eric Norum wrote:

On May 30, 2012, at 10:25 AM, Tim Mooney wrote:

…..

It seems that the integer to double conversion really is not exact
for 1.

That seems like a pretty huge flaw in the architecture!  I've never
encountered a machine with arithmetic that bad!

I have experienced very bad rounding errors (in a gensub, in C code)
on a ppc604/mv5100 - to the point that I had to re-order the arithmetic.
Bad arithmetic isn't unheard of.

Aloha,
Maren

Replies:

Re: calc VAL field not updating from bi VAL field Eric Norum

References:

Re: calc VAL field not updating from bi VAL field Tim Mooney

Re: calc VAL field not updating from bi VAL field Eric Norum

Navigate by Date:

Prev: Re: calc VAL field not updating from bi VAL field Tim Mooney

Next: Re: calc VAL field not updating from bi VAL field Eric Norum

Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <2012>  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 

Navigate by Thread:

Prev: Re: calc VAL field not updating from bi VAL field Eric Norum

Next: Re: calc VAL field not updating from bi VAL field Eric Norum

Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <2012>  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024