Page 1 of 1

Format and precision of numeric values

Posted: Wed May 21, 2008 9:44 am
by Phil Jollans
Does the ISO-MME standard specify the format of the numeric values in the test channel files?

In particular, does it specify the number of decimal digits, or significant digits?

Is scientific format (with E, e.g. 1.003E-2) allowed, or required, or even disallowed?

Phil

Re: Format and precision of numeric values

Posted: Mon May 26, 2008 5:44 am
by derpmann
The ISO MME standard only differs in "Integer" and "Float".
For the values of sample "Float" is specified.
"Float" has no restrictions for the number of decimal digits or the number of significant digits.
The scientific format is not dissallowed, therefore it is allowed.

Peter