The PRECISION statement determines the decimal precision of numeric values used in the current program.
Format
|
PRECISION n |
Parameter(s)
|
n |
An expression evaluating to the number of fractional digits to which numeric values is calculated. Any fractional digits in the result of a conversion that exceed n are rounded off. n may be any integer from 0 through 9, with a default value of 4. A precision of zero (0) implies that all values are treated as integers throughout the program. |
Description
Precision can be declared anywhere within the program.
|
NOTE |
Only one PRECISION declaration is allowed in a single program; if more than one is encountered, a warning message is printed and the second PRECISION declaration is ignored. However, if external subroutines are called, each external subroutine should include a PRECISION statement matching that of the main program. If the precision differs between the calling program and the subroutine, a warning message is printed and the second declaration is ignored. |
Trailing fractional zeros are dropped during output. Therefore, when an internal number is converted to an ASCII string, the result might appear to have fewer decimal places than the precision setting allows. However, regardless of the precision setting, the calculation always reflects the maximum accuracy of which the computer is capable. Some functions in mvBASIC do not produce a guaranteed 9 digits of decimal precision.
In mvBASIC, a four-decimal constant has the range:
-922,337,203,685,477.5807 to 922,337,203,685,477.5807
If the precision is set to a value less than 4, the range of allowable numbers is increased accordingly.
Example
This example demonstrates how the PRECISION statement affects the output of a calculation of pi:
|
PRECISION 2 CIRC = 25.13276208 RAD = 4 PI = CIRC / RAD / 2 PRINT PI |
This output displays:
|
3.14 |
See Also