[ https://issues.oasis-open.org/browse/ODATA-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61543#comment-61543 ]
Susan Malaika commented on ODATA-784:
-------------------------------------
Below is some feedback on the varying XDIV results :-
The right answer, to 48 digits, is:
0.000999945008289544118720320744191458969347194855888
So the values below, in order are:
a) Correct, except that if the result were 'correctly rounded' the last digit would be 7, not 6 (the next digit is a 9)
b) Ditto, the last digit should be 3, also 10 digits is quite short
c) This one is OK; 21 digits is good
d) Plain wrong; if rounding to 4 digits (extremely short!) the result would be 0.0009999
e) Last digit is wrong
f) Same as (b)
g) Correct
h) Correct, but 5 digits is not really reasonable
i) Correct, but exponential notation not really necessary when only three 0s after the decimal point.
> Need to specify the behaviour of arithmetic operators on Decimal type
> ---------------------------------------------------------------------
>
> Key: ODATA-784
> URL: https://issues.oasis-open.org/browse/ODATA-784
> Project: OASIS Open Data Protocol (OData) TC
> Issue Type: Improvement
> Components: OData URL Conventions
> Affects Versions: V4.0_OS
> Environment: Proposed
> Reporter: Evan Ireland
> Priority: Minor
> Fix For: V4.01_WD01, V4.0_ERRATA03
>
>
> The rules for scale of results of decimal arithmetic operators are not specified.
--
This message was sent by Atlassian JIRA
(v6.2.2#6258)
