[swift-evolution] Pre-proposal: Safer Decimal Calculations

[Rainer Brockerhoff]

On 3/24/16 09:54, Stephen Canon wrote:
> On Mar 23, 2016, at 5:26 AM, Rainer Brockerhoff via swift-evolution 
> <swift-evolution at swift.org> wrote: ...
>> On 3/22/16 23:20, Michael Gottesman via swift-evolution wrote:
>>>> Pre-proposal: Safer Decimal Calculations Proposal: TBD 
>>>> Author(s): Rainer Brockerhoff Status: TBD Review manager: TBD 
>>>> ... Full Decimal type proposal ...
>>> Rainer: I quickly skimmed this. Just to make sure I am 
>>> understanding 100%: you are proposing a fixed point decimal 
>>> calculation or a floating point decimal calculation. The former, 
>>> no?
>> 
>> Right, fixed-point. (NSDecimalNumber is decimal floating-point, of 
>> course).
> 
> What you’re describing is actually not a fixed-point format at all, 
> but rather a variant of what IEEE 754 calls an “extendable precision 
> [floating-point] format”.

Stephen, thanks for replying in so much detail and setting me straight
on nomenclature. Nothing better than talking to a specialist ;-)

I'm working on a rewrite of my text (between medical time-outs) but
meanwhile, some comments on some of your comments.


> A fixed-point format has a *fixed* radix point (or scale) determined 
> by the type.  A couple examples of common fixed point formats are: 
> ... Your proposed format, by contrast encodes the radix point / scale
> as part of the number; instead of being constant for all values of
> the type it “floats”, making it a floating-point format.

See your point, I missed the "determined by the type" part.

In fact, one alternative I'm considering of proposing is to do a real
fixed-point type, with enough digits (on both sides of the decimal
point) to be useful for most real-world problems. Referring, of
course, to
http://www.jpl.nasa.gov/edu/news/2016/3/16/how-many-decimals-of-pi-do-we-really-need/,
but see below...


> The usual thing would be to use a sign-magnitude representation 
> (where the significand is unsigned and the signbit is tracked 
> separately), rather than a twos-complement significand.  It just 
> works out more nicely if you can treat all the words in the 
> significand the same way.

Yep, I jumped the gun here, prematurely thinking of optimized
assembly-language implementations.


> To the IEEE 754 recommendations, it sounds like you would want to
> add a policy of either growing the precision to keep results exact
> when possible, or indicating an error when the result is not exact.

Initially my idea was growing the precision to keep results exact, up to
some maximum, and then rounding. And, in practice, having round up/down
functions to N digits.

I see that conforming to IEEE 754 with, as you said, "±0, ±infinity,
quiet and signaling NaNs" etc. goes beyond my aims here, since:

> It’s worth noting that I’ll be trying to drive the FloatingPoint
> protocol to match these requirements, so we can really just say “your
> type should conform to FloatingPoint”.
> ...
> How do you propose to handle division / square root, where the
> results are essentially never finitely representable?

So, I see several, maybe partially conflicting, aims here.

There's the mismatch between decimal representation of binary formats,
causing confusion for very common cases like 0.01. There's your work in
upgrading the FloatingPoint protocol. There's the question of
modernizing NSDecimalNumber or writing a new decimal type. The
scientific community needs IEEE 754, the mathematical community needs
exact-precision bignums, the financial community needs predictable but
small decimal precision, the educators need simple decimal numbers for
teaching and graphing.

IMHO the existing Double, Float and CGFloat types don't cover all those
use cases.

Maybe we need a DecimalLiteralConvertible not as generic as
FloatLiteralConvertible, so that we can have a built-in type - call it
Decimal or SimpleDecimal - that would be inferred in a statement like
`let x = 0.01`
such that, thereafter, calculations with x are sufficient for 99% of
real-world graphing and financial calculations, with exact comparisons
and so forth, but with none of the IEEE 754 complications. (Of course if
you need trigonometry and square root etc. just convert to Double.)

I'm certainly not qualified to discuss the implementation details, so
I'm content to get the discussion rolling here.

Thanks again,
-- 
Rainer Brockerhoff  <rainer at brockerhoff.net>
Belo Horizonte, Brazil
"In the affairs of others even fools are wise
In their own business even sages err."
http://brockerhoff.net/blog/