[swift-dev] Making the sign of NaNs unspecified to enable enum layout optimization

[Joe Groff]

> On Oct 20, 2016, at 10:38 AM, Stephen Canon <scanon at apple.com> wrote:
> 
>> 
>> On Oct 20, 2016, at 10:04 AM, Joe Groff via swift-dev <swift-dev at swift.org> wrote:
>> 
>> 
>>> On Oct 20, 2016, at 9:42 AM, Jordan Rose <jordan_rose at apple.com> wrote:
>>> 
>>> Some disconnected thoughts:
>>> 
>>> - “Does not interpret” does not mean “does not preserve”. The very next sentence in the standard is "Note, however, that operations on bit strings—copy, negate, abs, copySign—specify the sign bit of a NaN result, sometimes based upon the sign bit of a NaN operand."
>>> 
>>> - If we were to claim a class of NaNs, I would pick signalling NaNs rather than positive or negative ones. AFAIK most NaN-embedding tricks avoid signalling NaNs because they can, well, signal, even though (again AFAIK) most modern systems don’t bother.
>> 
>> Claiming sNaNs would be unfortunate since "signaling" is about the only semantically distinct bit NaNs normally have, and I think we should minimize interference with users who are taking advantage of signaling or NaN payloads for their own means. (OTOH, on some platforms like x87 it's already practically impossible to preserve the signaling bit, since even loads will immediately raise the exception and quiet the NaN, and there would be some nice safety benefits to getting a trap early if a Float? is bitcast to a Float without being formally checked first.)
>> 
>>> - I don’t feel like we have a coherent story here. We know that APIs taking “Double” or “Float” can represent any bit pattern. The last plan I heard for floating-point comparison treats NaNs as unordered relative to one another, even in a total order comparison. (I maintain that this is unsound.) And this proposal would treat some or all NaNs as invalid. I feel like we need to pick one approach here.
>> 
>> I'm not saying that they'd be invalid, only that the language doesn't guarantee to preserve these representations exactly. That seems orthogonal to the comparison issue—whatever rule we come up with for float ordering, all NaNs ought to be treated uniformly by that rule.
> 
> In particular, the language doesn’t guarantee to preserve the representation exactly specifically **for conversions to Float? and back**.  IEEE 754 has no notion of optionals, so the IEEE 754 rules don’t directly constrain the design space here.
> 
> The primary constraint is that conversion to Optional and back should preserve values.  I don’t think that it needs to preserve *encodings*, however (e.g. IEEE 754 conversions canonicalize decimal significands, and I’m OK with conversions to/from Optional doing the same; we can mod NaNs out by some reserved payload bit and call them different encodings of the same NaN value and the same rationale applies).
> 
> Preserving values implies preserving substitutability.  That should preserve nan-ness for sure, and should either preserve signaling-ness or actually “signal” (in the IEEE 754 sense) at the point of conversion to Optional.  One can argue that preserving the signbit of the NaN would be nice (for substitutability in copysign), but one can also argue that this is really a property of the encoding, not the value.
> 
> Like Joe, I don’t think that comparison really enters into this (the issue was never totally resolved to my understanding, but my recollection is that we mostly left it as "<=> should order according to the IEEE 754 level 2 abstraction”—meaning all NaNs are equal to each other for the purposes of <=>).
> 
> I would be pretty opposed to claiming sNaNs for this purpose.  The two better options I see are:
> 
> 	- Use the signbit of NaN.
> 	- Reserve NaNs whose significand begins with `b11` (these bit patterns are already reserved by the `Float(nan: signaling:)` constructor).
> 
> copysign( ) is a reason to not pick the first option.  I’m not very worried about it, but it is a reason.  I see no problem with the second option.

As we discussed in person this morning, de-canonicalizing b11 might be a better compromise to minimize the potential impact of layout optimizations. That would leave the implementation with 2^51 NaN representations (50 significand bits, plus the sign bit) in Double to play with, which ought to be enough for anyone™. I liked the idea of using the sign bit originally since testing for NaNs and sign bits is something that can be easily done using common FPU instructions without crossing domains, but as you noted, it sounds like comparison and branching operations tend to do that anyway, so masking and branching using integer operations shouldn't be too much of a burden. Jordan's question of to what degree we consider different NaN encodings to be distinct semantic values is still an interesting one, but if we take only the b11 NaN payloads away, that should minimize the degree to which the implementation needs to be considered as a constraint in having that discussion.

-Joe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-dev/attachments/20161020/d4725382/attachment.html>