[swift-dev] Making the sign of NaNs unspecified to enable enum layout optimization

[Jordan Rose]

Some disconnected thoughts:

- “Does not interpret” does not mean “does not preserve”. The very next sentence in the standard is "Note, however, that operations on bit strings—copy, negate, abs, copySign—specify the sign bit of a NaN result, sometimes based upon the sign bit of a NaN operand."

- If we were to claim a class of NaNs, I would pick signalling NaNs rather than positive or negative ones. AFAIK most NaN-embedding tricks avoid signalling NaNs because they can, well, signal, even though (again AFAIK) most modern systems don’t bother.

- I don’t feel like we have a coherent story here. We know that APIs taking “Double” or “Float” can represent any bit pattern. The last plan I heard for floating-point comparison treats NaNs as unordered relative to one another, even in a total order comparison. (I maintain that this is unsound.) And this proposal would treat some or all NaNs as invalid. I feel like we need to pick one approach here.

Jordan


> On Oct 19, 2016, at 20:42, Joe Groff via swift-dev <swift-dev at swift.org> wrote:
> 
> I had a discussion with Steve this morning about the potential for enum layout optimization with floating-point payloads. One great thing about floating-point is that it has NaNs, and lots of them. For the most part, the only significant semantic difference among these NaNs is whether they're signaling or not, so it's tempting to reclaim some of these representations for other purposes. Javascript engines are of course famous for "NaN-encoding" values, representing JS numbers as native Doubles and packing pointers to non-number object instances into the same representation by using NaN bit patterns. In Swift, we could do similar things for enums with floating-point payloads, making 'Float?' and 'Double?' take the same amount of space as non-optionals, and automatically applying Javascript-style layout optimizations when defining enums with mixed float and object payloads. IEEE 754 is ambivalent about the role of NaN payloads, and there are libraries that use payloads for interesting diagnostic purposes, but the standard declares up front that it "does not interpret the sign of a NaN" (section 6.3). Reserving "negative" NaNs with the sign bit set as extra inhabitants could let us do enum layout optimization without interfering with the ability for libraries to freely use NaN payloads.
> 
> However, the way we usually handle enum optimization with extra inhabitants is problematic for floats. We normally say that it is undefined behavior for a value to have an extra inhabitant representation—a class reference cannot be null, a Bool can only be 0 or 1, and so on. With floats, we need to interoperate with numerics code not written in Swift, and we want to be able to read floating-point data out of memory that may use arbitrary bit patterns. We don't want every double-returning C function or load from memory to require a check for reserved values afterward. Making it undefined behavior for floats to have "extra inhabitant" representations is thus probably not practical.
> 
> Instead of saying that extra inhabitants are undefined behavior, we could instead continue to allow Floats and Doubles to have arbitrary bit patterns, and only check for reserved values at the point we construct an enum that wants to use reserved values for layout. If we reserve negative NaNs, then for example, constructing a Float? or Double? from a nonoptional value would check whether the payload value is NaN and if so, clear the sign bit at that point. That way, we don't have any ABI problems with Floats and Doubles from foreign sources, but still get the benefits of layout optimization for Swift types. On the other hand, this would mean that supposedly-idempotent operations like '.some(x)!' lose the sign information for NaNs. Since we wouldn't want to prevent the optimizer from folding those kinds of operations away, we could define Swift's semantics to say that querying the sign of a NaN value produces an unspecified value. This matches the intent of IEEE 754, and shouldn't impact most numerics code in practice. If we were interested in pursuing enum layout optimization with float payloads, I think this would be the best approach.
> 
> -Joe
> _______________________________________________
> swift-dev mailing list
> swift-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev