[swift-dev] Making the sign of NaNs unspecified to enable enum layout optimization

Joe Groff jgroff at apple.com
Wed Oct 19 22:42:57 CDT 2016

I had a discussion with Steve this morning about the potential for enum layout optimization with floating-point payloads. One great thing about floating-point is that it has NaNs, and lots of them. For the most part, the only significant semantic difference among these NaNs is whether they're signaling or not, so it's tempting to reclaim some of these representations for other purposes. Javascript engines are of course famous for "NaN-encoding" values, representing JS numbers as native Doubles and packing pointers to non-number object instances into the same representation by using NaN bit patterns. In Swift, we could do similar things for enums with floating-point payloads, making 'Float?' and 'Double?' take the same amount of space as non-optionals, and automatically applying Javascript-style layout optimizations when defining enums with mixed float and object payloads. IEEE 754 is ambivalent about the role of NaN payloads, and there are libraries that use payloads for interesting diagnostic purposes, but the standard declares up front that it "does not interpret the sign of a NaN" (section 6.3). Reserving "negative" NaNs with the sign bit set as extra inhabitants could let us do enum layout optimization without interfering with the ability for libraries to freely use NaN payloads.

However, the way we usually handle enum optimization with extra inhabitants is problematic for floats. We normally say that it is undefined behavior for a value to have an extra inhabitant representation—a class reference cannot be null, a Bool can only be 0 or 1, and so on. With floats, we need to interoperate with numerics code not written in Swift, and we want to be able to read floating-point data out of memory that may use arbitrary bit patterns. We don't want every double-returning C function or load from memory to require a check for reserved values afterward. Making it undefined behavior for floats to have "extra inhabitant" representations is thus probably not practical.

Instead of saying that extra inhabitants are undefined behavior, we could instead continue to allow Floats and Doubles to have arbitrary bit patterns, and only check for reserved values at the point we construct an enum that wants to use reserved values for layout. If we reserve negative NaNs, then for example, constructing a Float? or Double? from a nonoptional value would check whether the payload value is NaN and if so, clear the sign bit at that point. That way, we don't have any ABI problems with Floats and Doubles from foreign sources, but still get the benefits of layout optimization for Swift types. On the other hand, this would mean that supposedly-idempotent operations like '.some(x)!' lose the sign information for NaNs. Since we wouldn't want to prevent the optimizer from folding those kinds of operations away, we could define Swift's semantics to say that querying the sign of a NaN value produces an unspecified value. This matches the intent of IEEE 754, and shouldn't impact most numerics code in practice. If we were interested in pursuing enum layout optimization with float payloads, I think this would be the best approach.


More information about the swift-dev mailing list