[swift-evolution] [Pitch] New Version of Array Proposal

[Daryle Walker]

> On Jul 30, 2017, at 4:54 PM, Félix Cloutier <felixcca at yahoo.ca> wrote:
> 
>> Le 29 juil. 2017 à 16:01, Daryle Walker <darylew at mac.com> a écrit :
>> 
>>> On Jul 25, 2017, at 2:14 AM, Félix Cloutier via swift-evolution <swift-evolution at swift.org> wrote:
>>> 
>>> It seems to me that I'm the one advocating for fewer language changes, and the majority of sub-features that are being requested here don't feel particularly more general to me.
>>> 
>>> In its current form, this proposal tackles a contentious syntax to declare fixed-length arrays and a similarly contentious syntax to initialize them, it has multi-dimensional arrays, unpacking of arrays as parameter lists, zero-size types with alignment requirements, sized array literals, explicit conversions between array types, rules for deterministic initialization, vector types. Out of these, I could see parameter unpacking and vectors used for more than fixed-size arrays. (I haven't been through the entire discussion, though, so you might be able to point out a few more.)
>> 
>> On “unpacking of arrays as parameter lists,” are you talking about tuple conversion? That’s mainly to convert legacy C-conversions, which are manually homogenous tuples, to proper arrays. I added a little generalization instead of requiring a strict flat “(T, T, …, T, T)” format for the tuple type. If I didn’t need that legacy support, I probably wouldn’t have added tuple conversion at all.
> 
> No, I am talking about the `b[ [;1, 4] ]` syntax.

I didn’t have that at first. Later, I added “#indexOf”. That operator does not dump a bunch of comma-separated tokens; it returns a full object (a one-dimensional fixed-size integer array). To actually be usable in indexing a parallel array of the same shape, I needed to either explode the array to a comma-separated list of integers, or add a subscript operator that takes all the indexes as a single object. The second one seems easier, especially since “comma-separated tokens of some kind” isn’t a Swift entity (at least not yet).

>> So zero-size arrays should have no alignment requirements? Looking at a playground, empty tuples (both “()” and “((), ())”) have zero size but a stride of 1. Since elements of an array are spaced out by stride instead of size, what happens if an empty array is used as an element type. Zero-size arrays should take up at least a stride of 1, but if not rounded up to the element type’s alignment, then the array alignment won’t match the element alignment. These worries are why I initially banned empty arrays from being element types. Should I go back to that?

In the update I’m working on, empty arrays take no size when they’re sub-objects (in a structure, tuple, or fixed-size array), but a minimal size when they’re top-level objects (globals and function locals).

>> The proposal technically does not affect the rules for deterministic initialization. The only point of note is since FSAs are aggregate compound types, an instance's elements’ DI statuses are tracked separately, just like tuple members. Everything in the rest of that section, besides future suggestions, falls into place using the existing DI rules.
> 
> I was merely enumerating the entirety of the features that your proposal requires to be implemented, as a response to Tino saying that I was suggesting more features that were less general.
> 
>> Since FSAs are a new kind of type, new ABI entries need to be formed. A reason to mention vector mode now is that using a processor vector-unit type or not affects the implementation of a FSA, hence its ABI. I want to put all the parameters on FSA that could affect the ABI in at once, so we don’t have to propose another ABI change later. This definitely includes vector mode, and may include multi-dimensionality.
>> 
>>> There's also a number of issues that aren't addressed in the document, which I feel are obscured by the sheer number of things that it asks to consider: for instance, as we mentioned on another thread, Swift anonymous types can't conform to protocols, so it's not clear what fixed-size arrays have to be under the hood to be iterable.
>> 
>> Did you look over the document? There’s a whole section on element traversal.
> 
> Yes, there is a section on array traversal, but there is a difference between saying "this needs to work" and "here's how it's going to work". If you're looking for what I missed, that would be that fixed-size arrays are meant to be their own special category of types, and not sugar on top of something else.
> 
> Speaking of things that are not addressed in the document, then, would be what any of this looks like at the SIL level. In fact, your proposal never spells out "SIL”.

Because I don’t know that much about it. I know a little more about LLVM and the ABI; so their contributions to implementation are mentioned in the proposal.

I’m not a compiler writer, so it’s less me implementing all the features at the compiler and support levels and more getting/giving mutual feedback with the people who do that work on what can/should be done.

So, I need to research this SIL and figure out if I need to add descriptions for this level.

>> If you really want a Collection, and/or you want a Swift version of the “T[]” parameter interface from C where the argument can be an array of any length but a constant type, then use “withUnsafe(Mutable)Flattening”.
> 
> We can already essentially do that with tuples. In fact, that's the current workaround for C arrays.

Tuples don’t have run-time indexing (for now, anyway), so don’t work as a solution for Collection.

Manually homogenous tuples are a workaround, so that use should be retired once something more on-point comes along. (I’m trying to design said “more on-point.”)

> Do you have code that currently tries to do C interop with fixed-size arrays? Have you identified pain points with that existing Swift code that you would like to make better with your fixed-size array proposal? Or is helping C interop only a secondary goal?
> 
>> If you want to iterate without converting to a Collection first, then you can use a FSA as the target of a “for-in” loop directly. If you need the iteration counter during the for-loop, a spiritual equivalent to Collection’s “enumerated," then use the new “#indexOf” primary expression.
> 
> It is my humble opinion that fixed-size arrays lose a lot of their usefulness if they can't at least be treated as Sequences.

I do have thoughts on how FSAs could work with the common algorithms, but I think that Sequence & Collection may have been the wrong abstraction to put them. Some of the algorithms can work for anything container-like, not just sequenced collections. We need new protocols to separate concepts more and have Sequence/Collection conform to those.

>> Iteration order of a FSA during a “for-in” loop is unspecified. That’s because I want to allow the compiler to pick what it thinks is the best order (which may not be storage order (but probably would be)). And I want to allow the compiler to use simultaneous (or other overlapping) orders. And that previous point is to allow processor vector-unit types as FSA implementations.
> 
> Why should fixed-size arrays have this special behavior over normal arrays? I find it hard to justify that fixed-size arrays iterate fundamentally differently from both variable-sized arrays in Swift and fixed-size arrays in C. I also don't see why the unspecified faster iteration algorithm could fundamentally only apply to fixed-size arrays. To me, that's a prime example of something that could be dropped.

FSAs intentionally don’t conform to Collection, because multi-dimensional arrays shouldn’t have to conform to a linear (by nature, hence the name “Sequence”) standard, at least by default. I didn’t want FSAs to be potentially held back by concessions to linearity. Why throw away optimizations that letting the compiler/system decide the visitation path could enable?

The existing containers can’t follow the road to arbitrary visitation orders because their very own base protocol, Sequence, is modeled not to do this. (They’re linear and forward, i.e. sequential.)

— 
Daryle Walker
Mac, Internet, and Video Game Junkie
darylew AT mac DOT com