[swift-evolution] [Proposal] Random Unification

Tue Jan 9 15:07:12 CST 2018

> On Jan 9, 2018, at 8:28 AM, Nate Cook <natecook at apple.com> wrote:
> 
>> On Jan 9, 2018, at 4:12 AM, Jonathan Hull <jhull at gbis.com <mailto:jhull at gbis.com>> wrote:
>> 
>> Some thoughts:
>> 
>> - How do I randomly select an enum?
> 
> Vote for SE-0194! :)
> 
>> - I like that RandomNumberGenerator doesn’t have an associated type. I agree that we should just spit out UInt64s for simplicity.
> 
> It simplifies things a lot, at some performance cost. For example, the LCRNG type really produces UInt32s, so I have to use two of its outputs to generate one value, even if I don’t need that many bits.

I agree with you, but just to play devil’s advocate, why not have two outputs: one for UInt32 and another for UInt64?  The generator itself would only have to provide one of the UInt types and the other would be provided by default implementation (either by stacking 2 UInt32s or discarding half of a UInt64).  I know this is less minimal, but I don’t think it is actually complicated or hard to use. You just ask for the type that is most useful for you.

>> - I don’t like how it is so closely tied with Range.  I realize that both Int and Float work with Ranges, but other random types do not (e.g. CGVectors).  You are special casing FixedWidthInteger and BinaryFloatingPoint, which are very important… but we lose the ability to deal with other randomly generated types.
>> 
>> - Following on the previous point, I don’t like that the code for dealing with Integers/Floats is in Range.  It feels like things aren’t properly encapsulated. 
> 
> I actually agree with you, and for getting individual values prefer the form `let x = Int.random(in: 1…10)`. Here’s how I got to what’s in the playground:
> 
> 1) We definitely want to be able to select a random element from a collection.
> 2) Given that, we’ll have (1…10).random() and (0..<10).random() even if those aren’t what we prefer, and people will use them.
> 3) If people use that construction for integers, it will be strange to not have the same facility for floating-point numbers.
> 4) Once we have the range-based capability for both, the type-based versions are redundant (i.e., they can be added in the future if we decide we made the wrong decision by excluding them).
> 
> You’re of course correct that a pattern of range-based random functions doesn’t extend well to other types. I show on the last page a couple different ways of writing those, for Bool and Data. Most of the other types you’d want to create lie outside the Swift standard library, so we can’t address really those here.

Right. I guess my thought is that I would like them to be able to use a standard creation pattern so it doesn’t vary from type to type (that is the whole point of “unification” in my mind).  In my own code, I have a concept of constraint, of which a set are passed to the object being created. This allows me to random create colors which look good together, etc….  I then have some convenience methods which just automatically create an appropriate constraint from a range where appropriate.  I’d really like to see something standard which allows for constraints other than simple ranges.

I think I would feel at least a little better if the range stuff worked for any type which can be put in a range…

I also feel like the focus on ranges to the exclusion of everything else is one of those cute things that will come back to bite us later.  My main focus is on random things which are presented to the user in some way, and I realize that is a different use-case than most programmers. Even for things like pure number generation, I am worried about things like sig-figs and not just range.

>> - Why bother supporting non-closed Ranges at all?  If you only allow closed ranges, then you can’t end up with an empty range. The only difference in behavior I can think of is on floating point, but I can’t think of a use-case where excluding the supremum is actually useful in any real world way.
> 
> Half-open ranges are a major use case for generating random numbers, particularly when working with collections. Whenever you see that someone’s written `random() % n`, that’s the half-open range 0..<n.

Ok, you convinced me on the half-open ranges.

I still think the best way to handle the 'random() % n' issue is to look for it as a pattern and then just issue a warning/fixit.

>> - This may sound strange, but I would really like to see Bool handled as a default implementation on the generator protocol itself.  On my own version of this I have both the ‘coinFlip()’ and ‘oneIn(_ num:Int)’ methods which I find extremely useful.  CoinFlip just gives you a random bool, whereas you can say things like oneIn(100) to get ‘true’ roughly 1 out of every 100 times you call it.  These are useful for branching randomly.  They are most useful on the source/generator itself because it is ergonomic when you need to rewind the source.
> 
> Bool is certainly a very important type to be able to randomly generate. I’m not opposed to it being included in a proposal, but it’s simple enough to do on your own that it didn’t pass the “minimal” test that I was using in the playground. You could use something like this static method:
> 
>     extension Bool {
>         static func random(
>             probability: Double = 0.5, 
>             using generator: RandomNumberGenerator = Random.default
>         ) {
>             return (0.0 ..< 1.0).random(using: generator) < probability
>         }
>     }
> 
> I don’t think there should be any value-producing methods on generators—most users shouldn’t need to think about generators at all, and the ones who have a specific need (repeatability, rewinding, etc) should be able to use the same APIs as the people who aren’t thinking about them.

I can add them easily myself. That said, it is very nice ergonomically for branching randomly.

	if myGenerator.oneIn(3) {
		//This happens 1/3 of the time
	}else{
		//This happens 2/3 of the time
	}

>> - IMO distributions should be sources/generators themselves which just wrap another source.  We could have a subprotocol of RandomNumberGenerator which just semantically guarantees uniform distribution, and then distributions that need it could be sure of the input distribution.  Notice this doesn’t limit the distribution to only be used for Integers as they are in the demo. They can be used anywhere a source can be used.
> 
> I’d really like to maintain a clear line between generators and distributions (which I don’t think we need an additional protocol for). Distributions create values of a specific kind of type, with a particular distribution, which usually isn't suitable to use as the input for another algorithm that needs random data. Generators just pump out (hopefully) uniformly distributed bits, which distributions and other algorithms can then interpret and shape.

This gets to the heart of my main concern with the approach from the playground.  Everything is special-cased.  If the distribution only works for FixedWidthIntegers, then I can’t use it to create colors with that distribution, for example.

What I like about wrappers is that they fit anywhere a generator fits, so the receiver doesn’t really have to know/think about it at all.  You don’t have to rewrite it for each type you want it to work for.

>> - Having a subprotocol for generators which can be rewound is extremely important for entire classes of real-world problems.  I have spent a lot of time using this and it solves a LOT of problems. For example, I have a Lorem Ipsum Generator which takes Attributes and a CGSize to fill.  It works by branching (using the Bool methods above) and then rewinding bits which don’t fit (If you just futz with the last part instead of generating appropriate clauses, it won’t look right).  I also have a bunch of backtracking algorithms which rely on this rewind ability.  Plus numerous visual effects which rely on a repeatable rewindable source.
>> 	- Tl;dr: It isn’t enough to just have a seed, you need to be able to mark a state of a generator and return to that state later.
>> 
>> 	My RepeatableRandomSource Protocol has 3 extra methods:
>> 	- It takes a seed
>> 	- It has a mark() method which returns a token
>> 	- It has a returnToMark(_ mark:Mark) method which takes a token and restores the appropriate state 
> 
> That’s neat, I’ve never used a generator like this! The RandomNumberGenerator protocol would certainly allow for a generator with these features, and libraries or future additions to the standard library could include that kind of protocol.

Thanks.  I am coming to that conclusion.  I can build most of what I need on a simple base.  I guess I am sad that others won’t be able to share in it as easily, since it is a really nice/powerful way to code. I see so many visual bugs in UIs which have generated things randomly (e.g. things dance around or change when you resize them), and it is like this wouldn’t have happened if they had easy access to a repeatable/rewindable source.

One additional question.  How do you ergonomically get a Double which doesn’t have a range, but also isn’t NaN?

Thanks,
Jon

>> 
>> 
>>> On Jan 8, 2018, at 11:02 AM, Nate Cook via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>> 
>>> I created a playground to explore this question, starting with a minimal subset of the proposal’s additions and building from there. The attached playground demonstrates what’s possible with this subset on the first page, then uses subsequent pages to explore how the main random facilities of the C++ STL work under this model. (In my opinion, they work pretty well!)
>>> 
>>> The subset in the playground has three main differences from the proposal:
>>>  - It doesn't include a Randomizable protocol or a random property on numeric types.
>>>  - It doesn't include the static random(in:) methods on numeric types, either.
>>>  - The RandomNumberGenerator protocol doesn't have an associated type. Instead, it requires all conforming types to produce UInt64 values.
>>> 
>>> I’ve tried to include a bit of real-world usage in the playground to demonstrate what writing code would look like with these additions. Please take a look!
>>> 
>>> Nate
>>> 
>>> <Random.playground.zip>
>>> 
>>>> On Dec 2, 2017, at 9:50 PM, Dave Abrahams via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>>> 
>>>> I don’t have much to say about this other than that I think the discussion seems way too narrow, focusing on spelling rather than on functionality and composability.  I consider the “generic random number library” design to be a mostly-solved problem, in the C++ standard library (http://en.cppreference.com/w/cpp/numeric/random <http://en.cppreference.com/w/cpp/numeric/random>).  Whatever goes into the Swift standard library does not need to have all those features right away, but should support being extended into something having the same general shape. IMO the right design strategy is to implement and use a Swift version of C++’s facilities and only then consider proposing [perhaps a subset of] that design for standardization in Swift.
>>>> 
>>>> Sent from my iPad
>>>> 
>>>> On Dec 2, 2017, at 5:12 PM, Kyle Murray via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>>> 
>>>>> 
>>>>>> On Dec 2, 2017, at 6:02 PM, Xiaodi Wu via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>>>>> 
>>>>>> Instead, we ought to make clear to users both the features and the limitations of this API, to encourage use where suitable and to discourage use where unsuitable.
>>>>> 
>>>>> I like that you're considering the balance here. I've been lightly following this thread and want to add my thoughts on keeping crypto and pseudorandomness out of the name of at least one random API intended for general use.
>>>>> 
>>>>> For someone who doesn't know or care about the subtleties of insecure or pseudorandom numbers, I'm not sure that the name insecureRandom is effectively much different than badRandom, at least in terms of the information it conveys to non-experts. To Greg's point, that's the opposite of the signal that the API name should suggest because it's what most people should use most of the time. As you say, this API is being designed for general use.
>>>>> 
>>>>> There's a cost to adding extra complexity to names, too. I don't think it's far-fetched to suspect that people who find insecureRandom in an autocomplete listing or search will think "Where's the plain random function?"... and then go looking for a community extension that will inevitably provide a trivial alias: func random() { return insecureRandom() }. That's the sort of adoption I'd expect from something for new programmers, like Swift Playgrounds. Someone's introduction to randomness in programming should probably involve no more than a straightforward mapping from the elementary definition, rather than forcing a teaching moment from more advanced math.
>>>>> 
>>>>> I think there are better places for caveat information than in the API names themselves; documentation being one clear destination. This is in contrast with Unsafe*Pointer, where the safety element is critical enough to be elevated to be more than caveat-level information. You can go really far and create really cool things before these caveats start to apply. Using randomness as a black box in an intro programming environment seems like a much more common scenario than someone attempting to roll their first crypto by only reading API names and hoping for the best.
>>>>> 
>>>>> -Kyle
>>>>> _______________________________________________
>>>>> swift-evolution mailing list
>>>>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>>>> _______________________________________________
>>>> swift-evolution mailing list
>>>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>>> 
>>> _______________________________________________
>>> swift-evolution mailing list
>>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20180109/0a694dd7/attachment.html>