Bikeshedding the 'gen' keyword & syntax

[dhardy]

#117078 lists "unresolved" questions:

• Whether we should use some keyword other than gen.
• Whether we should reserve gen as a full keyword or as a contextual one.

From the RFC:

@yoshuawuyts For the FCP I started above, the gen keyword and gen { ...} blocks and async gen { ... } blocks are in-scope, while gen fn is out of scope.

So we need a keyword for "gen blocks"; gen fn might get added in the future.

---

Conflicts

This conflicts with rand::Rng::gen (see rust-random/rand#1435). There are quite a few hits within the rand crate. The rand_distr crate has 3 uses within documentation, 2 within tests and 14 in algorithms.

This isn't a lot of data, but hints at quite a few real uses of Rng::gen outside rand, not counting other possible uses of gen.

From the RFC

I haven't read all comments on the RFC, but several indicate that gen fn may not be a good thing.

From @Kray:

IMO this function syntax, while analogous to how async functions work, looks very surprising and non-obvious:

gen fn odd_dup(values: impl Iterator<Item = u32>) -> u32

Without inspecting the opposite end of the line, the function looks like it returns a single u32. I know async fn works the same way, but at least when you see an async fn foo() -> u32 you know it will eventually return a single u32.

Not to mention there's good arguments why async fn might have also been a mistake.

Because the "simple" case of an owned iterator doesn't require complicated type signatures unlike some Futures generated by async fn, wouldn't this be a good time to adopt this function expression syntax suggested many times in the past (including the blog post above)?

fn foo() -> impl Iterator<Item = u32> = gen { ... }

From @withoutboats:

(NOT A CONTRIBUTION)

I think there are several good arguments for replacing -> with something like yield or yields. I'm pretty much just summarizing here:

* Educational: it helps users understand that this yields many times, instead of returning once. It ties the declaration syntax to the different return keyword inside the body.

* Forward compatibility: I personally don't think general purpose coroutines should be declared with the same keyword as generators-as-Iterators, but this does leave space for that possibility in the syntax by allowing the syntax to support a separate `->` type declaration.

The only possible difficulty you need to consider is that you want to be forward compatibility with the ability to declare the type yielded by a gen { } block. This may lean toward yields; not sure if gen yield i32 could be supported by the parser or if its ambiguous.

From @emilyyyylime:

I was thinking potentially fn f() gen i32 for 'generates'

There is also discussion on usage of generator instead and discussion on gen as a contextual keyword.

---

Since I'm not fully read-up on the status of this RFC and its implementation, I'll just leave this here with the question of whether gen will become a keyword.

[jieyouxu]

I'll leave this as a C-discussion issue so we have an issue to discuss the gen syntax, otherwise it kinda becomes all over the place and discussions might be lost.

[jieyouxu]

Relevant Zulip thread regarding reserving the gen keyword: https://rust-lang.zulipchat.com/#narrow/stream/268952-edition/topic/.60gen.60.20keyword

[PatchMixolydic]

As a user, I'm not too concerned about what the keyword is, as long as it's fairly succinct and suggests either generators or iterators¹. gener seems a little weird, but would probably avoid most name collisions. generator seems a little too long, at least for Rust.

I'd be highly in favour of providing some sort of gen fn syntax to make it easier to use generators, both as iterators and as restricted coroutines, but I'm not entirely sure what that'd look like. In isolation, gen fn name(...) yield YieldTy { ... } seems fine to me, but that might lead to incongruencies if the language team ever stabilizes coroutines with return types other than ()².

Footnotes

1. Coroutines would also work, but might conflict with syntax space for general (semi)coroutines. ↩

2. Ignoring resume arguments, the obvious choices for general coroutine syntax would be co fn name(...) yield YieldTy -> RetTy, which is internally inconsistent, or co fn name(...) yield YieldTy return RetTy, which is inconsistent with normal fns. ↩

[Yokinman]

• I don't think async fn was a mistake. The consistency among keywords that affect a function's syntax is definitely useful, at least documentation-wise (const, async, unsafe, etc.). In my opinion, gen affecting the entire function's control flow should keep consistent with keywords like async.

• It feels natural to read -> T as, "this produces T", but it is different from the usual meaning. I think this syntax looks confusing when return ends the function without a value.

  It would also make it possible to confuse functions returning single types versus iterator types, just in terms of skimming function return types in documentation or source code. I would prefer anything else like:

  • gen fn() -> yield i32 - Could retain -> T as "produces T". e.g. coroutine fn() -> yield i32, return i32.
  • gen fn() yield i32 - Uses an existing keyword. Slightly awkward to read.
  • gen fn() yields i32 - Requires a new keyword. Reads very nicely.
  • gen fn() ^ i32 - Upwards arrow (yields to the caller, may resume later).
  • gen fn() >> i32 - Two arrows (produces multiple values).
  • gen fn() => i32 - Is -> meaningfully distinct from =>? I suppose -> is usually for types and => is usually for values.

I can't wait for generator functions, I love generator functions.

Your friend,
Yokin

[oli-obk]

One thing that I think is important is that we pick the same keyword for coroutines and generators, as they are very similar. Coroutines take arguments on every resume, even the first one, so they need a closure-style syntax (e.g. coro |x| {...} or gen |x| { ... }). So if gen is considered too breaking, we can instead use co {} or coro {} blocks, which both have significantly less edition breakage.

I personally am not convinced that the extra language surface of gen fn are worth it, and would like to see whether fn foo() -> impl Iterator<Item = i32> is too annoying in practice and use that as information on whether to add gen fn. It's also not clear to me whether gen fn is sugar for fn() -> impl Iterator or fn() -> impl PinnedIterator (doesn't exist yet, but we likely want it), and if we stabilize it as impl Iterator we can't take it back. This is in contrast to gen {} blocks, which can just implement more traits in the future without it being a breaking change.

[ds84182]

Eh. gen fn |x| {...} ¯⁠\⁠_⁠(⁠ツ⁠)⁠_⁠/⁠¯
use fn to disambiguate, or even require it.

Slight off topic: fn(x) => expr and fn(x) {} are just 2 to 4 extra characters and share similarity with match. Too late to change the syntax.

[rpjohnst]

One thing that I think is important is that we pick the same keyword for coroutines and generators, as they are very similar.

I don't think this is something we should commit to at this point. Both generators and async functions are specializations of the more general underlying control flow mechanism, exposing different subsets of that behavior- neither expose the "resume argument" to the user and both fix the yield and return types in particular ways.

Coroutines should remain free to change arbitrarily, so long as await and yield can desugar to them, until we're happy with them as a user-facing feature (if ever).

[oli-obk]

Yea, that makes sense. Especially since I was reminded that gen closures could be a different thing, the equivalent to async closures

[oli-obk]

Throwing some options out there:

• stick with gen, break lots of crates
• use gener (personal opinion: it's not clear what it means, tho we'll learn)
• impl Iterator {}, kinda long but very extensible to builtin syntax for other traits (impl Generator, impl Try), conflicts with inherent impls
• iter/iterator, breaks too much code
• something!, make it a builtin macro, so local definitions and imports override it
• generator, wordy, but explicit. We don't need to stick with the "keywords must be short" rule and it likely has no problematic breakage
• use an attribute: #[gen] {}, a bit odd, but very extensible
• ask T-lang to revisit gen becoming a contextual keyword
• yields {}, may be too confusing with yield
• prefix with some existing keyword (move gen {} could signal the difference to static gen {} which would not create iterator impls, but only some so-far-non-existent PinnedIterator trait). Not really necessary (we can just infer which traits to implement for gen blocks, though this is another nail in the coffin of gen fn). Also it's very confusing as move gen is different from gen move (move all variables instead of letting the generator be movable) and implies the existence of move gen move.

[tmandry]

Right now I'm -1 on any syntax that uses -> T to mean "yields type T"; I think using yield or yields is much better. I'm also okay with deferring the question and using -> impl Iterator<Item = T> for now; I think that's how the RFC ended up.

• yields {}, may be too confusing with yield

If we reserve yields it could be reused for the above purpose. I guess explicit type annotation for generator blocks would look like this then: yields T { ... }

• use gener (personal opinion: it's not clear what it means, tho we'll learn)

Seems no worse than gen in that regard; maybe slightly clearer.