# Design Meeting 2022-05-25: keyword generics

Link to doc: https://hackmd.io/Aw2L3VmPQsm0ANC_XMughQ

Process: Raise your hand in Zoom while you're reading ✋

# Discussion Notes

- Question to answer: do sets of effects need to be ordered?
- maybe another way to phrase the above: Do the effects need to capture structure

- is maybe-async in maybe-async too limited, or is it sufficiently expressive to still be useful?
- its utility *might* be limited to higher-order routines, like the `Option::map` example.

- can the return type of a function vary depending on keyword generic? (potentially by feeding the effect into a type-constructor for the return type)?

- Question: what should we do with `try`-stacking? What if we return an `Option<Result>` or `Result<Result<T>>`. How should effects apply to those?

### Carried vs Uncarried effects

scott: "carried" effects end up getting embedded into a return type that end up getting processed later, versus "uncarried" effects that just have an immediate meaning for the execution of the function, but no implications for the return type.

josh: also: Is "try" one effect, or is it something that varies depending on which type the "try" is operating on (e.g. Option vs Result)

scott: in any case, the potential difference in meaning of "async try" vs "try async" seems worrisome.

felix: definitely seems like "try" poses all sorts of issues, since it seems like it might "stack" (e.g. `Result<Result<T, E>, E>` or `Result<Option<T>, E>`)

josh: There's lot of open questions, and we should be clear about what would block stabilization of a feature vs what would block even starting the experimental work here.

Mark: Hesitant to introduce syntax into substantial parts of std even if hidden from rustdoc, that's a major step.

# Questions / Discussion Topics Queue

### Carried vs Uncarried effects

Scott: One thing I've noticed is that things tend to work differently for what I've called "carried" effects (like `try` and `async`, where it needs to wrap things in a different type and the behaviour can be "delayed" in a sense -- you can `?` or `.await` later) and "uncarried" effects (like `const` and `unsafe`, where there's no monomorphization difference that would be needed and you can't `.safe` later). Are we certain that we can handle these the same way? For example, ordering isn't meaningful for `const unsafe` or `unsafe const`, but `async try` vs `try async` are materially different, so does the order of the effect generics choose between these or something, for carried effects?

oli: In the MVP we opt to require `?` and `.await` at the immediate site. There is no delay. Quoting from the doc:
> any await inside a maybe-async function must be on a function call to another maybe-async function:

Scott: Does that imply that combining effects just isn't supported, since `.await?` has delayed the `?` part?

Scott: (Musing) Oh, "The interesting part here is that you can also just call this function in runtime code, thus sharing the implementation." also applies to `unsafe` in a way: A safe function is one "sharing the implementation" with its unsafe equivalent.

Scott: I'll also note that it's unclear to me that returning `impl TheEffectTrait` is always correct. For example, https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.try_find explicitly doesn't do that, since it wants people to be able to call combinators on the result, not just `?`.

Scott: So overall this proposal seems clearly good for uncarried effects, but I'm scared about the implications for carried things.

oli: all this work will be exploratory, I expect us to get a better picture of it once we have an explicit syntax to work with

## Josh: Should it be `effect A`/`effect true`, or should it be specifically `effect async` or perhaps `async true`?

oli: For now we're going with just a single bool
There may be desire to decouple them later, but there may also be interest to couple them (a function that is either async and const or neither)

Josh: If we intend to offer both coupled and decoupled in the future, it seems like `bar::<effect true>` is not forwards-compatible with that, in addition to not being self-documenting for what it's actually doing. We could have multiple generic parameters here, but having each one take a set of keywords seems better.

oli: yea, boolean flags are not great, this is like with function arguments. So we should indeed look into some typestrong thing here.

## Josh: I don't think we can defer the ambiguity of calling a maybe-async function from an async function; we can't force people to immediately use `.await`, even over an edition boundary, because they may want to use the future in some other way (e.g. `.instrument().await`, `join`). I think we need to sort out the right syntax for this first.

oli: right, this is not in the doc, but we'd like to basically use inference to go back from the await. This should be doable via generics on a trait (basically `MaybeFuture<effect E>`)

Josh: So if it infers to async it uses async, and if it doesn't it uses sync? That seems error-prone; forget an await and your code compiles using a sync call (and even works, except for being suboptimal). That has all the same problems as "automatic await", which is part of why several of us rejected automatic await.

oli: Yea, this is a concern, it may be trivial to lint though? Or we require async in async and you need to opt out.

Josh: I think settling this question needs to be a substantial part of the next step in getting this even prototyped.

oli: we can just require the explicit syntax for calling maybe-async from async at the beginning

Josh: As in `func::<effect true>` or however that ends up being spelled?

oli: yes

:+1:

oli: we mainly want to figure out how such a less explicit thing could work and what the problems are, we have no good plan here yet, beyond "make it more convenient". This can all live under an even less stable feature gate if we decide to even start working on it.

Josh: Given that this feature itself exists to make it more convenient, we definitely need to figure out at least some of what the endgame looks like, to know if we get convenience all the way through. (Notable alternatives: having separate names/functions for async and sync, and just having ways to share types between sync and async)

## Josh: Is there a way we could re-export a maybe-async name as a definitely-async or definitely-sync name?

oli: maybe via explicit opaque types? `opaque type Foo: async Fn() = some_fn`

## Mark: I don't think it makes sense to discuss now, but syntax here seems immensely important. Scattering all of std's docs for example with `effect X, ...` is terrible.

:+1:

Yosh: that makes sense! The primary focus so far has been to define a "desugared" variant of keyword generics: which is relatively close to how we think it would be represented in the compiler. But we haven't yet spent any time really thinking about whether and how more ergonomic syntax could be provided.

Mark: One of the reasons I note this is that -- per Josh's question above -- syntax will be pretty fundamental to early stage experimentation, I expect. (And I at least would be against landing code into std even hidden from rustdoc with `effect X` or similar proposed syntaxes; maintainability to newcomers is important too).

## Mark: select/specialization - my mental model is that async at least is pretty tied to "other bits" (e.g., a particular runtime's timer API, or epoll registration, etc.). Do we actually expect to be able to make code with the same interface compatible with both models with the same user interface?

I'm also thinking about e.g. reqwest's blocking API, which IIRC has some differences around pipelining (access to response body and headers separately? I forget details). But in general: do we expect this to be "enough"?

Yosh: We do! - Most of the differences on implementation can be divided into two categories:
1. Implementation of IO primitives, which live in something I refer to as "leaf futures".
2. Capabilities provided by the async model: e.g. ad-hoc concurrency, and ad-hoc cancellation of computation.

In the `async-h1` HTTP parser/serializer library the internals are almost entirely sequential: we expect that under this model we should be able to rewrite it to be "maybe async". The only open question I see is how to handle timeouts, but there are a few schemes.

As for the leaf futures: the `async-std` library has sufficiently proven that the existing functionality exposed by the stdlib can be ported to async code with no API changes modulo missing language features (async versions of traits, closures, drop, etc.). We want the proposal to make it possible to be to write "maybe async" versions of `TcpStream`, `File`, etc.

## Mark: user-defined generics -- not needed?

The async/const here make sense, but I'm not sure how to extend that to fallibility. We do have plans for yeet/try blocks, but if I want (for example) panic fallibility generics or allocation fallibility generics, those seem like they need to fit into this model. I'm not sure they fit will into the contexts proposal from Tyler (and others), given the need to vary return types and such.

oli: user-defined and "add more" are two somewhat separate topics. We do want a few more language effects if this pans out for const and async, but it's entirely unclear what it would even mean for a user to define a new effect.

One concrete-ish example: I may also want multiple of the same effect (particularly try), e.g., `Result<Result<T>>` or `Result<T>` or `T`, with panics in different places as an alternative (for example).

## Felix: Terminology: "Effect"? Or "Evaluation Context"?

pnkfelix: I'm still trying to tease apart whether this is really an effect system (which I usually think of as specifying constraints on the function body, e.g. "can read region R" or "can write region R" or "can throw an exception", or a way to thread down information about the context

oli: we use "effect" because it's similar and people tell us it's effects, but mostly it's being generic over $something as we want to avoid writing the same code twice with minor adjustments.

Yosh: I don't think we're particularly committed to any specific terminology either; I'd be happy to change words to different words if folks find that clearer.

## Josh: How would you write shared types between sync and async, if the types need to have slightly different semantics beyond just the functions called?

Josh: For instance, an async Thing may need to have a file descriptor marked as non-blocking, while a sync Thing may need to have a file descriptor marked as blocking?