Add a schematic state machine implementing Future

src/SUMMARY.md

@@ -382,6 +382,8 @@
 - [Async Basics](concurrency/async.md)
   - [`async`/`await`](concurrency/async/async-await.md)
   - [Futures](concurrency/async/futures.md)
+  - [State Machine](concurrency/async/state-machine.md)
+    - [Recursion](concurrency/async/state-machine/recursion.md)
   - [Runtimes](concurrency/async/runtimes.md)
     - [Tokio](concurrency/async/runtimes/tokio.md)
   - [Tasks](concurrency/async/tasks.md)

src/concurrency/async-pitfalls/pin.md

@@ -4,13 +4,10 @@ minutes: 20
 
 # `Pin`
 
-Async blocks and functions return types implementing the `Future` trait. The
-type returned is the result of a compiler transformation which turns local
-variables into data stored inside the future.
-
-Some of those variables can hold pointers to other local variables. Because of
-that, the future should never be moved to a different memory location, as it
-would invalidate those pointers.
+Recall an async function or block creates a type implementing `Future` and
+containing all of the local variables. Some of those variables can hold
+references (pointers) to other local variables. To ensure those remain valid,
+the future can never be moved to a different memory location.
 
 To prevent moving the future type in memory, it can only be polled through a
 pinned pointer. `Pin` is a wrapper around a reference that disallows all

src/concurrency/async/state-machine.md

@@ -0,0 +1,112 @@
+---
+minutes: 7
+---
+
+# State Machine
+
+Rust transforms an async function or block to a hidden type that implements
+`Future`, using a state machine to track the function's progress. The details of
+this transform are complex, but it helps to have a schematic understanding of
+what is happening.
+
+```rust,editable,compile_fail
+use futures::executor::block_on;
+use pin_project::pin_project;
+use std::future::Future;
+use std::pin::Pin;
+use std::task::{Context, Poll};
+
+async fn send(s: String) -> usize {
+    println!("{}", s);
+    s.len()
+}
+
+/*
+async fn example(x: i32) -> usize {
+    let double_x = x*2;
+    let mut count = send(format!("x = {x}")).await;
+    count += send(format!("double_x = {double_x}")).await;
+    count
+}
+*/
+
+fn example(x: i32) -> ExampleFuture {
+    ExampleFuture::Init { x }
+}
+
+#[pin_project(project=ExampleFutureProjected)]
+enum ExampleFuture {
+    Init {
+        x: i32,
+    },
+    FirstSend {
+        double_x: i32,
+        #[pin]
+        fut: Pin<Box<dyn Future<Output = usize>>>,
+    },
+    SecondSend {
+        count: usize,
+        #[pin]
+        fut: Pin<Box<dyn Future<Output = usize>>>,
+    },
+}
+
+impl std::future::Future for ExampleFuture {
+    type Output = usize;
+    fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
+        loop {
+            match self.as_mut().project() {
+                ExampleFutureProjected::Init { x } => {
+                    let double_x = *x * 2;
+                    let fut = Box::pin(send(format!("x = {x}")));
+                    *self = ExampleFuture::FirstSend { double_x, fut };
+                }
+                ExampleFutureProjected::FirstSend { double_x, mut fut } => {
+                    match fut.as_mut().poll(cx) {
+                        Poll::Pending => return Poll::Pending,
+                        Poll::Ready(count) => {
+                            let fut =
+                                Box::pin(send(format!("double_x = {double_x}")));
+                            *self = ExampleFuture::SecondSend { count, fut };
+                        }
+                    }
+                }
+                ExampleFutureProjected::SecondSend { count, mut fut } => {
+                    match fut.as_mut().poll(cx) {
+                        Poll::Pending => return Poll::Pending,
+                        Poll::Ready(tmp) => {
+                            *count += tmp;
+                            return Poll::Ready(*count);
+                        }
+                    }
+                }
+            }
+        }
+    }
+}
+
+fn main() {
+    println!("result: {}", block_on(example(5)));
+}
+```
+
+<details>
+
+While this code will run, it is simplified from what the real state machine
+would do. The important things to notice here are:
+
+- Calling an async function does nothing but construct a value, ready to start
+  on the first call to `poll`.
+- All local variables are stored in the function's future struct, including an
+  enum to identify where execution is currently suspended. The real generated
+  state machine would not initialize `i` to 0.
+- An `.await` in the async function is translated into a call to that async
+  function, then polling the future it returns until it is `Poll::Ready`. The
+  real generated state machine would contain the future type defined by `send`,
+  but that cannot be expressed in Rust syntax.
+- Execution continues eagerly until there's some reason to block. Try returning
+  `Poll::Pending` in the `ExampleState::Init` branch of the match, in hopes that
+  `poll` will be called again with state `ExampleState::Sending`. `block_on`
+  will not do so!
+
+</details>

src/concurrency/async/state-machine/recursion.md

@@ -0,0 +1,43 @@
+---
+minutes: 3
+---
+
+# Recursion
+
+An async function's future type _contains_ the futures for all functions it
+calls. This means a recursive async functions are not allowed.
+
+```rust,editable,compile_fail
+use futures::executor::block_on;
+
+async fn count_to(n: u32) {
+    if n > 0 {
+        count_to(n - 1).await;
+        println!("{n}");
+    }
+}
+
+fn main() {
+    block_on(count_to(5));
+}
+```
+
+<details>
+
+This is a quick illustration of how understanding the state machine helps to
+understand errors. Recursion would require `CountToFuture` to contain a field of
+type `CountToFuture`, which is impossible. Compare to the common Rust error of
+building an `enum` that contains itself, such as
+
+```rust
+enum BinTree<T> {
+    Node { value: T, left: BinTree<T>, right: BinTree<T> },
+    Nil,
+}
+```
+
+Fix this with `Box::pin(count_to(n-1)).await;`, boxing the future returned from
+`count_to`. This only became possible recently (Rust 1.77.0), before which all
+recursion was prohibited.
+
+</details>