Idea: TreeVar (context variables that follow stack discipline)

[oremanj]

A context variable is a convenient way to track something about the scope in which a task executes. Sometimes that "something" effectively constitutes a promise to child tasks that some resource remains usable. For example, trio_asyncio.open_loop() provides the asyncio loop via a contextvar, but it's not much use if the async with block has exited already.

Unfortunately, contextvars are inherited from the environment of the task spawner, not the environment of the nursery in which the task runs. That means they can't reliably provide resources to a task, because the resources might be released before the task finishes. Imagine:

async with trio.open_nursery() as nursery:
    async with trio_asyncio.open_loop() as loop:
        nursery.start_soon(some_asyncio_task)

It would be nice to have a kind of contextvar that is inherited by child tasks based on the value it had when their nursery was opened, rather than the value it had when the child task was spawned. (In theory, cancellation status could use this if it existed. In practice, we probably want to continue to special-case cancellation status because it's so fundamental.)

Implementation idea: each scoped contextvar uses an underlying regular contextvar whose value is a tuple (scoped contextvar's value, task that set it). Also, capture the contextvars context when creating a nursery (this is cheap, since contexts are meant to be shallow-copied all the time). When accessing the value of a scoped contextvar, check whether the underlying regular contextvar was set in the task we're currently in. If so, return the value from the underlying regular contextvar in the current context; otherwise, return its value in our parent nursery's captured context.

[njsmith]

First reactions:

Is this a problem? Answer: Maybe?

I see your point about trio-asyncio. But in your example, I'm not sure it would be so bad if some_asyncio_task instead raised an exception when you exited the loop block... compare to:

async with trio.open_nursery() as nursery:
    async with trio.open_tcp_stream(...) as stream:
        nursery.start_soon(some_stream_task, stream)

Seems like a pretty similar case, it does confuse new users sometimes, but... there's not really anything to do about it except teach folks what things mean.

I think the other way around might be potentially more compelling? If I do:

global aio_supporting_nursery
async with trio_asyncio.open_loop() as loop:
    async with trio.open_nursery() as aio_supporting_nursery:
        await sleep_forever()

# Elsewhere:
aio_supporting_nursery.start_soon(aio_task)

Then currently that won't work, but maybe it would be convenient if it did? I'm not sure whether this example is realistic or contrived :-)

Another option I'll throw out there: trio-asyncio could do some hacky stuff like, find the loop by walking up the task tree and checking for Task.context[trio_asyncio_cvar]. (Hmm. I guess that's similar to your version, except you're pointing out that for this to work 100% accurately, you need to attach the magic value to a nursery, not a task? The really hacky thing would be for trio-asyncio to just jam an attribute on the nursery it creates, like loop_nursery.__trio_asyncio_loop__ = loop.)

Should all contextvars work this way? It's a bit arbitrary whether new tasks inherit their context from the task that called start_soon or the task that holds the nursery. Currently it's from the task that called start_soon, but one could imagine switching this (and you mentioned wishing we could). So we should pause and consider whether the current behavior is ever useful.

I think it actually is. Here's a fairly realistic example: consider a web app, that needs to spawn a background task in response to a request. (And maybe there will be other requests later to check on the status of the background job, etc.) Generally this will require a global nursery to handle the background tasks, and a request handler that spawns a task into this nursery, and then reports back that the task has been started.

Now, let's say you're using a logging library that assigns each request a unique id, and then attaches that to all log messages generated by that request. You probably want that request-id to propagate into the background task, so any logs it generates can be traced back to the originating request. (Or at the least, that's more useful than every background task losing the logging context entirely!)

That's what Trio gives you now, with context propagating along start_soon.

If we do it, how should we implement it? Instead of tuples like (scoped contextvar's value, task that set it), wouldn't it be simpler to just have a Task.scoped_context, and ScopedContextVar.get() does return current_task().scoped_context[self._cvar]? Nurseries would capture it when created, and tasks would copy it from their parent nursery.

[njsmith]

More potential use cases to think about:

• this came up while discussing control-C handling, and the idea that protection should be inherited across tasks and nurseries following the same path that the exception would follow

• in Global deadlock detector #1085, @smurfix pointed out that it would be nice to have the option of marking a task as "not counting as active" for deadlock detection purposes – examples would include the run_sync_soon trio re-entry task, and potentially other system tasks like watchdog heartbeats or global connection pool managers ([discussion] [potential heresy] global connection pools and global tasks hip#125). It would make sense for this state to be inherited along a task tree as well.

Of course both of these cases involve deep involvement with trio's internals, so it would be easy to handle them as special cases, by putting some dedicated attributes on Task or whatever. I'm not sure if that means that this is a more general thing that lots of libraries will need, or if it's something that only comes up in these kinds of special cases.

[oremanj]

I don't know if I'd say that a lot of libraries would need it, but I think it's a feature that will be useful to libraries in at least some cases, and I don't think it costs much to support it in Trio core. This also fits with our principle of "as much of Trio as possible should be implemented in ways that don't require magic access to internals".

[oremanj]

Another use: Trio wants to clean up async generators in one way (#265); asyncio uses a different approach. trio-asyncio should use the right set of async generator semantics for the flavor of code that first iterates the async generator. A scopevar is the natural way to implement this.

I think this pushes me over the edge into "yes, useful".

[njsmith]

Isn't that just a matter of being able to sniff out which mode we're running in at any given moment? Trio-asyncio already has ways to do that, doesn't it?

[oremanj]

I was imagining that the global asyncgen hooks (presumably installed by Trio) would check the scopevar for an optional override of "what hooks to use in this context". That way Trio doesn't have to be directly aware of trio-asyncio. If the hooks were specific to trio-asyncio, then I agree that they could just use the sniffio state, but I think having trio-asyncio reliably install its own global hooks might be tricky in practice.

[smurfix]

trio-asyncio just uses a contextvar to store the loop. That works for applications because you typically have one loop at top level and there is no "outside" to call into trio-asyncio from.

It does not work quite as well for a Trio library which wants to use some asyncio code.
Imagine task A starting the pseudo-asyncio loop, setting up a connection, and exhibiting that to the rest of the program. Now if an "outside" task B comes along and calls that library, you get an interesting error.

I'd recommend to simply add a with_value_of(contextvar,func,*args) method to the nursery. That method would grab the contextvar's value from the nursery's context, sets the contextvar to it, and call the function.
The aforementioned library could then simply use that to restore the nursery's value of trio_asyncio.current_loop in its entry points. That's more efficient than always shlepping two contextvar stacks around, which you'd need to do because the very same library might want to inherit another contextvar, like current_remote or asgi.request or websocket or …, from the caller, not from the nursery.

Untested code:

class Nursery:
    …
    def with_value_of(self, contextvar, func, *args):
        ctx = self.parent_task.context
        val = ctx[contextvar]
        token = contextvar.set(val)
        try:
            return func(*args)
        finally:
            contextvar.reset(token)

[oremanj]

I'd recommend to simply add a with_value_of(contextvar,func,*args) method to the nursery. That method would grab the contextvar's value from the nursery's context, sets the contextvar to it, and call the function.

The problem is that there isn't currently any "nursery's context". There is the nursery's parent task's context, but that might reflect changes that were made after opening the nursery and would otherwise only ever be visible to the body of the nursery's async with block. If you want to know the state when the nursery was created, you have to save it.

That's more efficient than always shlepping two contextvar stacks around, which you'd need to do because the very same library might want to inherit another contextvar, like current_remote or asgi.request or websocket or …, from the caller

I agree that we can't treat all contextvars in this scoped fashion -- I think @njsmith did a good analysis of that above. But I think having one context for traditional contextvars and another for scopevars will work out fine in practice. Copying contexts is really cheap -- that's almost their defining feature, relative to something more mundane like a dictionary. This is important in asyncio where every single callback registration does a context copy. I think Trio can get away with doing two context copies rather than one on each new task creation.

[complyue]

with_value_of(contextvar,func,*args)

If we provide a similar API for frameworks and libraries like:

• with_effect(contextvar, eff_func, business_func, *args)
  define an effectful behavior identified by contextvar, in form of a function, then call a business function with that effect in context

• perform_effect(contextvar, *eff_args)
  call an effect with arguments, either directly from business_func as passed above, or some nested (sync called or awaited) business functions called from business_func, those functions may have been imported from other modular libraries merely aware of the value of contextvar and the expected duck typing rules.

then pretty much an effects system is implemented, relating to conventional algebraic effects construct, we use a contextvar plus duck typed function arguments in place of an Algebra Data Type which specifies the call convention in a well typed way.

data SomEffect = SomeEffect {
    arg1 :: Int
  , arg2 :: String
  };

While functional language implementers tend to use continuation passing to implement effect calls, so even exception handling could be implemented by a library as an effect, we don't have to go that way, with a procedural imperative language where exception handling is already established.

And effect calling don't have to always be about function calls, resolving a value in context is perfect valid use case IMO, and such a pair of APIs (with_effect/perform_effect) is so generic that frameworks and libraries can choose effect identifiers (contextvar as in this example) according to ad-hoc needs, regardless of any implementation details.

[oremanj]

An unresolved question: what should happen around nursery.start() when tasks can move from one scope to another? For some uses of scopevars (like "here is your trio-asyncio loop") you want the task to have access to the ultimate value from the beginning, while for others ("should I raise KeyboardInterrupt?") the behavior should change at the time of the call to task_status.started().

[complyue]

Sorry I'm not familiar with Trio yet. But I'm surprised to hear a task can move between scopes, isn't the idea of structural concurrency to describe tasks with lexical scoping, how can they move ? Or the task @oremanj refers to is actually an asyncio task? Does Trio use asyncio loops/tasks under the hood just as resources ?

[njsmith]

@complyue

with_effect(contextvar, eff_func, business_func, *args)
perform_effect(contextvar, *eff_args)

That's certainly a thing that you can do. But it's not clear to me what the goal is. What problem are you trying to solve?

But I'm surprised to hear a task can move between scopes, isn't the idea of structural concurrency to describe tasks with lexical scoping, how can they move ? Or the task @oremanj refers to is actually an asyncio task? Does Trio use asyncio loops/tasks under the hood just as resources ?

nursery.start is a special feature for tasks that have a complex "startup" phase: https://trio.readthedocs.io/en/stable/reference-core.html#trio.Nursery.start

We don't support moving tasks between scopes in general, but for nursery.start in particular, it basically starts out the task running under the start call, and then once it's successfully started up it moves it into the final nursery.

Trio is completely separate from asyncio; we don't use any asyncio code.

@oremanj

For some uses of scopevars (like "here is your trio-asyncio loop") you want the task to have access to the ultimate value from the beginning, while for others ("should I raise KeyboardInterrupt?") the behavior should change at the time of the call to task_status.started().

My first intuition for nursery.start+KeyboardInterrupt is that during the startup phase you want it to be protected if the starting context OR the final context have protection enabled? Consider the case where start is unprotected but the nursery is protected: I have this image of injecting a KeyboardInterrupt into the task during the startup phase, but then before it propagates out the task calls task_status.started(), switches into the nursery, and then the KeyboardInterrupt arrives in a nursery that was supposed to be protected.... this would require a lot of things to go wrong at once, but still.

This would be pretty straightforward to handle as a one-off in start, no matter how we end up storing the KeyboardInterrupt protection flag. But it feels consistent with the idea that the KeyboardInterrupt problem is a bit of a special case inside Trio, not just an instance of a more general problem.

I was imagining that the global asyncgen hooks (presumably installed by Trio) would check the scopevar for an optional override of "what hooks to use in this context". That way Trio doesn't have to be directly aware of trio-asyncio. If the hooks were specific to trio-asyncio, then I agree that they could just use the sniffio state, but I think having trio-asyncio reliably install its own global hooks might be tricky in practice.

I was thinking that when trio-asyncio would have hooks like:

if in_asyncio_mode:
    ...
else:
    call_original_hook(...)

And every time it starts a loop, it would check if it has its own hooks installed, and if not then install them.

You can't use trio-asyncio unless you're already using trio, and trio will install its hooks unconditionally at the beginning of trio.run, so I think this would be 100% reliable? The only way I can think of where it'd go wrong would be if there's yet a third library also fighting over the hooks in the same thread, but that's a challenge for the scopevar version too :-).

---

Overall, I'm just not quite comfortable that we understand the situation well enough yet to commit to adding a whole new variable mechanism...

I want to think harder about what exactly trio-asyncio needs, since that seems like the on that's potentially really compelling.

I'm also pondering: the critical thing about the ContextVar API specifically is that it's not our API, it's a generic API that gets used by all kinds of things that aren't just Trio specific. So we can provide whatever constellation of semantics we want, but deciding which semantics to apply to ContextVar specifically is important b/c that will by default affect all these libraries that aren't thinking about Trio at all.

One case is logging libraries. I suspect that these prefer the "causal" context propagation along start_soon, as discussed above.

Another case is decimal, which uses a ContextVar to store the decimal context. Say we have code like:

with decimal.localcontext() as ctx:
    ctx.prec = 42
    nursery.start_soon(...)

Would users expect that high-precision setting to be inherited by the child task, or not?

[njsmith]

Also, I'm not sure "scope" is the best name here... a lot of things are scopes :-). Maybe TreeVar?

[complyue]

A simplified motivating example for my case, but maybe less relevant to Trio so far:

class CurrSymbol:
  @static
  def curr():
    ...

class Price:
  def __init__(self, field='close'):
    self.field=field
  def tensor():
    return cxx.Price(CurrSymbol.curr(), self.field)

class RSI:
  def __init__(self, n, upstream=None):
    self.n=n
    self.upstream=upstream
  def tensor():
    return cxx.RSI(self.n, (self.upstream or Price()).tensor())

class SMA:
  def __init__(self, n, upstream=None):
    self.n=n
    self.upstream=upstream
  def tensor():
    return cxx.SMA(self.n, (self.upstream or Price()).tensor())

def trade(ind1, ind2):
  tensor1 = ind1.tensor()
  ...

def uiTradeBtn():
  sym = uiSymDropdown.sel() # AAPL
  ctor1 = eval(uiIndCode.text()) # SMA(10, RSI(5))
  with CurrSymbol(sym):
    trade( ctor1, ... )

This is like something in my current system with synchronous Python, assembling computation graph from tensors written in C++, CurrSymbol may need to be ported to using contextvar to work with async Python. Real case is more complex, some tensors may reference tensors created from different other symbols, so dynamically nested with CurrSymbol(): do exist, and during running of the tensors, the constructors will need to be used to construct more tensors according to the situation detected, with respective context setups necessarily different.

In my next generation system, with an effect system and dedicated syntax in Edh, I'll write it like:

class Price {
  method __init__(field as this.field='close') pass
  method tensor() {
    hs.Price(perform CurrSymbol, field)
  }
}

class RSI {
  method __init__(
    n as this.n,
    upstream as this.upstream=None,
  ) pass
  method tensor() {
    hs.RSI(n, case this.upstream |> Price() of
      { upstream } -> upstream.tensor() )
  }
}

class SMA {
  method __init__(
    n as this.n,
    upstream as this.upstream=None,
  ) pass
  method tensor() {
    hs.SMA(n, case this.upstream |> Price() of
      { upstream } -> upstream.tensor() )
  }
}


method trade(ind1, ind2) {
  tensor1 = ind1.tensor()
  ...
}

method uiTradeBtn() {
  sym = uiSymDropdown.sel() # AAPL
  effect CurrSymbol = sym
  ctor1 = eval(uiIndCode.text()) # SMA(10, RSI(5))
  trade( ctor1, ... )
}

to assembly tensors written in Haskell.

I feel similar needs could exist for contextual resources in web request handling, but don't have a concrete example.

[smurfix]

Would users expect that high-precision setting to be inherited by the child task, or not?

As a user I would either set up that context inside the task (if it's task-specific anyway), or before creating the nursery (if it is not).
The "before creating the nursery" part is relevant here because if we're talking about a client/server/worker setup, the server might start a task that sets up the worker environment and then opens a nursery which individual workers are started into. Currently this works OK if the worker environment contains a queue-of-requests and the task starting a worker is part of the worker's nursery, like this:

class WorkerController:
    async def run(self,*,task_status=trio.TASK_STATUS_IGNORED):
        self._q = Queue()
        with some_interesting_context() as ctx:
            async with trio.open_nursery() as n:
                task_status.started()
                async for whatever in self._q:
                    n.start_soon(self.do_work, whatever)
    async def start_job(self, whatever):
        await self._q.put(whatever)

On the other hand, if workers are started without queuing the job, as in

class WorkerController:
    async def run(self,*,task_status=trio.TASK_STATUS_IGNORED):
        with some_interesting_context() as ctx:
            async with trio.open_nursery() as n:
                self._nursery = n
                task_status.started()
                await trio.sleep(math.inf)
    async def start_job(self, whatever):
        self._nursery.start_soon(self.do_work, whatever)

they won't inherit that context. This may be kindof surprising. The problem is of course that things like decimal's context are contextvars just like asgi.request, but applying different rules to these depending on our whim won't work.
So we're left with either inventing a way to teach a nursery which contextvars are to be imported from their current setting in the nursery's task and which are cloned from the task calling nursery.start … or we make that explicit, as per my earlier suggestion … or we document the hell out of the fact that the second code snippet above doesn't work and must be replaced with

class WorkerController:
    async def run(self,*,task_status=trio.TASK_STATUS_IGNORED):
        with some_interesting_context() as ctx:
            async with trio.open_nursery() as n:
                self._nursery = n
                self._ctx = ctx
                task_status.started()
                await trio.sleep(math.inf)
    async def start_job(self, whatever):
        self._nursery.start_soon(self._ctx.run, self.do_work, whatever)

or maybe

class WorkerController:
    async def run(self,*,task_status=trio.TASK_STATUS_IGNORED):
        async with trio.open_nursery() as n:
            self._nursery = n
            task_status.started()
            await trio.sleep(math.inf)
    async def start_job(self, whatever):
        with some_interesting_context() as ctx:
            self._nursery.start_soon(self.do_work, whatever)

[oremanj]

Also, I'm not sure "scope" is the best name here... a lot of things are scopes :-). Maybe TreeVar?

Agreed that that's a much better name!

Consider the case where start is unprotected but the nursery is protected: I have this image of injecting a KeyboardInterrupt into the task during the startup phase, but then before it propagates out the task calls task_status.started(), switches into the nursery, and then the KeyboardInterrupt arrives in a nursery that was supposed to be protected.... this would require a lot of things to go wrong at once, but still.

That's a good point, and KI is edge-triggered so we can't fall back on the "just don't move it and it'll get cancelled soon enough" logic we currently use to avoid an analogous problem with cancellations. So we need some special logic for KeyboardInterrupt... but I think a piece of task-local state that propagates along parent/child task relationships is still going to be a useful input to that logic!

You can't use trio-asyncio unless you're already using trio, and trio will install its hooks unconditionally at the beginning of trio.run, so I think this would be 100% reliable? The only way I can think of where it'd go wrong would be if there's yet a third library also fighting over the hooks in the same thread, but that's a challenge for the scopevar version too :-).

Based on your comments in #265, I think we will have at least two libraries that want to extend asyncgen hooks behavior (trio-asyncio and tricycle), but not in the same parts of the task tree. That doesn't necessarily mean Trio needs to support task-local asyncgen hooks out of the box, because it's possible for each library to independently do something that composes well:

• firstiter hook: see if we're in a part of the task tree that wants this library's behavior, call our "real" firstiter hook if so, call the previously installed firstiter hook if not
• finalizer hook: see if this asyncgen was delivered to our firstiter hook, call our finalizer hook if so, call the previously installed finalizer hook if not

But that sort of falls apart if you can't determine "are we in a part of the task tree that wants this library's behavior?". trio-asyncio has an existing mechanism for this, because it can control the environment in which every individual step of an asyncio task runs. But that's not an option for everyone.

I'm also pondering: the critical thing about the ContextVar API specifically is that it's not our API, it's a generic API that gets used by all kinds of things that aren't just Trio specific. So we can provide whatever constellation of semantics we want, but deciding which semantics to apply to ContextVar specifically is important b/c that will by default affect all these libraries that aren't thinking about Trio at all.

Yeah, I'm sufficiently convinced that for normal contextvars the current behavior is fine. For anything that doesn't inherently "expire" at the end of some scope, the causal propagation is strictly more flexible in common cases (where tasks are spawned into a nursery from within that nursery), and if you really need to pick up the nursery's ambient context when spawning from outside of the nursery, you can do nursery.parent_task.context.copy().run(nursery.start_soon, ...).

Overall, I'm just not quite comfortable that we understand the situation well enough yet to commit to adding a whole new variable mechanism...

I'm sympathetic to this caution; I just don't see any good way to experiment with this without support from the Trio core. The best compromise I've been able to come up with, in terms of a minimal-size change to Trio itself, is to specify that nurseries capture their enclosing context as Nursery.context and tasks get an attribute named something like Task.eventual_parent_nursery that specifies where the task will be running after an eventual call to task_status.started(). I believe this gives enough rope to implement TreeVar outside Trio, at the cost of having to visit all parents of the current task each time a TreeVar is first accessed in a new task. The asymptotics of that aren't wonderful, but I think it would work OK in practice.

But I think the argument for TreeVar in Trio is pretty strong:

• trio-asyncio would use it for its loop cvar
• it would be useful in implementing Rethink KeyboardInterrupt protection APIs #733, even though it's not the whole solution
• it would more cleanly resolve the issue of different libraries wanting to do different things with async generators in different parts of the task tree
• it doesn't seem likely to me to make other features harder to implement (though maybe I'm missing something there?)

[richardsheridan]

I implemented a cache_scope feature for trio-parallel under the misguided assumption that a ContextVar would behave as a TreeVar does. This lead to a couple failures that are along the lines of what @oremanj and @smurfix wrote above:

• When using a guest mode app in which callbacks from the host loop enter Trio via a reference to a nursery, nursery.start_soon always has the global context. There is no way to have tasks launched this way share a context transparently.
• If a nursery scope encloses a cache_scope, or a nursery is otherwise passed around and launches tasks with the cache_scope context, the context leaks from the context manager in any task launched by start_soon that outlives cache_scope, and so I cannot transparently guarantee that every worker is reaped at the scope exit.

Now, these problems don't seem insurmountable. I could either build my own TreeVar or CancelStatus lookalike or I could change my application to avoid these issues. But TreeVar is exactly what I wanted and makes implementing my feature extremely ergonomic. Also, if Trio's cancellation is refactored to run on top of TreeVars, then my project doesn't have to do any testing to make sure that my scopes match CancelScope semantics with respect to, for example, task_status.started().

In short, +1 to merge #1543 ASAP.

[oremanj]

TreeVar is implemented in the latest release of tricycle. It's a little less performant than it would be with core support, but should have equivalent semantics.

I would be happy to revive #1543 if there's support for it being merged, but I haven't seen any further input on that PR or this issue since I left my last comment.

[smurfix]

FWIW, I'd like to have them.

I haven't seen any further input on that PR or this issue since I left my last comment.

Did you miss @richardsheridan's +1?