[wasi] async/await on WASIp2

[dicej]

I've been working to support async/await in C# for doing concurrent I/O when targeting WASIp2. A key piece of that is the wasi:io/poll#poll function, which accepts an arbitrary number of pollable handles representing e.g. socket readiness, HTTP protocol events, timer events, etc. It's analogous to the traditional POSIX poll(2) function. My goal is to provide a System.Threading.Tasks-based abstraction on top of wasi:io/poll that supports idiomatic use of async/await, including the various Task combinators such as WhenAny, WhenAll, WhenEach. I've done similar work for Python (using a custom asyncio event loop) and Rust (using a custom async runtime), and am hoping to do the same for .NET.

So far, I've managed to write a custom TaskScheduler which supports simple cases by maintaining a list of Tasks and a list of the Pollables those tasks are awaiting. It has a Run method which, in a loop, alternates between the Task list and the Pollable list, executing the former and calling wasi:io/poll#poll on the latter. That works well for simple cases.

However, I've had trouble building more sophisticated examples using e.g. the new Task.WhenEach combinator due to the somewhat pervasive use of ThreadPool as a deferred execution mechanism throughout the System.Threading.Tasks library code. Given that WASI does not support multithreading and won't for the foreseeable future, ThreadPool's methods currently throw System.PlatformNotSupportedException, which makes it something of a landmine. For example, even though there's nothing inherently multithreaded about Task.WhenEach, the current implementation relies on a ManualResetValueTaskSourceCore with RunContinuationsAsynchronously set to true, which means it always queues continuations using ThreadPool.UnsafeQueueUserWorkItem.

Given that significant pieces of .NET's async/await infrastructure currently relies on multithreading to function, I'm wondering what our options are for WASI. A few come to mind:

• Give up on async/await and use callbacks for concurrency in WASIp2. Besides being un-ergonomic, it would significantly restrict the set of both standard and third-party library features available for use on that platform, given that anything that deals with I/O is presumably either synchronous and multithreaded or asynchronous based on async/await.
• Support async/await, but disallow use of features such as Task.WhenEach which require ThreadPool as an implementation detail, and possibly provide alternative implementations of those features which are single-thread-friendly.
• Refactor the parts of System.Threading.Tasks which are not inherently multithreaded (but currently use ThreadPool) to support an alternative, single-threaded deferred execution mechanism on platforms that do not support multithreading.
• Provide a WASI-specific ThreadPool implementation which simply queues work items without executing them, and provide some public API for running them from the main (and only) thread, e.g. as part of an application event loop.

Thoughts?

I should note that I'm quite new to the .NET ecosystem, so I'm happy to be corrected if I've misunderstood anything.

See also #98957, for which I would consider this issue a prerequisite.

[dotnet-policy-service]

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

[lewing]

cc @agocke

[pavelsavara]

In Mono/Browser when single-threaded, we have thread-less thread pool implemented via emscripten Module.safeSetTimeout which is just wrapper around JavaScript setTimeout.

runtime/src/mono/System.Private.CoreLib/src/System/Threading/ThreadPool.Browser.Mono.cs

Lines 82 to 88 in 9fca0c3

	internal static unsafe void RequestWorkerThread()
	{
	if (_callbackQueued)
	return;
	_callbackQueued = true;
	MainThreadScheduleBackgroundJob((void*)(delegate* unmanaged[Cdecl]<void>)&BackgroundJobHandler);
	}

runtime/src/mono/mono/utils/mono-threads-wasm.c

Lines 348 to 386 in 9fca0c3

	// when this is called from ThreadPool, the cb would be System.Threading.ThreadPool.BackgroundJobHandler
	// when this is called from sgen it would be wrapper of sgen_perform_collection_inner
	// when this is called from gc, it would be mono_runtime_do_background_work
	#ifdef DISABLE_THREADS
	void
	mono_main_thread_schedule_background_job (background_job_cb cb)
	{
	g_assert (cb);
	THREADS_DEBUG ("mono_main_thread_schedule_background_job: thread %p queued job %p to current thread\n", (gpointer)pthread_self(), (gpointer) cb);
	if (!jobs)
	schedule_background_exec ();
	if (!g_slist_find (jobs, (gconstpointer)cb))
	jobs = g_slist_prepend (jobs, (gpointer)cb);
	}
	GSList *jobs;
	G_EXTERN_C
	EMSCRIPTEN_KEEPALIVE void
	mono_background_exec (void)
	{
	MONO_ENTER_GC_UNSAFE;
	GSList *j = jobs, *cur;
	jobs = NULL;
	for (cur = j; cur; cur = cur->next) {
	background_job_cb cb = (background_job_cb)cur->data;
	g_assert (cb);
	THREADS_DEBUG ("mono_background_exec on thread %p running job %p \n", (gpointer)pthread_self(), (gpointer)cb);
	cb ();
	THREADS_DEBUG ("mono_background_exec on thread %p done job %p \n", (gpointer)pthread_self(), (gpointer)cb);
	}
	g_slist_free (j);
	MONO_EXIT_GC_UNSAFE;
	}
	#else /*DISABLE_THREADS*/

runtime/src/mono/browser/runtime/scheduling.ts

Lines 47 to 67 in 9fca0c3

	function mono_background_exec_until_done () {
	if (WasmEnableThreads) return;
	Module.maybeExit();
	try {
	while (pump_count > 0) {
	--pump_count;
	if (!loaderHelpers.is_runtime_running()) {
	return;
	}
	cwraps.mono_background_exec();
	}
	} catch (ex) {
	loaderHelpers.mono_exit(1, ex);
	}
	}
	export function schedule_background_exec (): void {
	if (WasmEnableThreads) return;
	++pump_count;
	Module.safeSetTimeout(mono_background_exec_until_done, 0);
	}

runtime/src/mono/System.Private.CoreLib/src/System/Threading/ThreadPool.Browser.Mono.cs

Lines 124 to 136 in 9fca0c3

	// this callback will arrive on the bound thread, called from mono_background_exec
	private static void BackgroundJobHandler()
	{
	try
	{
	_callbackQueued = false;
	ThreadPoolWorkQueue.Dispatch();
	}
	catch (Exception e)
	{
	Environment.FailFast("ThreadPool.BackgroundJobHandler failed", e);
	}
	}

This has benefit of yielding to the browser event loop.
Yielding is necessary so that JS Promises could get resolved, for example fetch.

Hope this helps.

I would like to learn more on how yielding out of the current (dotnet) component need to look like for WASI p2/p3.

[pavelsavara]

(Just thinking aloud)

The list of "jobs" we have in the thread-less thread pool solves all non-external Tasks and their dependencies.
Also note that any locks are NOOP in our single-threaded build.
And the browser (host) is calling back to us when there is setTimeout tick or when JS promise gets resolved.

Do I understand it right that in WASI instead of being called by host on "event" we are expected to have loop in some main() and do poll() on all external promises.

Meaning that WASI promises will become resolved by the WASI host even without us yielding ?

Would that poll loop still work in preview 3 too ?
Would that poll loop still work in multi-threaded too ?
Does it need to be re-entrant ? (dotnet calls another component synchronously, which calls back to our async export)
I'm concerned about WASM stack.

Could wasi-clocks play the role of setTimeout ? Would it help to anything ?

How would that look like if we are WASI reactor which exposes async API ?
Should all async (component) exported functions implement such poll loop ?

[pavelsavara]

To be able to exit that poll() loop we need to know if there is no more work to do.
Probably it means that all Tasks are resolved/rejected/canceled.
And unresolved Tasks could depend on external promises, which is OK.

(For context: In browser we marshal JS Promise as C# Task, when the JS calls us .then(()=>{ ... callCSharp() }) we call TaskCompletionSource.SetResult for the marshaled Task)

I'm not sure what would be the answer if there are multiple-threads in WASI. Should we have multiple poll() loops ?
Do WASI promises have thread affinity ? (in JS they do)

Does WASI support cancellation ? In preview 3 ?

[pavelsavara]

Is pollable concept going to survive preview3 ?

Is pollable equivalent of a promise which could resove just once ?
Or it's a wait handle/stream/semaphore, which could be lifted multiple times ?

If multiple times, are we going to create Task instance for each promised "event" of the pollable ?

[agocke]

To simplify the discussion slightly, I don't think there is any strong dependency on multi-threading with the .NET async model. It is possible that you will need to use a different default scheduler in your app model, but I don't know of any requirement that multi-threading is required for .NET async.

@stephentoub would probably be the best person to chime in on specific technical decisions that would be recommended for this area.

[dicej]

For reference, the Discord discussion about this starts here: https://discord.com/channels/732297728826277939/732297825731215521/1245818492322844673

[dicej]

Do I understand it right that in WASI instead of being called by host on "event" we are expected to have loop in some main() and do poll() on all external promises.

(@pavelsavara discussed this on Discord, but I'll answer here also for the record). Yes, WASIp2 requires the guest to run an event loop (maybe in main, but more generally any export, e.g. wasi:http/incoming-handler). WASIp3 will change that by using Component Model support for async imports and exports, at which point the event loop will move to the host.

Meaning that WASI promises will become resolved by the WASI host even without us yielding ?

Yes, the host can make progress on the promises regardless of if or when the guest yields.

Would that poll loop still work in preview 3 too ? Would that poll loop still work in multi-threaded too ? Does it need to be re-entrant ? (dotnet calls another component synchronously, which calls back to our async export) I'm concerned about WASM stack.

Quoting my Discord comments:

WASIp3 will be based on a major update to the Component Model which supports async imports and exports. The async exports may either be "stackless" such that the guest returns a task handle to the host and exports a callback for progress notifications, or "stackful", meaning the guest blocks on a poll-style intrinsic and supports concurrent calls on other fibers.

We expect languages that have stackless, async/await style coroutines such as JS, Python, Rust, and C# will want to use the callback approach, in which case the application developer will export async functions and wit-bindgen will handle the details of assigning a unique handle to each Promise/Task/Future and maintaining the lookup table @pavel Savara described. I've already implemented this for Rust: https://github.com/dicej/component-async-demo/. This mechanism also allows application code to "spawn" tasks which may continue to run after the export function has returned a result (e.g. https://github.com/dicej/component-async-demo/blob/aba5ebf363d5830cbab20bf9a453f927c47e0605/http-echo/src/lib.rs#L48-L59). The host runs the top level event loop, dispatching events to the appropriate callback exports with the appropriate handles, which the guest will translate to Promises/Tasks/Futures and resume them.

Could wasi-clocks play the role of setTimeout ? Would it help to anything ?

Yeah, I was thinking the same thing, but I don't think we need to involve the host with deferred execution at all -- it can all be taken care of in some combination of the .NET runtime and wit-bindgen-generated code.

How would that look like if we are WASI reactor which exposes async API ? Should all async (component) exported functions implement such poll loop ?

All exported functions that want to use async/await will need to run an event loop of some kind, but we could make that an implementation detail in wit-bindgen so the app developer only needs to write an async function and let the generated code add the event loop implicitly.

[dicej]

I'm not sure what would be the answer if there are multiple-threads in WASI. Should we have multiple poll() loops ? Do WASI promises have thread affinity ? (in JS they do)

It's going to be a while before WASI has threads (i.e. post-WASIp3), so I don't think we need to worry about that now. By then, the event loop(s) will be in the host instead of the guest, so the .NET toolchain won't need to deal with it anyway beyond generating thread-safe code.

Does WASI support cancellation ? In preview 3 ?

WASIp2 does not support it, and p3 probably won't either, although I've been doing design work on cancellation with @lukewagner and we plan to support it post-p3.

[dicej]

Is pollable concept going to survive preview3 ?

No.

Is pollable equivalent of a promise which could resove just once ? Or it's a wait handle/stream/semaphore, which could be lifted multiple times ?

It can resolve more than once. I'm not a big fan of that since it makes the host implementation more difficult, but that's how it is :)

If multiple times, are we going to create Task instance for each promised "event" of the pollable ?

Good question. I would say that just because you can use a pollable more than once doesn't mean you have to, and if it makes things simpler we can just always only use it once. That's been my approach for async support in Rust and Python.

[dicej]

To simplify the discussion slightly, I don't think there is any strong dependency on multi-threading with the .NET async model. It is possible that you will need to use a different default scheduler in your app model, but I don't know of any requirement that multi-threading is required for .NET async.

Yeah, I don't think it's intentional, it just so happens that you currently can't use e.g. Task.WhenEach on WASI since it always tries to queue actions to ThreadPool, which immediately throws a PlatformNotSupportedException. If WASI were to get a usable ThreadPool implementation that just defers work without trying to spawn threads (which is starting to look like the best option), then that won't be a problem anymore.

@stephentoub would probably be the best person to chime in on specific technical decisions that would be recommended for this area.

I've been reading a lot of his blog posts lately while getting up-to-speed on how async works in .NET, so yeah, I'd love to get his perspective :)

[stephentoub]

To simplify the discussion slightly, I don't think there is any strong dependency on multi-threading with the .NET async model. It is possible that you will need to use a different default scheduler in your app model, but I don't know of any requirement that multi-threading is required for .NET async.

@stephentoub would probably be the best person to chime in on specific technical decisions that would be recommended for this area.

That's right. The async/await implementation in .NET does need some kind of scheduler that work can be queued to, but it's fine for that scheduler to just be for a single thread pumping in an event loop.

I do think you'll want to reroute ThreadPool.QueueUserWorkItem and friends to just schedule to the single event loop. There's a ton of code out there, including but not limited to the async/await infrastructure, that uses ThreadPool directly, or stand-ins for it (like Task.Run), and all of that code should be able to "just work" if those work items are routed to that dispatcher: it just happens it's a pool of size 1.

[dicej]

Thanks for all the feedback. I'm going to try to summarize what I've read (or inferred) so far:

• WASI should have a ThreadPool implementation that does the right thing for that platform: queue items to be executed by a top-level, single-threaded event loop.
• The application developer should not generally be responsible for that event loop or even know of its existence, i.e. it should be run implicitly by either wit-bindgen or the .NET runtime.
  • Instead, the application developer should simply export async function(s) to the host and trust that the details will be taken care of.
  • This will also help ease the burden of migrating from WASI p2 to p3, at which point the event loop will be the host's responsibility instead of the guest's. Application code that exports async function(s) should work fine in both cases.
• For WASIp2, we'll need some way to convert wasi:io/poll/pollable handles into Tasks, implicitly adding those handles to the set to be polled by the event loop.

Please let me know if I've misunderstood or left out anything important.

With the above in mind, here's a concrete proposal to drive further discussion:

• Create a ThreadPool.Wasi.cs that queues items for execution by the top-level event loop.
• Add a new WasiEventLoop class with two methods:
  • public static T Run<T>(Task<T> task): Runs the event loop, which is responsible for:
    • driving the async state machines
    • running any pending ThreadPool work items
    • blocking on wasi:io/poll/poll as necessary, passing any pollable handles which have been registered
  • public static Task Register(Pollable pollable): Registers the specified pollable with the event loop, returning a Task which will resolve when the pollable resolves.
• Add support to wit-bindgen for optionally generating async export bindings, generating a call to WasiEventLoop.Run for each export, passing the task returned by a call to the application-provided implementation.
• Also in wit-bindgen, specialize any WASI functions which return pollable so that they return Task instead. The generated code would call WasiEventLoop.Register to perform the registration.
• Possibly add some compiler magic to support static async void Main functions which uses WasiEventLoop.Run behind the scenes? I'm not sure if this would be useful as opposed to using wit-bindgen to target wasi:cli, but maybe?
• Once WASIp3 is released, add support for it in the next .NET release (version 10, probably?). We won't need WasiEventLoop as described above since the event loop will be run in the host, but we'll still need some way for wit-bindgen to run any pending ThreadPool work items, e.g. just prior to returning control to the host.

Note that WasiEventLoop is meant to be an implementation detail hidden from the application developer, although we could make it part of the supported public API if we wanted to give devs fine-grained control.

Thoughts?

[pavelsavara]

If we keep WasiEventLoop private rather than public API of the runtime, then the code generated by wit-bindgen needs to generate the unsafe accessors that SingleAccretion mentioned in the discord. I guess that's fine for now.

I think that we will also have to protect it from IL trimming (at least for Mono).

For registering external pollable, when would we free that handle ?
After the Task is resolved ?
Do we need to worry about pollable or p3 promises that never resolve, but the Task which was using them is GCed ?

I'm also missing the part in which dotnet Task is returned/exported as Pollable. Could you please comment on that ?
Such pollable needs to keep the TaskCompletionSource alive, probably via GCHandle.
What allocates the low level pollable handle (number/id) when we need to create one ?
Are we told that such pollable is not referenced anymore by the host or another component ?

[dicej]

For registering external pollable, when would we free that handle ? After the Task is resolved ?

Yes; I've been doing the equivalent in Rust and Python, and that's what I'm doing here: https://github.com/dicej/dotnet9-wasi-http-example/blob/da7541017f944247e7614a3cad2f663508118837/PollTaskScheduler.cs#L45-L52 -- i.e. for each ready pollable, we first dispose of the handle and then call TaskCompletionSource.SetResult.

Do we need to worry about pollable or p3 promises that never resolve, but the Task which was using them is GCed ?

I assume you can call TaskCompletionSource.SetResult safely even if the corresponding Task has been GCed, correct? I.e. it's just a no-op?

Same story for p3 promises -- the host will notify the guest when each one resolves, which will trigger a TaskCompletionSource.SetResult, which will be a no-op if nobody is listening anymore.

I'm also missing the part in which dotnet Task is returned/exported as Pollable. Could you please comment on that ? Such pollable needs to keep the TaskCompletionSource alive, probably via GCHandle. What allocates the low level pollable handle (number/id) when we need to create one ? Are we told that such pollable is not referenced anymore by the host or another component ?

The flow of control would look something like this:

1. The guest application code calls e.g. OutputStream.Subscribe (i.e. the method generated by wit-bindgen for wasi:io/streams/output-stream/subscribe), which is implemented by the host.
2. The host generates a pollable handle representing write readiness for that output-stream and returns it to the guest.
3. In wit-bindgen-generated code, the returned handle is immediately passed to WasiEventLoop.Register, which creates a TaskCompletionSource and adds the (Pollable, TaskCompletionSource) tuple to its pollables list (e.g. https://github.com/dicej/dotnet9-wasi-http-example/blob/da7541017f944247e7614a3cad2f663508118837/PollTaskScheduler.cs#L12-L17), returning the corresponding Task.
4. The application code gets the Task and does whatever it wants with it (e.g. await it, call ContinueWith on it, or whatever; presumably it will want to call OutputStream.Write once the Task resolves).
5. When WasiEventLoop.Run calls wasi:io/poll/poll next, it will pass all the pollable handles in its list, and the host will block until at least one of them is ready.
6. When that call returns a list of ready pollables, WasiEventLoop.Run will dispose each of those pollables and call TaskCompletionSource.SetResult on the corresponding source.
7. Each pollable that was not ready stays in the list so we can poll it again in the next iteration of the loop.

See https://github.com/dicej/dotnet9-wasi-http-example/blob/snapshot/PollTaskScheduler.cs for details. Does that help?

[pavelsavara]

Do we need to worry about pollable or p3 promises that never resolve, but the Task which was using them is GCed ?

I assume you can call TaskCompletionSource.SetResult safely even if the corresponding Task has been GCed, correct? I.e. it's just a no-op?

Sorry, my bad. Pollable will keep TaskCompletionSource alive and that will keep Task alive.

Does that help?

What I mean to ask is that we have export public static Task<string> Foo() which is wrapped as WASI export. That needs to marshal the Task instance that C# created (from C# computation, not from pollable) into new pollable and return it to the host. When C# resolves the Task, we need to signal the host that the pollable was resolved and pass the string which was the promised payload.

We can pass pollable as result or as parameter of a function. And you can do it on imports and exports.
That's 4 combinations, but possibly only 2 ways how to marshal that.
Pollable -> Task
Task -> Pollable

I'm asking about the other direction.

See https://github.com/dicej/dotnet9-wasi-http-example/blob/snapshot/PollTaskScheduler.cs for details.

I'm not sure that you need your own TaskScheduler, I think ThreadPool would be enough. I'm not 100% confident, thought.

[dicej]

What I mean to ask is that we have export public static Task<string> Foo() which is wrapped as WASI export. That needs to marshal the Task instance that C# created (from C# computation, not from pollable) into new pollable and return it to the host. When C# resolves the Task, we need to signal the host that the pollable was resolved and pass the string which was the promised payload.

Ah, I see what you're saying. For WASIp2, there's no concept of guest-created pollables, so we can't actually return a Task, pollable, or anything like that to the host. Instead, the wit-bindgen-generated code needs to call Foo and pass the resulting Task to WasiEventLoop.Run, which will block until the Task resolves. Then it can return the string result to the host. From the host's point of view, it just called a synchronous function and got back a string.

For WASIp3, there will be a new Component Model ABI for async exports, and in that case the guest can and should return to the host if the Task has not resolved. The host will later call the guest via a callback function when progress is made on any of the async imports the guest called. For now, though, I think we should focus on how to support WASIp2 since we're still in the design and prototype phase for WASIp3.

We can pass pollable as result or as parameter of a function. And you can do it on imports and exports. That's 4 combinations, but possibly only 2 ways how to marshal that. Pollable -> Task Task -> Pollable

There's no need for a Task -> Pollable translation since there's no way to represent a guest task/promise/future as a Pollable in WASIp2. All exports are synchronous from the host's perspective.

See https://github.com/dicej/dotnet9-wasi-http-example/blob/snapshot/PollTaskScheduler.cs for details.

I'm not sure that you need your own TaskScheduler, I think ThreadPool would be enough. I'm not 100% confident, thought.

Yes, I think you're right. I just referred to that because I expect WasiEventLoop will closely resemble what I wrote there.

[pavelsavara]

The plan looks good to me.

Let's try it in NativeAOT LLVM branch. And we could catch up with it in Mono later.

@yowl @SingleAccretion do you have any comments ?

cc @silesmo @jsturtevant

[SingleAccretion]

My primary concern is layering, i. e. the question of who provides the event loop implementation (WasiEventLoop above). My preference is for this to live under wit-bindgen (not as generated code, necessarily, but as a supporting assembly/nuget, for example), at least for now. The main reason is that testing it would require the tool.

For reference, the interface between ThreadPool and event loop on Browser is setTimeout(ThreadPool.RunQueuedWorkItems, ...) (including the ability to specify the delay, used for the timer queue). It looks to me the same should work for WasiEventLoop. @dicej is that true?

[dicej]

My primary concern is layering, i. e. the question of who provides the event loop implementation (WasiEventLoop above). My preference is for this to live under wit-bindgen (not as generated code, necessarily, but as a supporting assembly/nuget, for example), at least for now. The main reason is that testing it would require the tool.

That works for me.

For reference, the interface between ThreadPool and event loop on Browser is setTimeout(ThreadPool.RunQueuedWorkItems, ...) (including the ability to specify the delay, used for the timer queue). It looks to me the same should work for WasiEventLoop. @dicej is that true?

I think that should be fine, yes, although I'm having trouble finding a RunQueuedWorkItems function in the source tree. Did you mean BackgroundJobHandler perhaps?

If someone can explain to me the proper way for a hypothetical wit-bindgen runtime helper library to call a private function analogous to ThreadPool.BackgroundJobHandler, then I can take care of the rest. Unlike the browser case, I don't think we can make a host call for something like this -- there's no equivalent to setTimeout in WASI, and it's not really the host's business anyway since the event loop is the guest's responsibility.

[dicej]

Actually, I think a better interface for WASI would be something like private static List<(Action, ulong)> TakeJobs() which returns a list of jobs and their delays in milliseconds, which WasiEventLoop can schedule using e.g. wasi:clocks/monotonic-clock/subscribe-duration (or just run immediately if the delay is zero). Each call to TakeJobs would clear the internal state of ThreadPool, i.e. those jobs become the responsibility of WasiEventLoop.

[SingleAccretion]

Did you mean BackgroundJobHandler perhaps?

Yep. It was just an example function name.

If someone can explain to me the proper way for a hypothetical wit-bindgen runtime helper library to call a private function analogous to ThreadPool.BackgroundJobHandler, then I can take care of the rest.

It doesn't really matter a whole lot what the private interface looks like mechanically. For example, it can be a private static method SetEventLoopQueueFunction(delegate*<delegate*, ...> pQueueFunc) that would set the queue function on the thread pool, with wit-bindgen being expected to call that function somewhere on the startup path.

The main concern is that the interface be small. The thread pool's queue internal machinery is pretty involved. It seems unlikely that we will be able to fit through something like TakeJobs seamlessly (and without losing fidelity, e. g. diagnostics).

[dicej]

Sounds good. How would the wit-bindgen helper library call SetEventLoopQueueFunction then? @pavelsavara mentioned unsafe accessors and IL trimming, but I don't know what that translates to concretely.

[SingleAccretion]

unsafe accessors

See https://learn.microsoft.com/en-us/dotnet/api/system.runtime.compilerservices.unsafeaccessorattribute?view=net-8.0.

[maraf]

Is the WasiEventLoop ideal place to schedule finalizers?

[SingleAccretion]

Is the WasiEventLoop ideal place to schedule finalizers?

No. Finalization should work for fully synchronous applications.

[dicej]

Another question about the division of responsibility between the .NET runtime and wit-bindgen:

I'm currently working on adding a wasi:http-based WasiHttpHandler to System.Net.Http, analogous to the existing BrowserHttpHandler as a backing implementation of HttpClient, per #98957. One of the things WasiHttpHandler will need to do is turn a Pollable into a Task it can await. Earlier, I proposed that WasiEventLoop (to be provided by a wit-bindgen helper library) would have a Register function for this purpose, but we probably don't want WasiHttpHandler to call that directly because it would make the .NET runtime (circularly) dependent on the wit-bindgen helper library. What would be the best way to structure this?

The first thing that comes to mind would be to have the WASI implementation of ThreadPool (EDIT: or some other class, since this isn't a ThreadPool thing per se) include an internal function for converting a Pollable to a Task and have it throw an exception until/unless wit-bindgen generated code sets a handler for it. So wit-bindgen-generated code would be responsible for calling both the SetEventLoopQueueFunction we discussed above and a SetPollableRegisterFunction. The upshot is that you wouldn't be able to use HttpClient on WASI without wit-bindgen being involved in some way, and that includes any tests we add to the .NET runtime, but I don't think there's any plausible way around that (other than writing bindings by hand, which I don't recommend).

I do expect we'll need to bundle wit-bindgen-generated bindings as part of the .NET runtime for WASI so that WasiHttpHandler can use them, as well as for testing. That doesn't mean the .NET runtime will depend on wit-bindgen per se -- just that we'd need to generate the bindings, check them into the repo, and provide instructions and/or a script for regenerating the bindings as needed. For testing, we'd also need to bundle a copy of the helper library that implements WasiEventLoop, but not ship it as part of the runtime.

As usual, I'm mostly thinking aloud here and am open to whatever the experts here think is best.

[pavelsavara]

My primary concern is layering, i. e. the question of who provides the event loop implementation (WasiEventLoop above). My preference is for this to live under wit-bindgen (not as generated code, necessarily, but as a supporting assembly/nuget, for example), at least for now. The main reason is that testing it would require the tool.

I probably don't fully follow. Testing would also require runtime in all cases, right ?

I'm currently working on adding a wasi:http-based WasiHttpHandler to System.Net.Http

I thought that you are going to use wit-bindgen which will generate the code of the wasi:http into the codebase of the runtime.
If we don't want it to be generated on every build of the runtime (and make dependency on rust/bindgen toolchain), we would commit the generated C# code into the runtime repo.

And I though that this generated code would contain the generated implementation of the WasiEventLoop ?
If not why not ?

And if not, perhaps runtime could have

internal interface IWasiEventLoop
{
    void PollAllPollables();
    Task RegisterPollable(Pollable pollable);
}

And static member on the WASI ThreadPool into which the bindgen could install it via "unsafe accessors".

partial class ThreadPool
{
    private static IWasiEventLoop? WasiEventLoop;
    internal static void SetEventLoopQueueFunction(IWasiEventLoop wasiEventLoop)
}

If we don't want to include Pollable type in the runtime, it could be just System.Object for now.

Task RegisterPollable(object pollable)

Or even just the handle value

Task RegisterPollable(uint pollableHandle)

[pavelsavara]

Well extra internal interface type IWasiEventLoop is not making it easier either, so few registered delegates instead is fine too.

[dicej]

I'm currently working on adding a wasi:http-based WasiHttpHandler to System.Net.Http

I thought that you are going to use wit-bindgen which will generate the code of the wasi:http into the codebase of the runtime. If we don't want it to be generated on every build of the runtime (and make dependency on rust/bindgen toolchain), we would commit the generated C# code into the runtime repo.

Yes, that's the plan. I'm writing WasiHttpHandler itself by hand, making use of the generated bindings, which I plan to commit to the runtime repo with instructions and/or a script to regenerate them as needed.

And I though that this generated code would contain the generated implementation of the WasiEventLoop ? If not why not ?

We were leaning towards putting reusable code like WasiEventLoop, Result, and Option into a NuGet package that wit-bindgen-generated code could refer to, which would reduce duplication in cases where an application depends on multiple packages which each use wit-bindgen-generated code. But yes, the amount of reusable code is quite small, so we can just have wit-bindgen "generate it" (i.e. copy it into the output) along with everything else.

[pavelsavara]

WasiEventLoop ... wit-bindgen "generate it" (i.e. copy it into the output) along with everything else.

Eventually that code belongs to runtime repo, I think. I still don't understand the preference to have it as part of wit-bindgen. @SingleAccretion could you please elaborate ?

[SingleAccretion]

could you please elaborate ?

As I said above:

The main reason is that testing it would require the tool.

The event loop must be generated at every entry point. The only entry point the runtime controls is Main - inserting substantial event loop code into UCO methods would go against UCO design principles.

I therefore see async-capable applications as a wit-bindgen feature, since the pollable marshaling is what actually enables true asynchrony.

Additionally, WasiEventLoop is a design with an expiration date on it, which means we can't make it public in /runtime. So even if it were in the runtime, wit-bindgen would have to use private reflection to manipulate it - an odd composition.

This view is not set in stone. If we can come up with a layering that makes more sense given the constraints we have, let's make it happen.

Let's refocus this http issue a bit. Say I am making bindings to APIs that return pollables from wasi:xyz, to be published on nuget.org. How would I do that?

[dicej]

Additionally, WasiEventLoop is a design with an expiration date on it, which means we can't make it public in /runtime. So even if it were in the runtime, wit-bindgen would have to use private reflection to manipulate it - an odd composition.

Perhaps there's a middle ground: WasiEventLoop lives in the runtime, but is private, and wit-bindgen uses unsafe accessors to make use of it. And generates a static Task Register(Pollable) function that application code can use, but that just defers to WasiEventLoop's version.

Let's refocus this http issue a bit. Say I am making bindings to APIs that return pollables from wasi:xyz, to be published on nuget.org. How would I do that?

There's a precedent for having wit-bindgen reuse an existing type instead of generating its own, so this could work one of two ways:

• If we decided the .NET runtime should include an "official", public Pollable, then wit-bindgen would use that instead of generating its own version
• If we decided not to include Pollable in the .NET runtime (and I tend to agree we shouldn't since it's going away in WASIp3), then we could have an "official" WASIp2 nuget package that your wasi:xyz package could depend on, and perhaps wit-bindgen would just always generate code that depends on that WASIp2 package, ensuring everyone uses the same type.

[SingleAccretion]

Perhaps there's a middle ground: WasiEventLoop lives in the runtime, but is private, and wit-bindgen uses unsafe accessors to make use of it.

That is what I was referring to - unsafe accessors are (fast) private reflection.

Unless we decide to consume WasiEventLoop in the runtime, e. g. by saying that we would support async Main 'out of the box', I don't see the need for it to live in the runtime - nothing would use it (except tests, but tests can reference external packages just fine).

This is again mostly driven by the compatibility guarantees - once a public API is there in the runtime, it is there forever. If WasiEventLoop was that "forever" model for WASI, I don't think we would hesitate to put it into the runtime, but we know that it is not.

If we decided not to include Pollable in the .NET runtime (and I tend to agree we shouldn't since it's going away in WASIp3), then we could have an "official" WASIp2 nuget package that your wasi:xyz package could depend on, and perhaps wit-bindgen would just always generate code that depends on that WASIp2 package, ensuring everyone uses the same type.

This is the sort of model I was thinking about. wasi:http should use the same solution as wasi:xyz.

Since we know WasiEventLoop won't be public runtime API, we also know that this will be sort of required. Even if the physical implementation ends up living somewhere in this repository, the assembly providing access to it won't be part of the shared framework.

[dicej]

This is the sort of model I was thinking about. wasi:http should use the same solution as wasi:xyz.

It certainly makes sense for a NuGet package that uses wasi:xyz to depend on a WASIp2 package, but I'm adding support for wasi:http to System.Net.Runtime, which can't have any NuGet dependencies. So at a minimum we'd need to enable a "system" mode for wit-bindgen to generate bindings suitable for use in this repo (i.e. do not require an external NuGet package). Those bindings would include a version of Pollable for internal use, and we'd need to make sure both that Pollable and the one in the WASIp2 NuGet package can be registered with WasiEventLoop (probably by converting both to the underlying uint handle, as @pavelsavara suggested).

Anyway, it sounds like the important point here is that we don't want to make any temporary APIs public in this repo, and we can be flexible about where each bit of code lives as long as we stick to that rule.

[pavelsavara]

Thanks!

[pavelsavara]

I'm adding support for wasi:http to System.Net.Runtime, which can't have any NuGet dependencies.

I realized that some C# programs don't need to use HTTP and so the whole thing could be trimmed.
In which case we should be able to drop the runtime's dependency on wasi:http and on WasiEventLoop

async Main 'out of the box',

We always do async main in the browser for yielding reasons. I guess the similar reasons will come to WASI with p3.

If there is any import which uses pollable, it also means that there must be async main or async export.
In order to be able to do top level await in C#.
Even if we are able to implement it in p2 via blocking poll()

So, maybe always async main would simplify the WASI design too ?

[dicej]

I realized that some C# programs don't need to use HTTP and so the whole thing could be trimmed. In which case we should be able to drop the runtime's dependency on wasi:http and on WasiEventLoop

Most of it could be trimmed, yes. But at minimum we'll still need a way for wit-bindgen-generated code to register a handler for ThreadPool work items so that application code can use async/await even if it's not using System.Net.Http.

Speaking of trimming: how can I tell the compiler not to trim an internal function that's never called from within its assembly (e.g. to make it available to call externally via UnsafeAccessor)?

[pavelsavara]

Speaking of trimming: how can I tell the compiler not to trim an internal function that's never called from within its assembly (e.g. to make it available to call externally via UnsafeAccessor)?

You can add DynamicDependencyAttribute onto something else that you know is not trimmed.
More in docs and runtime docs

You can also add xml config (but C# code is preferable). See

runtime/src/mono/sample/wasm/browser-advanced/ILLink.Descriptors.xml

Lines 1 to 8 in f6dc71b

	<linker>
	<!-- this is here to show how to configure the trimming -->
	<assembly fullname="System.Private.Uri">
	<type fullname="System.Uri">
	<method signature="System.Void .ctor(System.String)" />
	</type>
	</assembly>
	</linker>

runtime/src/mono/sample/wasm/browser-advanced/Wasm.Advanced.Sample.csproj

Line 27 in f6dc71b

<TrimmerRootDescriptor Include="$(MSBuildThisFileDirectory)ILLink.Descriptors.xml" />

Also this seems related #101195

[SingleAccretion]

Speaking of trimming: how can I tell the compiler not to trim an internal function that's never called from within its assembly (e.g. to make it available to call externally via UnsafeAccessor)?

Normally nothing special would be needed since trimming is performed on the whole final app, and it understands UnsafeAccessor.

[SingleAccretion]

If there is any import which uses pollable, it also means that there must be async main or async export.
In order to be able to do top level await in C#. Even if we are able to implement it in p2 via blocking poll()

Right.

So, maybe always async main would simplify the WASI design too ?

Very possible. It depends on what kind of app models WASI would be primarily targeting - library-like or application-like. We can't make library-likes work without code generation à la JSExport.

[dicej]

Very possible. It depends on what kind of app models WASI would be primarily targeting - library-like or application-like. We can't make library-likes work without code generation à la JSExport.

I would say WASI (and the Component Model on which it is based) is primarily targeting library-like things going forward. wasi:cli/run is one of many interfaces standardized as part of WASI, and it's the one which application-like things can export (while also exporting others, if desired). One can use wit-bindgen to generate bindings for that interface and thereby create an application-like component without any help from the runtime. And the new componentize-dotnet project should make that easy by bundling all the needed tools together into a single package.

So I would say there's no urgent need for built-in async main support in the runtime; componentize-dotnet will support targeting arbitrary WIT "worlds", wasi:cli being just one example. I guess it comes down to whether developers will want to target WASI without using componentize-dotnet or wit-bindgen (and would presumably be okay with only targeting wasi:cli and nothing else).

[dicej]

Here's what I have so far: dotnet/runtimelab#2614

Note that I'm targeting the feature/NativeAOT-LLVM branch of the runtimelab repo in that PR since (AFAIK) that's the only branch capable of generating WASIp2 components. Happy to retarget it if that changes.

I've run some high-level, manual integration tests using https://github.com/dicej/dotnet9-wasi-http-example/tree/snapshot, and things are looking good there. Now I'm going to look at adding and/or reusing tests in the runtime repo and add the result to that PR.

[dicej]

Question: what should Task.Wait do on WASIp2? Should it run a nested version of WasiEventLoop? Currently it just busy waits indefinitely, which isn't great.

If I'm reading the code correctly, it looks like it's unsupported on the browser when multi-threading is disabled, so maybe we do the same for WASI? https://github.com/dotnet/runtimelab/blob/53342bee4b89a6c066d6eb3c7549028ca479c451/src/libraries/System.Private.CoreLib/src/System/Threading/ManualResetEventSlim.cs#L394-L397

[pavelsavara]

If I'm reading the code correctly, it looks like it's unsupported on the browser when multi-threading is disabled, so maybe we do the same for WASI?

Yes, I think we need the same for single-threaded WASI.

[pavelsavara]

Normally nothing special would be needed since trimming is performed on the whole final app, and it understands UnsafeAccessor.

The first pass of trimming is done (without final app) on the individual runtime assemblies separately.
See #106009

There is already ton of descriptors dealing with it in shared and in mono