[NativeAOT-LLVM] Merge tests when targeting Wasm

[yowl]

With the nativeaot merge test wrapper, what is the intention, if any, for how it should be used to run non-host, e.g. Wasm, tests. For windows-x64 it looks like nativeaot.dll is published as a NAOT exe, but for Wasm it would appear that dotnet run nativeaot.dll would be desired, we need the writing of the results file on the host, not the target, and NAOT'ing this dll does not appear to contribute much.

Is this thinking correct, before I go off changing things?

[SingleAccretion]

@jkoritzinsky would you have any advice? The only 'custom' thing in the WASM setup is that we modify nativeaottest.cmd/sh to do ./wasm-host.exe App.wasm instead of just ./App.exe.

[jkoritzinsky]

My recommendation would be to merge some of the tests in-process instead of using the out-of-process model.

Alternatively, for now you can use the BuildAllTestsAsStandalone environment variable until I get around to the last bits of cleanup here.

[SingleAccretion]

My recommendation would be to merge some of the tests in-process instead of using the out-of-process model.

This would not be feasible. NAOT tests are sensitive to whole program analysis effects, they need to be compiled separately. Additionally, some tests use a JS host that can't be moved in-process even in principle.

Edit: or did you mean to have one "merged test" per each actual test?

@yowl I would not have objections to some can-down-the-road kicking on this problem...

[jkoritzinsky]

Some of the smoke tests can be merged. Others can't.

I'm also okay marking each test as a merged test, marked as containing tests in the assembly, and has the wrapper generator disabled. Should get close to the experience we want.

[yowl]

The first problem is the merged wrapper writes to files on the host, so it cant be compiled as wasm. Sent from Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Jeremy Koritzinsky ***@***.***> Sent: Saturday, September 28, 2024 3:38:32 PM To: dotnet/runtimelab ***@***.***> Cc: Scott Waye ***@***.***>; Mention ***@***.***> Subject: Re: [dotnet/runtimelab] [NativeAOT-LLVM] Merge tests when targeting Wasm (Issue #2695) Some of the smoke tests can be merged. Others can't. I'm also okay marking each test as a merged test, marked as containing tests in the assembly, and has the wrapper generator disabled. Should get close to the experience we want. — Reply to this email directly, view it on GitHub<#2695 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAUYCKPHWLXIONS2MJVW5XLZY4VWRAVCNFSM6AAAAABO76GGO2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBRGAYTGNRRGQ>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[jkoritzinsky]

When building for WASM, it uses the XHarness runner and writes to stdout instead.

If you want something that doesn't go through xharness for WASM, then you'll need to build something (as that's not a scenario I had to handle in main so I didn't). I'd recommend hooking into whatever the libraries tests do (if you have those running).

[yowl]

Not sure I follow, are we talking about https://github.com/dotnet/runtime/blob/ba9b3ba83603d1f54c00c45a87344568d4277966/src/tests/Common/XUnitWrapperGenerator/XUnitWrapperGenerator.cs#L434 ? It does file IO here https://github.com/dotnet/runtime/blob/ba9b3ba83603d1f54c00c45a87344568d4277966/src/tests/Common/XUnitWrapperGenerator/XUnitWrapperGenerator.cs#L471

[jkoritzinsky]

Yes that's what I was referring to. That file IO is just deleting temp files (on iOS and Android) and no-ops on Wasm today I think (as everything is guarded by File.Exists).

Either that or it's using some sort of synthetic filesystem (which is also fine).