Publish AOT for ARM64 on a Raspberry Pi – practical advice?

[ygoe]

Hello,

I'm writing an application that is intended to run on a Raspberry Pi primarily. To save more resources, I want to use native AOT compilation for it. The app code supports that already. My main development environment is Visual Studio 2022 on Windows 11 (x64).

Now there are a few issues here. While it's trivial to install the .NET 8.0 Runtime on any modern Raspberry Pi one way or the other, or publish a self-contained app that includes the runtime for the ARM64 architecture, publishing AOT for a foreign OS and architecture is hard.

The first hurdle to overcome is Windows/Linux. As the documentation explains, I cannot compile natively on Windows for Linux. That means that my compilation efforts must be on a Linux system.

The obvious choice is to install the full .NET SDK on the Raspberry Pi, copy the source code and do the compiling there. Well, my poor old RPi 3B+ (with 512 MB RAM) died twice. At first, it choked on the multi-threaded compilation after a few minutes. With the -m:1 option (disabling parallel builds), it did better but eventually ran out of memory a bit later and froze. I'm going to get a RPi 4 with 2 GB RAM to try again. I'd like to avoid getting more expensive hardware with more memory or even a massively overgrown RPi 5.

My next try was to use WSL (this time on a Windows 10 machine). So I installed the default "Ubuntu" image (which turns out to be 22.04). It doesn't know the "clang" package, so I went on and retried with the "Ubuntu-24.04" image. Same issue. But "gcc" worked instead (still for linux-x64). So I continued on the cross-architecture path and tried to install the aarch64 compilers, but those were not discovered by dotnet publish. Eventually I gave up on it as well.

I haven't tried using a full Ubuntu OS in a proper VM yet. But my build environment would already get a bit clunky there.

So, how do you folks do this? Develop on Windows and AOT-compile for a Raspberry Pi (with standard 64-bit Lite OS). Has this ever been done in reality? Which tools would I use and which Raspi model would work, if it was to do the compilation?

Maybe a few numbers here: The hello-world app for native Linux/x64 is around 1.5 MB with .NET 8. My application is currently around 15 MB for native Windows/x64. It's going to grow a little more, but many libraries are already in there. I wouldn't consider this big, but it's also not trivially tiny anymore.

[MichalStrehovsky]

The problem with crosscompilation is basically the need to provide the tools that can target the architecture (e.g. clang, objcopy, ld), and libraries from the target OS to link to (e.g. libc, etc.).

There are some instructions for crossbuilding here - they assume a Ubuntu Linux x64 host and a Linux arm64 target: https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/cross-compile

It's not the only possible route, one could also use something like Zig (a crosscompiler for C) together with some glue like https://www.nuget.org/packages/PublishAotCross#readme-body-tab.

[ygoe]

Quick update: I now used full-blown VMs to run Ubuntu and try it out. In short: it worked (for an empty console app).

First I used Ubuntu 22.04 and the provided instructions. Everything went well. Now I retried with Ubuntu 24.04 to have a newer OS version. The instructions needed to be adapted a little and in the end it also worked. Interestingly, the size of a dotnet new console program went up from 1 594 520 to 1 651 864 bytes with the new Ubuntu. (.NET SDK 8.0.303 installed with script, not package)

Here are the full instructions for Ubuntu 24.04:

# Install dotnet
wget -nv https://dot.net/v1/dotnet-install.sh -O dotnet-install.sh
chmod +x ./dotnet-install.sh
./dotnet-install.sh
export PATH="$PATH:$HOME/.dotnet"

# Install cross-compile tools
sudo dpkg --add-architecture arm64
sudo bash -c 'cat > /etc/apt/sources.list.d/arm64.list <<EOF
deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ noble main restricted
deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ noble-updates main restricted
deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports/ noble-backports main restricted universe multiverse
EOF'
sudo apt -qq update
sudo apt -qqy install clang llvm binutils-aarch64-linux-gnu gcc-aarch64-linux-gnu zlib1g-dev:arm64

# Create and build project (example)
mkdir ~/src
cd ~/src
dotnet new console
dotnet publish -c Release -p:PublishAot=true -r linux-arm64 --self-contained

# Copy program to Raspberry Pi (example)
scp bin/Release/net8.0/linux-arm64/publish/src pi@raspberrypi:/home/pi/src

Now can I also have this in WSL somehow? That would be much simpler to access the source code on my local system.

[ygoe]

Looks like it already works in WSL on Windows 11 (image "Ubuntu", 22.04). This seems to be a different image than on Windows 10.