Building Mac Il2cpp on Mac OSX Docker

[shadowfox87]

Description
We can currently build for Linux il2cpp and Windows il2cpp using the 1 docker image for Unity on Linux. However, Mac il2cpp can only be built currently on Mac OS unfortunately. After some digging, I did find that there's docker image for Mac OSX: https://github.com/sickcodes/Docker-OSX

If we can automate the build for mac il2cpp, then we can build for all 3 platforms on gitlab. Can we possibly get a Unity image for mac os?

[webbertakken]

Yes, we'll need to develop one in this repository and integrate it with the versioning backend. It's part of the long term plans. Contributions are welcomed on this.

[hangarter]

Hi! I'd like to help :)
I've noticed github has github-hosted VMs for building pipelines (windows and macos). How could I start?
I had a look on the Dockerfiles and I saw it's used ubuntu as base, maybe it's possible to replace the base image and run on these github-hosted VMs? Apparently for public opensource projects it's free.

[webbertakken]

We'd need to develop a base image for both Windows and MacOS. These base images require the host OS to also be Windows or MacOS respectively (so they would run on different runners).

Generally we want to move towards a folder structure with each base-os as a folder in the root.

How could I start?

You can fork this repo and try to reproduce the same images (base, hub, editor) for Windows and MacOS. Initially they'd only need to build for targetPlatforms that are not currently available, so Windows/MacOS IL2CPP and UWP.

[hangarter]

Sweet, I've just forked your repo. It will be great if I manage to contribute :-D
https://github.com/hangarter/docker/tree/main/base-macos

I swapped the image by one mac compatible and it does build an image on my mac (without the dependencies yet), which is great (although the base image has 3+GB on this first test, ugh!)

So I was reading the Dockerfile, I reckon some dependencies I could sort out using homebrew, but I couldn't follow all the general dependencies:

FROM sickcodes/docker-osx:naked   --- First change, simple but I tested and this image builds

# Global dependencies -- I can search homebrew library on these, but as macOS is a little different, do we need all of this?
RUN apt-get -q update \
 && apt-get -q install -y --no-install-recommends apt-utils \
 && apt-get -q install -y --no-install-recommends --allow-downgrades \
 ca-certificates \
 libasound2 \
 libc6-dev \
 libcap2 \
 libgconf-2-4 \
 libglu1 \
 libgtk-3-0 \
 libncurses5 \
 libnss3 \
 libxtst6 \
 libxss1 \
 cpio \
 lsb-release \
 xvfb \
 xz-utils \
 && apt-get clean

# Toolbox
RUN apt-get -q update \
 && apt-get -q install -y --no-install-recommends --allow-downgrades \ -- apt-get is ubuntu, will use homebrew...
 atop \ -- brew install htop (maybe?)
 curl \ -- brew install curl
 git \ -- brew install git
 git-lfs \ -- brew install git-lfs
 wget \ -- brew install wget
 && git lfs install --system --skip-repo \ -- as it is
 && apt-get clean -- remove (no need)

# Disable default sound card, which removes ALSA warnings
ADD config/asound.conf /etc/

# Support forward compatibility for unity activation
RUN echo "576562626572264761624c65526f7578" > /etc/machine-id && mkdir -p /var/lib/dbus/ && ln -sf /etc/machine-id /var/lib/dbus/machine-id -- This would work on macOS (probably)

# Used by Unity editor in "modules.json" and must not end with a slash.
ENV UNITY_PATH="/opt/unity" -- This would work on macOS (probably, use EXPORT instead?)

# Fixes a Gradle crash while building for Android on Unity 2019 when there are accented characters in environment variables
ENV LANG C.UTF-8 -- This would work on macOS (probably, use EXPORT instead?)
ENV LC_ALL C.UTF-8 -- This would work on macOS (probably, use EXPORT instead?)

Not sure if I'm on the right path (70%?), but it seems like a start.

So would be glad to help with the base images if you have the time!

[webbertakken]

It's a start, but the hard part is not copying each logical step. It's evaluating each statement to see if it is indeed needed. We have added comments to most additions. Each package has been added after trial and error. Most have been added in pull requests that describe the reason for adding it.

After that we need the hub image to be able to run on top of it, and finally the editor image built from that.

As you mentioned; for MacOS images we should indeed be using the MacOS best practices, like using homebrew and such.

Feel free to open a PR with your proposal once you get hub working or even the editor.

[hangarter]

Hey I did some more research and the docker-osx project above mentioned is a macOS vm inside a Linux container, my approach above was wrong as everything was running in the Alpine linux and not on actual osX.
This means it would have to run docker on a linux docker host and the container would have macOS and run everything.
It seems we can pass in parameters on the docker command line and it would run everything inside the VM that's inside the container. It uses Linux KVM to host the VM with macOS that's inside the container.
Is that something that's still applicable to your project? I don't have a linux machine configured so that would take me some time to set up but I'm willing to have a shot at it.

[webbertakken]

Since GitHub offers MacOS runners, we can run MacOS containers on top of it. I don't see the need for extra complexity at this point, and there may be much complexity with such a deep stack. Recommend staying with MacOS container on top of MacOS runner

Note that GitHub provides the runners for free so you can experiment as much as you want.

[hangarter]

You're right about no need for extra complexity, it makes sense.

So I'm still researching and I don't seem to find any docker support for running macOS natively, maybe I'm not understanding docker features right? I was reading the github-runners docs and it looks like a macOS VM but I couldn't find anywhere an image (official) that runs a docker container based on macOS.

I checked here: https://github.com/docker-library/official-images
Then I went to docker hub to find an image: https://hub.docker.com/search?q=macos&type=image
And it seems the only supported OS's are Linux and Windows (inside the container). I mean, there's support for other architectures like ARM and others but no trace of macOS.

Not sure if it's a docker limitation or I'm not understanding it, but if macOS is not supported I'm glad to work on windows support to contribute to the project.

[GabLeRoux]

Well, as stated in issue description, there might not be an official one, but there is this:
https://github.com/sickcodes/Docker-OSX

[hangarter]

Yes that's the only option I found too!
It has to run on a Linux host system though :-(
I'll have a go using the Github runners and see how this works out!

[GabLeRoux]

I also found this serverfault thread:
https://serverfault.com/questions/607443/can-mac-os-x-be-run-inside-docker

Where someone shared this:
https://github.com/Cleafy/sxkdvm

But sickcodes/Docker-OSX seems easier to use. The only alternative I see as of today would be to not use docker and use github's mac runners directly. I'm not sure what is the best option, but there sure is some experimentation needed here and there.

[webbertakken]

Judging by this answer on StackOverflow it looks like it's not really a thing.

So maybe we'll just have to run typescript directly to issue the right commands to install (or pull) and run Unity editor.

Otherwise we'd go back to having multiple semantical steps like install, activate, and maybe more, which is viable but also a step back, as it becomes harder to configure for the end-user.

Edit: another idea: We could take the bottom out of a linux container, and mount whatever MacOS native is needed. I haven't done something like it before but it might turn out to be the most light weight option (with the least changes required), if it's at all possible..

Edit: here's an even better thread about kernels and docker (see the commends and newer answers too).

[webbertakken]

This discussion is relevant for a possible solution: containerd/containerd#5525.