This past weekend, I published a Docker image for Project64 (a Nintendo 64 emulator for Windows). That’s here. If you just want to run Project64 in Docker: check out the readme, docker pull jbergknoff/project64:v2.3.2, and go nuts.
In short, because Wine and Project64 are unstable together. Docker gives me a way to explicitly enumerate what they depend on to work, and a way to easily reset to a clean slate when they break.
I wanted to show Banjo-Tooie to the kids after this video came out. I had played it a bit about a year ago, using Project64 on Wine (maybe with PlayOnLinux, I don’t remember), but eventually some configuration got corrupted, and Project64 would error out, complaining about some Direct3D DLL file, and refusing to run anything. I tried changing configuration, wiping out the Wine prefix (the directory tree where Wine operates, controlled by environment variable WINEPREFIX) and reinstalling, etc. Nothing worked. I gave up on that. I tried running in mupen64plus. The game ran, but constantly froze. Oh well. At that point I left it alone.
Last week I came back to this. I had since formatted the hard drive and reinstalled the operating system (unrelated), so I tried again with PlayOnLinux, and this time it worked… sort of. I booted the game. Then I tried to switch to fullscreen. Big mistake. Project64 crashed and would no longer start, putting up a message box with an error about “line 88” and then quitting. No logs to be found, nothing useful on the internet. I found the Project64 source, and line 88 was the catch clause of a big try/catch, and I didn’t see any way to recover a stack trace.
Okay, so go back to a blank config file, or reinstall, or nuke the whole Wine prefix and start over. Nope, nothing worked.
Wine and Project64 are both terrific achivements of software, but they are incredibly brittle, at least in combination (I’m not familiar enough to attribute that to one or the other). Wine is very sensitive to the global configuration of the system, e.g. the set of drivers and OS packages installed. Anything running in Wine seems like a poster child for distribution in a Docker image in order for people to have a consistent experience (I’ve written about Docker as a distribution mechanism).
Honorable mention to PlayOnLinux, which is supposed to help with that reproducible experience. It was totally inconsistent for me. It worked once or twice at the beginning (I was deleting and recreating the Wine prefix in attempts to get things working). But after a while, and up until I gave up and uninstalled it, it would just hang creating the Wine prefix, with no clear way to debug or fix it. I think that was the same issue discussed here.
Seems easy enough, just use Wine to run the Project64 installer in a Dockerfile, and then run wine Project64.exe at run time. It turns out there are a few difficulties.
Wine’s installation instructions are not great. First we read a forum post to find out what FAudio is and how to install it, then we add the PPA, okay, then try to install winehq-stable=4.0.4~focal, and it errors out with
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
winehq-stable : Depends: wine-stable (= 4.0.4~focal)
E: Unable to correct problems, you have held broken packages.
Cool. We don’t get the error if we install 5.0.1~focal. I was trying 4.0.4 because PlayOnLinux configured Project64 with Wine 4, but I ended up building the image with Wine 5 and it worked fine. Anyway, I ended up using somebody else’s image as a base, and sidestepping the difficulty.
You’d normally install Project64 by running its installer, a GUI application that you have to click through. That’s not going to fly in a Dockerfile. One easy way out: the stable build is only published as an executable installer, but the development builds also have zip files available.
But it is possible to run the installer headless. I looked at the Project64 source for the installer. Google told me that the .iss file is configuration for a tool called Inno Setup. Inno Setup graciously offers some command line options that make it possible to run the installer without interaction.
So this almost works:
wine setup.exe /SP- /SILENT /SUPPRESSMSGBOXES
except that it still wants to create a window, and that fails. If we wrap it with xvfb-run, though, it does work.
When you RUN wineboot as part of a Dockerfile, it exits 0 but the Wine prefix that it creates is not complete. This was the source of a lot of confusion for me, and it took me a long time to understand what was happening. I built an image that superficially worked. Project64 would start, but when I tried to run a game, the game wouldn’t run and an error would be printed to stdout:
0016:err:ole:CoGetClassObject class {bcde0395-e52f-467c-8e3d-c4579291692e} not registered
0016:err:ole:CoGetClassObject no class object {bcde0395-e52f-467c-8e3d-c4579291692e} could be created for context 0x1
0016:err:dsound:get_mmdevenum CoCreateInstance failed: 80040154
If you search that UUID on the web, there is a lot of discussion, but nothing that seemed relevant to what I was seeing. When I would run winecfg, it would tell me there was no audio driver selected. However, pacat /dev/urandom in the container would play sound, so clearly a misconfiguration of Wine while PulseAudio itself was working.
When I ran the steps of the Dockerfile manually in one long-running container, I didn’t run into the same problem. I looked at the differences in the file systems, and saw that $WINEPREFIX/system.reg was big (~2 MB) in the working container and tiny (~6 KB) in the broken container. Those observations led me to this GitHub issue which pointed out that wineboot kicks off a bunch of stuff in the background but doesn’t wait for it to finish. I added a && sleep 10 and, voila, the Dockerfile-built image worked.
There’s a lot to juggle to get a complicated thing like a video game running in Docker. As I mentioned in my other article, I actually consider this a benefit. Figuring out the correct set of docker run arguments can be tedious, but it makes explicit some of the things that would normally be taken for granted (read: wouldn’t work on anybody else’s machine). In this case, we need
X11 forwarding. This is a well-known technique used any time you need to run a GUI in Docker. We forward the X socket and use the host’s $DISPLAY environment variable.
-e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix
NVIDIA drivers. On a computer with a NVIDIA graphics card, and NVIDIA drivers installed on the host, I was running into issues with OpenGL:
libGL error: failed to load driver: swrast
I tried various things, including running with nvidia-docker, which turned out to not be necessary. The solution was to volume in the Nvidia drivers from the host, and setting LD_LIBRARY_PATH to look there.
-v /usr/lib/i386-linux-gnu:/nvidia-drivers -e LD_LIBRARY_PATH=/usr/lib/i386-linux-gnu:/nvidia-drivers
I also had to add --privileged. Without it, starting a game would fail with an error executing X_GLXCreateContext. I tried substituting --device /dev/dri, --device /dev/video0, --device /dev/nvidia0 but those didn’t make it work. I’m sure it’s not necessary to use --privileged, but I wasn’t willing to invest more time on this.
If we just set LD_LIBRARY_PATH=/nvidia-drivers, then some of the shared libraries volumed in from the host can shadow the shared libraries in the container. I tried that first, but it caused some error (can’t remember now), so I explicitly tell it to first search in the default place.
PulseAudio. To get sound, we need to volume the PulseAudio socket into the container and tell PulseAudio where it is, which looks like:
-e PULSE_SERVER=unix:/run/user/1000/pulse/native -v $XDG_RUNTIME_DIR/pulse/native:/run/user/1000/pulse/native
The way I have it setup, the PulseAudio socket is owned by my user, and any other user (including root) is unable to use it. As a result, I need the container to run as user 1000. This is brittle, but it’s another aspect I didn’t want to spend more time on (maybe adding the user to the audio group would be enough).
Game Controller.
--device /dev/input/js0
This shouldn’t be strictly necessary if we’re passing --privileged, but let’s pretend we’re not passing --privileged.
This works pretty well. There are two outstanding issues that I know about:
The image is enormous (3.49 GB according to docker images). Generating the Wine prefix as part of the image adds a gigabyte to it, in its drive_c/windows directory. I assume that stuff is all copied from somewhere else on disk (looks like maybe /opt/wine-stable/lib?), and I wonder if symlinks would help here to not waste so much space.
If I had other things to run in Wine, though, I could share the base layer with the Wine prefix, and have small images on top of it.
Banjo-Tooie runs but complains about “no controller”! Other games use the controller just fine. I didn’t have the problem back when I was running outside of Docker. So maybe it’s something wrong with the Docker setup, but then why would other games not have the issue? I see the same issue in Project64 v2.3.2 (latest stable) and the latest development build. I haven’t solved this one yet.
Edit: This went away when I reverted to config defaults (i.e. not using my volume for the Project64/Config directory). I’ll also keep the config directory tracked in git from now on so I can checkpoint when things work and quickly revert when they break.