Linux X86 Builds on Apple Silicon Are Impractical

In Jupiter Broadcasting’s Matrix General Chat room we all got into a bit of a conversation spiral about running Intel x86/x64 VMs and code under emulation. Will the hypothetical new Rosetta for Linux make it possible to use Intel x86/x64 Linux on their platforms at reasonable speeds, or at least x64 only apps as needed within ARM64-based VMs? The conversation ended with the point that for most of my needs all I really need is to be able to build x64 Linux apps on my machine. Is that possible even with the performance hit of running emulation? Yes it is very possible. Is it practical though? Sadly, no it is not.

Benchmark

To test the question about whether this is possible or not I started with doing some performance benchmarks just to get a rough scale of what the performance hit would be. I ran these all on my 2020 M1 MacBook Pro which runs at 3.2 GHz. I found a convenient UI front end to QEMU called UTM that would easily allow me to run an x64 flavor of Linux. I had some problems with figuring out the right QEMU settings that would work so instead I just downloaded a pre-built Ubuntu 14.04 VM image and then installed Ubuntu Server 20.04 over top of it. I also tweaked the memory to give it 4 GB of RAM instead of the 512MB it came with. I did leave it with the 1 CPU though. For the ARM64 Virtual Machine benchmarks I already have a licensed copy of Parallels Desktop that I use on a daily basis. To make it a one-for-one comparison I gave it the exact same system settings in terms of memory, number of CPUs, etc. With all of that configured I downloaded the appropriate copies of GeekBench fo the host OS (Apple Silicon version), and the respective Linux flavors. The results were not encouraging:

As we can see the hit for the ARM64 is noticeable but pretty minor, coming it at 97% of the bare metal system performance. The QEMU x64 VM on the other hand is performing extremely poorly with 6.7% of the bare metal performance. When I started the discussion I had recalled that performance was a problem which is why I didn’t do it. I never quantified how bad the performance was because the UI was sufficiently non-responsive enough. Having only 6-7% of the performance of the host system would be a huge factor in that. But again, for a build system that had no UI maybe it wouldn’t be that bad, so the original discussion in the forum went. If these benchmark numbers held up then that means it would take 14-15x longer to do compilations on the x64 VM compared to the ARM systems. Let’s give it a try just to see how bad it actually would be…

Real World Build Test Using Flutter

Benchmarks can be deceiving. I didn’t think it would be an order of magnitude off, so the question of if it was practical to use x64 VMs as build machines on Apple Silicon was probably already answered. But since I went through all the trouble of building these machines I figured I might as well try building a real application. My original intention was to build Kyanite , my own app, but there were platform dependent libraries for some of the graphics stuff that isn’t supported on ARM platforms right now (a topic for another post). I therefore decided to instead use the standard Flutter Gallery demo site/app that they show off major features with, Flutter Gallery GitHub source code here . The below steps are very truncated versions of the standard install steps for Flutter from their standard documentation for Linux .

First install the the system dependencies needed to get your Flutter Desktop build toolchain working:

sudo apt install unzip clang cmake ninja-build pkg-config libgtk-3-dev liblzma-dev

Next checkout the latest Flutter version from the repository and initialize it:

cd /tmp
git clone https://github.com/flutter/flutter.git -b master
export PATH=/tmp/flutter/bin:$PATH
flutter

Now check out the Flutter Gallery source code, configure the Linux Desktop build target, have it create the Desktop build files, and then build a release binary of the project:

git clone https://github.com/flutter/gallery.git
cd gallery
flutter config --enable-linux-desktop
flutter create .
flutter build linux --release

With this you have a binary and all of the dependencies have been pulled down. To make sure that network hiccups weren’t a factor in any timings do a clean rebuild and time the final command:

flutter clean
time flutter build linux --release

And with that we have the final build times for these tests. The ARM64 build would complete in about one minute. The x64 build on the other hand would take on the order of 15 minutes. Sadly that means that that 14-15 times performance difference translated over directly to the build process as well. It’s not unexepected but still.

Conclusion

So in conclusion, can one use an x64 VM for doing their Intel builds on Apple Silicon? Obviously I got a build going so it is technically possible. However with it taking 15 minutes to do a build, not counting how much longer it took to pull down dependencies etc, it is actually faster to leverage cloud infrastructure to spin up a VM from scratch, run configuration scripts, and do the build up there than it would be to do this. That would still be excruciatingly slow compared to having a dedicated build machine, or a warmed up one created on demand each day. Perhaps the new Rosetta for Linux stuff will change that calculus. For now though if you really need x64 Linux builds this isn’t practical to do on Apple Silicon.

Aside

One question I have is why the QEMU performance is so much worse. This was a problem going the other way, from Intel to ARM, in the early days of Android development. Back when I was running VirtualPC on PowerPC we were able to get Windows and x64 Linux running at reasonable but noticeably slower speeds. I think it was like a factor of 2-3 slower. Why can’t we achieve these usable levels of emulation of Intel on ARM and vice versa? Is it the processor complexity just doesn’t allow for it or there isn’t a concerted effort to hyper-optimize these things since there isn’t a big enough of a market for it? I’m thinking it has to be the former otherwise Google would have fixed this for their Android development ecosystem.