Thanks so much! I will try this way with UTM.
The PHP is so bad on docker at M1, let's check if can be better!
And please let me know if you have some big changes about this performance tools with M1 while this time, because probably something changed 1year to now.

Thanks for the note.

I have not completed a thorough test in a few months, therefore cannot objectively confirm any improvements.

However, the Docker Desktop for Mac v4.6.0 release, which included virtiofs file sharing, did make a big difference.

I know others have had good results with alternative container orchestration, such as Podman, etc. Therefore, that might be worth exploring.

However, I suspect the UTM approach would still be the most performant, as it gets closest to native performance running on Linux.

Good luck! I'll be sure to post an update if I do more testing.

Good comparative, but I think the conclusion is use easy, Linux for everything :)

Ha! I couldn't agree more!!! Bring on the year of the Linux desktop ;-)

https://www.lifeintech.com/...

Did you happen to try an amd64 UTM VM? I'm currently switching to a 2021 MBP from an Asus Zephyrus G14 running Linux. Getting my local development cluster (kubernetes via docker with k3d) running is my last hurdle.

Several of the images we rely on do not build arm64 versions, so I am stuck either with an amd64 VM or rebuilding all those images myself. I'm about to try an amd64 VM but am curious as to the performance delta, as I don't have a working version in arm64 to benchmark against. Thanks!

Thanks for the note.

I haven't tested virtualising an x86-64 OS using UTM. However, I assume the performance would be significantly impacted.

For example, building an x86 Docker image resulted in a build time of 1210 seconds, compared to just 9 seconds on native ARM64.

Although it is painful, I suspect your best bet would be to rebuild your images using ARM64. Depending on your stack, you might be able to find a starting point on Docker Hub / GitHub.

Luckily, I have found someone who has dropped some equivalents for the Bitnami images with arm64 support (Bitnami only provides amd64 - https://github.com/ZCube/bi.... I have also found a Postgres image with the extensions we need (https://hub.docker.com/r/su..., so I *should* be good to go with a few more tweaks to our helm charts.

Thanks for the response!

Good article!
Would be nice to see tests from bare metal Windows(WSL2) on MACs! ;)

Thank you for this article.

Can you tell us more how do you use Combination of coding on Mac OS via VS Code and Ubuntu server on UTM?
Are you provisioning code via shared folder to Ubuntu server and building/running docker containers on Ubuntu?
If not please explain more, we are considering how to increase speed in building and running docker containers, right now its at least 15 minutes, and if we are building images it can take up to 30 minutes.

Its combination of lots of things, golang on backend, react on frontend, migrations are alembic, database is postgres, and many more services.

Thanks for the note.

My setup is fairly basic.

Each morning I launch a Ubuntu Server (ARM64) running on UTM, which includes Docker. I then have a shared folder, SSH and git configured to enable simple synchronisation.

I develop in VS Code on macOS and when needed, trigger the build on Ubuntu either directly from the command line or via git.

Even with the latest version of Docker Desktop for Mac, I find this approach to be an order of magnitude quicker.

I hope that helps.

When moving to using docker on Apple M1 / arm64, does this mean all docker images that were previously built on an Intel / amd64 system must now be rebuilt to run on M1?

Does this also mean if an image that is built from other legacy images (i.e. older versions of images from docker hub, other authors, etc.) those lower level images must also be rebuilt to use/run them on M1?

And once built on M1, will those images run on amd64?

Thanks for the note.

Docker supports multi-architecture images (x86 and ARM64). However, you will see a significant performance impact if you do not select a native image for your architecture (e.g. Apple M1 = ARM64).

Therefore, ideally, you should look for images on Docker Hub that are native ARM64.

Hi - great write up, very timely and nearly exactly what I was looking for in terms of analyzing a similar usage of hardware. I currently have a 2019 MBP w/64GB as my primary and "everything" computer (workstation/development environment, all regular MacOS computer needs), I run Docker Desktop for application development and I also use Parallels for Windows application development from within the MacOS environment. I have ordered its replacement, a 2021 MBP M1 Max w/64GB (expected delivery in December 2021) and have been a bit nervous things might not work at least as they do on the 2019 MBP.

This article is the first I've heard of UTM but if I understand your setup for your best test (green in your chart) is using UTM as a replacement for the MacOS host operating system (like we do with an older MacMini with ESXi v6).. is that correct?

If so, does that mean you also (can?) have MacOS installed as a VM for your MacOS usage of the device in addition to the Ubuntu Server VM? I would need to be able to continue to use the same hardware as my everything where MacOS is the main usage environment (even if it isn't the lowest layer).

Are you able to just use that MacOS VM on top of the UTM host just as if it were the host OS for everything? And when in the MacOS VM, I assume you can just ssh in a terminal into the Ubuntu Server VM and have a shared file system between the two VMs so code editing and file navigation all can just appear as if it is still happening on the MacOS environment even though it is running on the Ubuntu Docker Desktop?

I am very interested to learn more about this to maximize the performance for docker but not have to give up my MacOS primary work environment.

Thanks!

Thanks for the note and congratulations on your new Mac! The M1 Max 64GB will be a beast!!! I have the 32GB variant.

Regarding my setup, UTM is virtualisation/emulation software that runs on top of macOS (like VMware Fusion or Parallels Desktop).

Therefore, I have macOS 12.0.1 as my primary operating system, with Ubuntu Server running as a virtual machine on UTM.

The difference with UTM is that it uses the Apple Hypervisor framework introduced in macOS 11. That delivered almost native CPU performance.

With Ubuntu Server running in the virtual machine, I can achieve phenomenal Docker performance (better than my 16C/32T desktop PC).

I have documented my setup of UTM in the following article: https://www.lifeintech.com/...

If you decide to run Docker natively on macOS, you should achieve (at minimum) equal performance compared to an equivalent Intel-based Mac. However, ensure you are using the correct image (ARM64 vs. x86).

Hope that helps. Enjoy!

Ah - ok - I get it, so UTM being just like Fusion or Parallels or VirtualBox - for some reason I thought by your table it was the bare metal OS replacement and my brain was recently in the middle of working on ESXi on our MacMini as that scenario.

So generally speaking the idea is to switch to a Ubuntu Server VM and run the docker engine and docker cli inside the VM... Are you still using a macOS coding environment? I also recently stumbled on VSCode which I've started to fall in love with and I've seen some articles about connecting it to a "remote" environment to code so I assume that means I can code in VSCode on the macOS side while the runtime environment is in the UbuntuVM.

I wonder if I did the same with Parallels vs UTM, how would it compare between the 2 and/or would it still be a boost in performance vs natively on macOS with Docker Desktop? Maybe a comparison with Parallels Ubuntu VM is another addition to your chart in this article :-)

I continue to pull my hair out with file system performance and mounting issues on the macOS native side, even with my beast of a machine! - I keep thinking they will fix it in the next release... but it's been years now.

I will setup a Ubuntu Server environment to see how the docker development environment on a VM while working from my macOS works on my current Intel based MBP while I patiently wait for my M1 Max.

Correct, UTM is comparable to Fusion and Parallels (although it is free and open source).

I have not used Parallels on M1, but assuming it is using the Apple Hypervisor, the performance should be comparable to UTM.

Regarding the developer experience, I work in macOS (VSCode), with the Ubuntu Server available via SSH or Git. The only thing the Ubuntu Server is used for is to build the code, therefore it doesn't even need a GUI.

Hope that helps! It should eliminate all file system bottlenecks with Docker, as you will be connected natively.

Good luck!