Building for ARM causes error often about buildx HOT 13 CLOSED

Just an update from the future. Qemu 4.2.1 and Go 1.16.5 fairly instantly blows on docker buildx (arm64), but setting GOMAXPROCS=1 does fix the scheduling problems (apparently Qemu does some weird multi-cpu, not hardware conforming).

If a signal is raised, the process still crashes, and after 5-10 minutes of build time, the builder hangs.

from buildx.

@andrewrynhard tested with QEMU 4.0.0 and that seems to have solved his problems. QEMU 4.0.0 will be in the next release of Docker Desktop. I think we can close this ticket for now. Thanks to Andrew for the testing.

from buildx.

Are these using QEMU? If so what version/source? We have some go patches in https://github.com/moby/qemu for (that I believe are also the patches that ship with Docker desktop).

from buildx.

Also, you do not need to use --platform=$TARGETPLATFORM. That is the default.

from buildx.

I have a bit of a strange setup. I run all of my development out of docker containers with a compose file using Docker Desktop on a Mac:

version: '3.7'

services:

    docker:
      image: docker:19.03.0-beta5-dind
      container_name: docker
      network_mode: "host"
      volumes:
        - var-run:/var/run
        - ${WORKSPACE}:/root:delegated
        - /dev:/dev
        - tmp:/tmp
      restart: always
      privileged: true
      command: [ '--storage-driver', 'overlay2' , '--experimental' ]

    workspace:
      image: andrewrynhard/workspace:latest
      container_name: workspace
      working_dir: /root/src/github.com/talos-systems
      network_mode: "host"
      ports:
      - "3000:3000"
      volumes:
        - var-run:/var/run
        - ${WORKSPACE}:/root:delegated
        - /dev:/dev
        - tmp:/tmp
      restart: always
      stdin_open: true
      tty: true
      privileged: true
      depends_on:
        - docker

volumes:
  var-run:
  tmp:

I don't even see QEMU installed in DIND container :/

from buildx.

I don't even see QEMU installed in DIND container :/

QEMU is registered directly in the kernel, you don't need anything specific inside container.

Can you post the exact version of Docker Desktop.

@tuonga Can you verify that Desktop has the qemu go patches. Do these traces look similar to the problem the patch addresses?

from buildx.

Docker Desktop 2.0.4.1 (34207)

from buildx.

@tonistiigi the desktop should have the patches.
@andrewrynhard can you post the output of the following? In the mean time we'll take a deeper look.

docker run arm64v8/alpine sh -c "/proc/1/exe"

from buildx.

@tuonga

From my Mac:

Andrews-MBP:~ andrewrynhard$ docker run arm64v8/alpine sh -c "/proc/1/exe"
Unable to find image 'arm64v8/alpine:latest' locally
latest: Pulling from arm64v8/alpine
0362ad1dd800: Pull complete
Digest: sha256:9d2b3c9ba598ccd7c4aa49e5ad5b4cf47fb882adc3f5955d5e160b07f26a68ee
Status: Downloaded newer image for arm64v8/alpine:latest
qemu: no user program specified

From DIND setup:

[I] $ docker run arm64v8/alpine sh -c "/proc/1/exe"
Unable to find image 'arm64v8/alpine:latest' locally
latest: Pulling from arm64v8/alpine
0362ad1dd800: Pull complete
Digest: sha256:9d2b3c9ba598ccd7c4aa49e5ad5b4cf47fb882adc3f5955d5e160b07f26a68ee
Status: Downloaded newer image for arm64v8/alpine:latest
qemu: no user program specified

from buildx.

@tuonga
Andrews-MBP:~ andrewrynhard$ docker run arm64v8/alpine sh -c "/proc/1/exe"

I am really sorry, I meant

``docker run arm64v8/alpine sh -c "/proc/1/exe --version"`

from buildx.

[I] $ docker run arm64v8/alpine sh -c "/proc/1/exe --version"
qemu-aarch64 version 3.1.0 (v3.1.0-1-ga67c442e89-dirty)
Copyright (c) 2003-2018 Fabrice Bellard and the QEMU Project developers

from buildx.

what's the status of qemu 4 for buildx on linux?

currently running: Docker version 19.03.5-ce, build 633a0ea838

from buildx.

(same with qemu 4.2.0 and ubunu 20.04 on ARM64 hosts)

from buildx.