EOF Error on podman machine start (Mac M1)

[cevich]

Issue Description

While testing rollout of new podman-machine CI for the Mac M1, a rare (<1%) test failure was observed. 99% of other experimental-ci runs (of basic tests) executed at/around the same time passed.

Steps to reproduce the issue

Steps to reproduce the issue

1. Unknown 😞

Describe the results you received

Cirrus Log

------------------------------
run basic podman commands
/Users/MacM1-6-worker/ci/task-5689431747723264/pkg/machine/e2e/basic_test.go:15
  Basic ops
  /Users/MacM1-6-worker/ci/task-5689431747723264/pkg/machine/e2e/basic_test.go:28
  > Enter [BeforeEach] run basic podman commands - /Users/MacM1-6-worker/ci/task-5689431747723264/pkg/machine/e2e/basic_test.go:21 @ 11/27/23 17:52:24.248
  < Exit [BeforeEach] run basic podman commands - /Users/MacM1-6-worker/ci/task-5689431747723264/pkg/machine/e2e/basic_test.go:21 @ 11/27/23 17:55:07.007 (2m42.76s)
  > Enter [It] Basic ops - /Users/MacM1-6-worker/ci/task-5689431747723264/pkg/machine/e2e/basic_test.go:28 @ 11/27/23 17:55:07.007
  /Users/MacM1-6-worker/ci/task-5689431747723264/bin/darwin/podman machine init --image-path /private/tmp/ci/podman_test2852998336/fedora-coreos-38.20230925.dev.0-applehv.aarch64.raw.gz --now c475cf10e0e5
  Machine init complete
  Starting machine "c475cf10e0e5"
  Error: EOF
  [FAILED] Expected
      <int>: 125
  to match exit code:
      <int>: 0
  In [It] at: /Users/MacM1-6-worker/ci/task-5689431747723264/pkg/machine/e2e/basic_test.go:35 @ 11/27/23 17:59:45.116
  Full Stack Trace
    github.com/containers/podman/v4/pkg/machine/e2e_test.glob..func1.3()
    	/Users/MacM1-6-worker/ci/task-5689431747723264/pkg/machine/e2e/basic_test.go:35 +0x1c0
  < Exit [It] Basic ops - /Users/MacM1-6-worker/ci/task-5689431747723264/pkg/machine/e2e/basic_test.go:28 @ 11/27/23 17:59:45.116 (4m38.109s)
  > Enter [AfterEach] run basic podman commands - /Users/MacM1-6-worker/ci/task-5689431747723264/pkg/machine/e2e/basic_test.go:24 @ 11/27/23 17:59:45.116
  /Users/MacM1-6-worker/ci/task-5689431747723264/bin/darwin/podman machine rm --force c475cf10e0e5
  < Exit [AfterEach] run basic podman commands - /Users/MacM1-6-worker/ci/task-5689431747723264/pkg/machine/e2e/basic_test.go:24 @ 11/27/23 17:59:45.372 (256ms)
• [FAILED] [441.125 seconds]
run basic podman commands
/Users/MacM1-6-worker/ci/task-5689431747723264/pkg/machine/e2e/basic_test.go:15
  [It] Basic ops
  /Users/MacM1-6-worker/ci/task-5689431747723264/pkg/machine/e2e/basic_test.go:28
------------------------------

Subsequent test (there's only one) in this run passed.

Describe the results you expected

All basic tests should always pass.

podman info output

N/A

Podman-remote & tests built from d1236f46fc5cb29e8da296c3662f8ce869865030

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Additional environment details

Mac OS 14
MACHINE_TEST_TIMEOUT=500
vfkit version: 0.5.0
Image: https://fedorapeople.org/groups/podman/testing/applehv/arm64/fedora-coreos-38.20230925.dev.0-applehv.aarch64.raw.gz

Additional information

Chatted with @ashley-cui about this, she said "My hunch is a garbled machine config file due to a race somewhere". So this could be a high-value/high-appreciation fix, but with potentially incredibly difficult debugging required.

[Luap99]

I am going to assume this works on main