Podman manifest create for image list/index produces docker v2s2 format only

[mkarwin]

Issue Description

Currently when podman is used to build a container image, you can specify image format as 'oci' (v1) or 'docker' (v2s2). But this applies to images only, not the image list/index manifest creation.
You can easily build an image using docker format generating parameter (default) podman build --format docker --manifest test-docker . as this produces:

"mediaType": "application/vnd.docker.distribution.manifest.v2+json"

while running oci format build podman build --format oci --manifest test-oci . yields:

"mediaType": "application/vnd.oci.image.manifest.v1+json"

so this works great for specific images.
However, once you add multi-platform image builds or multi-image list/index manifest creation into the picture the situation is different, as you always end up with:

"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.list.v2+json"

regardless if you run podman build --platform linux/amd64,linux/arm64 --format oci --manifest test-oci . or podman build --platform linux/amd64,linux/arm64 --format docker --manifest test-docker ., which may be expected as podman build --format call is specific to the images, not the list/index manifest creation.
However, there's no option to actually "init" the multi-image manifest as either docker-format list or oci-format index as manifest creation command yields docker v2s2-only artifacts.
As such, you end up with either all-docker compliant format (default, when explicitly requesting docker format) or a mixed-bag when requesting OCI-format even if you request/want to get OCI-specific outputs.

Steps to reproduce the issue

1. Let's take a small and easy Dockerfile of:

FROM alpine:latest
RUN date > /build_date
CMD cat /build_date && arch && uname -a

2. Build multi-image manifest targeting multi-platforms with podman build --platform linux/amd64,linux/arm64 --format oci --manifest test-oci . expecting OCI format for both images and image list/index.
3. Furthermore, create/init manifest with podman manifest create test-2
4. Try to "use" OCI format using the format switch/param (expected to be present and be working similar to same on podman build command) to "enforce" OCI-format:

podman manifest create --format oci test-3

Describe the results you received

If you check the mediaType of the manifests you'll see they are docker-type only:

1. Check the mediaType of the produced manifest:

podman manifest inspect test-oci
{
    "schemaVersion": 2,
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "manifests": [
        {
            "mediaType": "application/vnd.oci.image.manifest.v1+json",
            "size": (...),
            "digest": "sha256:(...)",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            }
        },
        {
            "mediaType": "application/vnd.oci.image.manifest.v1+json",
            "size": (...),
            "digest": "sha256:(...)",
            "platform": {
                "architecture": "arm64",
                "os": "linux",
                "variant": "v8"
            }
        }
    ]
}

2. Check mediaType of freshly created manifest:

podman manifest inspect test-2
{
    "schemaVersion": 2,
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "manifests": null
}

3. Get an unknown flag/error when trying to "use" OCI format using the format switch/param (expected to be present and be working similar to same on podman build command) to "enforce" OCI-format since that is not yet supported:

podman manifest create --format oci test-3
Error: unknown flag: --format
See 'podman manifest create --help'

Describe the results you expected

When creating new manifest list/index with OCI requested format in the image build use OCI-specific index with mediaType of application/vnd.oci.image.index.v1+json (OCI vs docker multi-image manifests) instead of application/vnd.docker.distribution.manifest.list.v2+json.
Do not change mediaType/format of the preexisting index/list when adding images to some existing one (this I believe is the current approach when using podman manifest append).
When creating new manifest from scratch using podman manifest create support generation of OCI index type through an exposed optional --format parameter, same as for image build command.

podman info output

host:
  arch: amd64
  buildahVersion: 1.31.3
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: conmon-2.1.8-1.el9.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.8, commit: 879ca989e09d731947cd8d9cbb41038549bf669d'
  cpuUtilization:
    idlePercent: 99.68
    systemPercent: 0.16
    userPercent: 0.16
  cpus: 8
  databaseBackend: boltdb
  distribution:
    distribution: '"almalinux"'
    version: "9.0"
  eventLogger: file
  freeLocks: 2048
  hostname: (...)
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.15.133.1-microsoft-standard-WSL2
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 974614528
  memTotal: 8151609344
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.7.0-1.el9.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.7.0
    package: netavark-1.7.0-1.el9.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.7.0
  ociRuntime:
    name: crun
    package: crun-1.8.7-1.el9.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.8.7
      commit: 53a9996ce82d1ee818349bdcc64797a1fa0433c4
      rundir: /mnt/wslg/runtime-dir/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  pasta:
    executable: ""
    package: ""
    version: ""
  remoteSocket:
    path: /mnt/wslg/runtime-dir/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.1-1.el9.x86_64
    version: |-
      slirp4netns version 1.2.1
      commit: 09e31e92fa3d2a1d3ca261adaeb012c8d75a8194
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.2
  swapFree: 2147483648
  swapTotal: 2147483648
  uptime: 191h 5m 21.00s (Approximately 7.96 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.access.redhat.com
  - registry.redhat.io
  - docker.io
store:
  configFile: /home/(...)/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/(...)/.local/share/containers/storage
  graphRootAllocated: 1081101176832
  graphRootUsed: 2142650368
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 31
  runRoot: /mnt/wslg/runtime-dir/containers
  transientStore: false
  volumePath: /home/(...)/.local/share/containers/storage/volumes
version:
  APIVersion: 4.6.1
  Built: 1695842412
  BuiltTime: Wed Sep 27 21:20:12 2023
  GitCommit: ""
  GoVersion: go1.19.10
  Os: linux
  OsArch: linux/amd64
  Version: 4.6.1

Podman in a container

No

Privileged Or Rootless

None

Upstream Latest Release

Yes

Additional environment details

Using WSL2 with Alma Linux 9 distribution.

Additional information

Issue happens for all requested OCI-format multi-platform builds.

[rhatdan]

@nalind PTAL

[nalind]

The original implementation created v2s2 because the OCI format didn't include a MediaType field, so an empty OCI index was hard to guess the MIME type for. When we modify one, by adding or annotating, though, the format used to save it should switch to OCI if any fields that only exist in OCI are populated.
At push time, it'll be converted to one format or the other to be acceptable at the destination, or if --format is specified.
Is something breaking, or is this a looks-wrong kind of thing?

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[mkarwin]

So in order to get OCI manifest one needs to annotate or add items to the manifest that are OCI-specific in order to enforce a switch to the OCI index format? But the moment we push it to registry it might still be converted back to docker ones?
That's not exactly a solution, not when in theory most of what OCI offered can be somewhat realised in docker schema v2s2.
Furthermore, you might want to create/use OCI format even for the basic/simplest manifests/images from the get go, eg. to test/produce OCI format in your pipeline/tools per the existing requirements, eg. contractual or organisational.
Moreover, you're already offering --format switch for container image producing commands, why not add it to manifest creation one so that you can start with OCI format, after all OCI does support mediaType key per OCI index documentation? I think that's a huge miss/issue here...

[codingben]

Why we still have this issue open, and the priority wasn't raised yet? Something is really bad with Podman and I lost trust from using it. Currently only Docker is the stable solution.

We have pushed OCI-based image index and manifests, and Podman returns wrong results about media type of image index:

╰─$ podman manifest inspect quay.io/boukhano/cirros:6.1                   
{
    "schemaVersion": 2,
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "manifests": [
        {
            "mediaType": "application/vnd.oci.image.manifest.v1+json",
            "size": 418,
            "digest": "sha256:3a37dc3fd4e8460225ef5040308838ae344327c4901cffa653f3e157aae6c4f7",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            }
        },
        {
            "mediaType": "application/vnd.oci.image.manifest.v1+json",
            "size": 418,
            "digest": "sha256:cfefc66dad4addd779ff91774c1592f7a8cc87d3a905ea86bbcd9441045fe143",
            "platform": {
                "architecture": "arm64",
                "os": "linux"
            }
        }
    ]
}

Only Docker returns the media type correctly:

╰─$ docker manifest inspect quay.io/boukhano/cirros:6.1                                         1 ↵
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.oci.image.index.v1+json",
   "manifests": [
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 418,
         "digest": "sha256:3a37dc3fd4e8460225ef5040308838ae344327c4901cffa653f3e157aae6c4f7",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 418,
         "digest": "sha256:cfefc66dad4addd779ff91774c1592f7a8cc87d3a905ea86bbcd9441045fe143",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      }
   ]
}

[codingben]

I've opened a new issue about it, because we have experienced that issue with pulling via Podman only: #22196

[ArieLevs]

Hi, any update/estimate for this issue?
seems I hit exact same issue but only when using docker format (podman in container podman/stable:v5.1)
when using podman build --format oci ...:

{
    "schemaVersion": 2,
    "mediaType": "application/vnd.oci.image.index.v1+json",
    "manifests": [
        {
            "mediaType": "application/vnd.oci.image.manifest.v1+json",
            "size": 4709,
            "digest": "sha256:2cd20d8bd46f0b7b44568f01cde3fa1c3aa182d5b09a023352f920dadd38a9c4",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            }
        }
    ]
}

when using podman build --format docker ...:

{
    "schemaVersion": 2,
    "mediaType": "application/vnd.oci.image.index.v1+json",
    "manifests": [
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "size": 4559,
            "digest": "sha256:1e435a113cb6b449f39bf99e1a9f100462da0de3e71f5c83fdbb37716ef3f4df",
            "platform": {
                "architecture": "amd64",
                "os": "linux"
            }
        }
    ]
}

[ArieLevs]

creating manifest manually podman manifest create some-name
then podman manifest inspect some-name results with:

{
    "schemaVersion": 2,
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "manifests": []
}

(why explicitly docker?)

then running podman build --manifest some-name --format oci ... and podman manifest inspect some-name again

{
    "schemaVersion": 2,
    "mediaType": "application/vnd.oci.image.index.v1+json",
    "manifests": [
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "size": 2001,
            "digest": "sha256:daa423bd37daf90eab2d75765e80f912831872bd6fb76b663391957065002b02",
            "platform": {
                "architecture": "arm64",
                "os": "linux"
            }
        }
    ]
}