Support overlayfs whiteouts on checkout

[mangelajo]

Introduces an intermediate format for overlayfs storage, where .wh-ostree. prefixed files will be converted into char 0:0 whiteout devices used by overlayfs to mark deletions across layers.

The CI scripts now uses a volume for the scratch directories previously in /var/tmp otherwise we cannot create whiteout devices into an overlayfs mounted filesystem.

Related-Issue: #2712
(cherry picked from commit e234b63)

[openshift-ci]

Hi @mangelajo. Thanks for your PR.

I'm waiting for a ostreedev member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[mangelajo]

@cgwalters this backported almost cleanly, only this line conflicted: https://github.com/ostreedev/ostree/pull/2722/files#diff-55145af577b7523e080c400b2cf80caa616163c8eb9c622c1c4ae1d7bf763124L22

[mangelajo]

Oh I see this branch does not run CI in the same way, those are my local results:

============================================================================
Testsuite summary for libostree 2022.2
============================================================================
# TOTAL: 999
# PASS:  955
# SKIP:  44
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

[majopela@lenovo ostree]$ git branch -v
  check-ci-mknod           5d72edf9 [ahead 1] DNM TEST : check CI whiteouts
  main                     c6c3c5ae Merge pull request #2711 from lucab/ups/sign-assertions
  reintroduce-char-devices fffb5b1c Re-introduce support for character device
* rhel8-whiteouts          0085494e [ahead 1] Support overlayfs whiteouts on checkout
  support-char-whiteouts   c2ff1cc1 Apply reviewer comments
  support-whiteouts        e234b630 Support overlayfs whiteouts on checkout
  test-mount               b14d6a17 [ahead 1] Make sure CI uses the host native filesystem for tests

[cgwalters]

https://bugzilla.redhat.com/show_bug.cgi?id=2134629