Add support for retrieving data from OCI registries

[Blokje5]

Fixes #271

Draft PR showcasing the OCI getter that is used in Conftest. Let me know if it would be something go-getter is interested in including upstream.

Several existing registries with OCI support can be detected and the URL will recieve the oci:// protocol. Alternatively, the oci:// protocol can be added to the URL for other (e.g. private) registries. The deis/ORAS library is used to fetch the OCI artifacts from the storage.

[hashicorp-cla]

All committers have signed the CLA.

[LittleChimera]

This would be also useful for fetching kustomize layers!
@99 @azr?

[developer-guy]

Brilliant idea @Blokje5 👏 I really would love to get involved in this PR, @Blokje5 how can I get involved?

[Blokje5]

@developer-guy If you want you can continue working on the PR. So far it was mostly a PoC of how to include ORAS in go-getter with rather minimal tests.

I am also waiting for feedback from some of the maintainers to see what they think of including this in go-getter.

[developer-guy]

kindly ping 🙋🏻‍♂️

[phisco]

+1 for this to be addressed 🙏

[developer-guy]

kindly ping ☝️

[stevehipwell]

Is anyone still planning to work on this?

[developer-guy]

definitely me ! 🙈

[stevehipwell]

@developer-guy then Cosign verification of modules pulled over OCI?

[developer-guy]

of course we can add that verification logic into the pulling process

[stevehipwell]

@developer-guy I assume that we'd need frontend (e.g. Terraform) integration to make it useful as without configuring the verification we'd just be checking that someone (anyone) ran Cosign on the binary?

[developer-guy]

we'd just be checking that someone (anyone) ran Cosign on the binary.

We can add validation by checking the subject field that points out the person who created that signature if you want more granular control over the resource you pull by limiting people you can trust.

[stevehipwell]

@developer-guy I was questioning how the subject validation data would be passed in without frontend support? E.G. How would I validate a TF module from an OCI repo was signed by a specific party without Terraform providing the capability to pass the information through to this library?

[davidspek]

This would also unblock hashicorp/terraform#31463.

[Blokje5]

Ok I am working on improving this PR, given that there are now also open questions in the Terraform community. I just updated it with later versions of the ORAS lib.

I still have a few open questions:

• How is auth generally handled in go-getter? E.g. I can handle it like conftest (loading docker config settings), but maybe there is a preference for how authentication is handled in go-getter.
• Regarding testing: Would it be ok to setup e.g. a registry in a docker container for tests? Or is it generally preferred to use a mock implementation (e.g. like https://github.com/oras-project/oras-go/blob/main/example_test.go#L110).

cc: @picatz