Ratify to support out-of-box experience for typical scenarios

[yizha1]

What would you like to be added?

This issue is for Ratify to provide out-of-box experience in the context of Notary Project signature validation on K8s clusters.

There are three typical scenarios:

• Scenario 1: The k8s workload uses images signed and stored in a public registry.
• Scenario 2: The k8s workload uses images signed and stored in a private registry with certificates stored in AKV
• Scenario 3: The k8s workload uses images signed and stored in a both public and private registry.

After helm installatioin, Ratify should be ready to work. Users should not be required to configure any custom resources by default. For advanced users or any runtime updates, users can update custom resources accordingly based on their needs. Thus, Ratify helm chart should support necessary parameters to set up resources: ORAS Stores, KMP and Notation Verifier.

Scenario 1: As images are stored in a public registry, users can skip the configuration for ORAS store, but able to create the following resources by specifying helm chart values:

• Inline KMPs for storing the PEM formatted public root certificates
• Notation Verifier for setting up Notary Project trust policies

Scenario 2: Users should be able to create the following resource by specifying helm chart values:

• ORAS stores for accessing private registries
• AKV KMPs for accessing certificates stored in AKV
• Notation Verifier for setting up Notary Project trust policies

Scenario 3: Users should be able to create the following resource by specifying helm chart values:

• ORAS stores for accessing private registries
• AKV KMPs for accessing certificates stored in AKV
• inline KMPs for storing the PEM formatted public root certificates
• Notation Verifier for setting up Notary Project trust policies separately for images in public registries and in private registries

Currently, helm chart parameters exist for ORAS stores and KMP resources, but not for Notation Verifier. We need to add parameters for setting up Notation Verifier. The issue #1943 asked for trustedIdentities configuration, but it is not enough, we should allow users to

• set up multiple trust policies
• For each trust policy, users can set up
  • RepositoryScopes: a list of repository URLs
  • KMPs for trustStores
  • trustedIdentities

An example of Notation Verifier parameters will be provided in the issue comment later.

NOTE: This issue does not include the configuration required for setting up admission policies, which could be tracked in another issue for UX improvements.

Anything else you would like to add?

No response

Are you willing to submit PRs to contribute to this feature?

• Yes, I am willing to implement it.

[susanshi]

We discussed this during the community meeting today. We can explore adding steps in helmfile or directly update the helmCharts. @yizha1 , please provide samples of the desired --set values command and result CR for reference.

[yizha1]

The standard Notary Project trust policy:

{
    "version": "1.0",
    "trustPolicies": [
        {
            "name": "acme-rockets-images",
            "registryScopes": [ "registry.acme-rockets.io/software/net-monitor", "registry.acme-rockets.io/software/net-logger" ],
            "signatureVerification": {
              "level" : "strict",
            },
            "trustStores": ["ca:acme-rockets", "tsa:trusted-tsa"],
            "trustedIdentities": [
              "x509.subject: C=US, ST=WA, L=Seattle, O=acme-rockets.io, OU=Finance, CN=SecureBuilder",
              "x509.subject: C=US, ST=WA, L=Seattle, O=acme-rockets.io, OU=Design, CN=SecureBuilder",
            ]
        },
       {
            "name": "public-images",
            "registryScopes": [ "registry.wabbit-networks.io/software/net-utils" ],
            "signatureVerification": {
              "level" : "strict",
            },
            "trustStores": ["ca:wabbit-networks", "tsa:trusted-tsa"],
            "trustedIdentities": [
              "x509.subject: C=US, ST=WA, L=Seattle, O=wabbit-networks.io, OU=Security Tools"
            ]
        }
    ]
}

An example for the default Notation configuration in values.yaml

notation:
  enabled: true
  trustPolicies:
    - name: default
      registryScopes:
        - "*"  // Empty or "*" for Any images
      trustStores:
        - type: ca
            - filePath: "./notation.cert" // MUST provide the root certificate
        - type: tsa
      trustIdentities:
          - "*" //Empty or "*" for any identitifies issued by the root certificate, not secure

Notation verifier based on the default configuration:

apiVersion: config.ratify.deislabs.io/v1beta1
kind: Verifier
metadata:
  name: verifier-notation
spec:
  name: notation
  artifactTypes: application/vnd.cncf.notary.signature
  parameters:
    verificationCertStores:
      ca:
        certs1:
            - gatekeeper-system/inline-kmp-notation
    trustPolicyDoc:
      version: "1.0"
      trustPolicies:
        - name: acme-rockets-images
          registryScopes:
             - "*"
          signatureVerification:
            level: strict
          trustStores:
            - ca:certs1
          trustedIdentities:
            - "*"

An example for user updated configuration in values.yaml

notation:
  enabled: true
  trustPolicies:
    - name: acme-rockets-images
      registryScopes:
        - "registry.acme-rockets.io/software/net-monitor"
        - "registry.acme-rockets.io/software/net-logger"
       trustStores:
          - type: ca
             certificates:
                - azurekeyvault:
                      vaultURI:
                      tenantId:
                      certificates: []
                      keys: []
                      refreshInterval:
          - type: tsa
             certificates:
                - filePath: "./tsa.cert"
        trustIdentities:
           - "C=US, ST=WA, L=Seattle, O=acme-rockets.io, OU=Finance, CN=SecureBuilder",
           - "C=US, ST=WA, L=Seattle, O=acme-rockets.io, OU=Design, CN=SecureBuilder"
    - name: public-images
       registryScopes:
         - "registry.wabbit-networks.io/software/net-utils"
       trustStores:
          - type: ca
             certificates:
                - filePath: "./notation.cert"
          - type: tsa
             certificates:
                - filePath: "./tsa.cert"
        trustIdentities:
           - "C=US, ST=WA, L=Seattle, O=wabbit-networks.io, OU=Security Tools"

After helm installation

There will be four resources created based on the above example:

• An inline KMP based on the value of filePath ./notation.cert
• An inline KMP based on the value of filePath ./tsa.cert
• An AKV KMP based on the value of azurekeyvault
• A Notation Verifier as the following

apiVersion: config.ratify.deislabs.io/v1beta1
kind: Verifier
metadata:
  name: verifier-notation
spec:
  name: notation
  artifactTypes: application/vnd.cncf.notary.signature
  parameters:
    verificationCertStores:
      ca:
        certs1:
            - gatekeeper-system/akv-kmp
        certs2
            - gatekeeper-system/inline-kmp-notation
      tsa:
         certs:
            - gatekeeper-system/inline-kmp-tsa
    trustPolicyDoc:
      version: "1.0"
      trustPolicies:
        - name: acme-rockets-images
          registryScopes:
             - "registry.acme-rockets.io/software/net-monitor"
             - "registry.acme-rockets.io/software/net-logger"
          signatureVerification:
            level: strict
          trustStores:
            - ca:certs1
            - tsa:certs
          trustedIdentities:
            - "x509.subject: C=US, ST=WA, L=Seattle, O=acme-rockets.io, OU=Finance, CN=SecureBuilder",
            - "x509.subject: C=US, ST=WA, L=Seattle, O=acme-rockets.io, OU=Design, CN=SecureBuilder"
        - name: public-images
          registryScopes:
             - "registry.wabbit-networks.io/software/net-utils"
          signatureVerification:
            level: strict
          trustStores:
            - ca:certs2
            - tsa:certs
          trustedIdentities:
            - "x509.subject: C=US, ST=WA, L=Seattle, O=wabbit-networks.io, OU=Security Tools"

[susanshi]

Hi @yizha1, in response to " how to connect the KMP to the notation verifier". In this sample verifier, trustStore contains a reference to kmp.

verificationCertStores:
      ca:
        ca-certs: 
          - gatekeeper-system/kmp-akv-ca
      tsa:
        tsa-certs: 
          - gatekeeper-system/kmp-akv-tsa
 trustStores: # trustStore must be trust-store-type:trust-store-name specified in verificationCertStores
            - ca:ca-certs
            - tsa:tsa-certs

[yizha1]

Hi @yizha1, in response to " how to connect the KMP to the notation verifier". In this sample verifier, trustStore contains a reference to kmp.

verificationCertStores:
      ca:
        ca-certs: 
          - gatekeeper-system/kmp-akv-ca
      tsa:
        tsa-certs: 
          - gatekeeper-system/kmp-akv-tsa
 trustStores: # trustStore must be trust-store-type:trust-store-name specified in verificationCertStores
            - ca:ca-certs
            - tsa:tsa-certs

During installation, users do not know the name of a KMP. Ratify needs to render KMP resources based on user input and then reference it in the verifier resource.

[yizha1]

@shahramk64 @susanshi @akashsinghal @binbin-li @duffney I made an update on the comment #1965 (comment). Any suggestions on simplying the configuration are welcome.

For helm installation command, as there are several parameters and layers for Notation configuration, I would suggest that users can use -f values.yaml combined with --set xxx parameters. Or can we split the notation configuration into a separate file, named notation-verifier-values.yaml, then users can only update this file and use -f notation-verifier-values.yaml.

Example:

helm install ratify \
    ratify/ratify --atomic \
    --namespace ${RATIFY_NAMESPACE} --create-namespace \
    --set featureFlags.RATIFY_CERT_ROTATION=true \
    --set azureWorkloadIdentity.clientId=${IDENTITY_CLIENT_ID} \
    -f notation-verifier-values.yaml

[shahramk64]

@yizha1: It seems like it's not feasible to change the schema of the trustPolicies. The schema is determined by Notation and it's defined here, and Ratify should provide the same set of properties.

[yizha1]

@yizha1: It seems like it's not feasible to change the schema of the trustPolicies. The schema is determined by Notation and it's defined here, and Ratify should provide the same set of properties.

@shahramk64 can you clarify your questions? The proposal is not change the trust policies schema, Ratify just fetch the necessary values from the input and build the trust policies as required by Notary Project trust policies.

[shahramk64]

@yizha1: It seems like it's not feasible to change the schema of the trustPolicies. The schema is determined by Notation and it's defined here, and Ratify should provide the same set of properties.

@shahramk64 can you clarify your questions? The proposal is not change the trust policies schema, Ratify just fetch the necessary values from the input and build the trust policies as required by Notary Project trust policies.

Hi @yizha1, sorry for the confusion.
In your example of values.yaml:

trustStores:
          - type: ca
             certificates:
                - azurekeyvault:
                      vaultURI:
                      tenantId:
                      certificates: []
                      keys: []
                      refreshInterval:

trustStores is defined in a way that contains AKV information. I thought that's updating the trustStore schema, but that's just in the values.yaml. However, That info is currently being provided in the values.yaml file in azureKeyVault attribute.

I moved the trustPolicies attributes to the values.yaml file so that the user can provide their own set of values, and I set the default values to the existing values in verifier.yaml (so that the tests pass). This way the user can define multiple trustPolicies in values.yaml and provide their own trustStores and trustedIdentities in each trustPolicy. The thing that doesn't seem to be feasible is matching the certStores with trustedIdentities automatically during the installation for each trustPolicy. I am not sure if that was your intension though. Please have a look at my PR and let me know if it's not providing the capability that you are after.

cc: @susanshi