Preferential order of decryption

[mkamioner]

Hi amazing crypto heroes of AWS!

We are using the library to encrypt our sensitive settings in our project (passwords, API keys etc). We encrypt with two keys, one is a "development-key" and the other is the "production-key". When we decrypt in production, we would like the SDK to not use the development key, however it tries both and always fails the decrypt on the development key. I know we can set up a KmsKeyProvider to only have the ARN of the production key, but that would make our code a lot messier (having to pass it from a dynamic environment variable etc).

Is there a way to set the order of which keys will be used to decrypt when encrypting? Such to say when we encrypt we will use keys "production-key" and then "development-key". During decryption the SDK will try the first and only if it fails will go onto the second. The failure of trying to use the development key in production is taking IO and impacting our Lambda cold start times.

I feel like this feature should already be supported and maybe I am just not able to figure it out. I see that the SDK gives preferences to keys that are in the same region, but both keys are in the same region in my case 🤕

Any insight or help would be greatly appreciated!

Stay awesome and stay safe,
Mo Kamioner

[mattsb42-aws]

Hi, thanks for the question!

There are a few things that are blocking you from doing what you want, but fortunately we have work in flight that should enable you to do this. The biggest piece is adding support for keyrings[1], our new configuration construct that is simpler to use and create than master key/providers and also happens to be more powerful in this case. We're tracking that work here[2], and the code is on the keyring branch if you want to take a look. Note that some if the specific APIs will change a bit between now and release, as noted in the issues in the milestone[1].

The second piece is what order the encrypted data keys (EDKs) are attempted. In the current published version of the AWS Encryption SDK for Python, the client reads the EDKs from the header and puts them in a set, that it then passes into the Cryptographic Materials Manager (CMM). I originally did that to try to clear out duplicates, but we have since determined that processing them in a predictable order is more important. Because of that, we changed the client to read the EDKs into an ordered list so that they are always processed in the order that they were written. That change is landing with the keyring feature work.

Once keyrings land, one thing you could do is to always configure your KMS keyring to use your production CMK as the "generating" CMK, then use the dev CMK as an additional CMK. This will result in the first EDK in the header being the one encrypted by your production CMK, and so will always be the first one that a KMS discovery keyring (analogous to the KMS master key provider's decrypt behavior).

That would work, but to me it feels a bit implicit, and as we know, "Explicit is better than implicit." :)

Another approach you could take is a keyring concept I've been toying with for a "filtering keyring". The keyring interface is pretty simple[3] and it should be much easier to write custom keyrings than master keys/providers. One thing you could do here would be to make a keyring that automatically filters out any EDKs and only passes on specific EDKs to another keyring. Maybe filter on the AWS account ID in the ARN.

ex:

from typing import Iterable

import attr
from attr.validators import instance_of
from aws_encryption_sdk.keyrings.base import Keyring
from aws_encryption_sdk.materials_managers import (
    DecryptionMaterials,
    EncryptionMaterials,
)
from aws_encryption_sdk.structures import EncryptedDataKey


@attr.s
class AccountFilteringKeyring(Keyring):
    _inner_keyring = attr.ib(validator=instance_of(Keyring))
    
    def on_encrypt(
        self,
        encryption_materials: EncryptionMaterials,
    ) -> EncryptionMaterials:
        return self._inner_keyring.on_encrypt(encryption_materials)

    def on_decrypt(
        self,
        decryption_materials: DecryptionMaterials,
        encrypted_data_keys: Iterable[EncryptedDataKey],
    ) -> DecryptionMaterials:
        inner_edks = []
        for edk in encrypted_data_keys:
            # Check for the account ID or somesuch
            # before selecting an EDK to pass to the inner keyring.
            if filter_my_edks(edk):
                inner_edks.append(edk)
        return self._inner_keyring(decryption_materials, encrypted_data_keys)

[1] https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/choose-keyring.html
[2] https://github.com/aws/aws-encryption-sdk-python/milestone/1
[3] https://github.com/awslabs/aws-encryption-sdk-specification/blob/master/framework/keyring-interface.md

[mkamioner]

Hey @mattsb42-aws -- Any update on when this terrific new feature will be ready?

[mattsb42-aws]

Hi @mkamioner,

We don't have a specific timeline yet. As we were wrapping things up to release the keyring feature in Python and Java we realized that there were several unanswered question in the specification that we needed to resolve before going from 2 keyring implementations (C and JS) to 4. We're working through those issues now, some of which will require changes to keyrings in Java and Python. We're tracking the spec issues here[1] and the Python issues here[2].

[1] https://github.com/awslabs/aws-encryption-sdk-specification/milestone/1
[2] https://github.com/aws/aws-encryption-sdk-python/milestone/1

[lucasmcdonald3]

Hi @mkamioner,

Thanks for your patience. Keyrings are now available in the ESDK.

You can use a multi-keyring for your use case: https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/use-multi-keyring.html
If you're only using AWS KMS keys, you can simplify your setup by using an AWS KMS keyring: https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/use-kms-keyring.html

You can configure your decryption order by providing generator and child keys to your keyring, the way @mattsb42-aws described.

Check out our examples directory for code examples configuring keyrings: https://github.com/aws/aws-encryption-sdk-python/tree/master/examples/src

Let us know if you have any questions.

Thanks,
Lucas