GenerateKeyError on CentOS

[vikas027]

Environment

OS: CentOS 7 (Interestingly it works well on OSX with exact same versions)
pip: 9.0.1
python: 2.7.14
aws-encryption-sdk: 1.3.3
aws-encryption-sdk-cli: 1.1.4

Problem

Unable to encrypt file. It just fails with GenerateKeyError

Logs

$ aws-encryption-cli --encrypt  \
  --input server.crt \
  --master-keys key=$cmkArn \
  --metadata-output ~/metadata \
  --encryption-context purpose=test \
   --output .
2018-03-08 00:30:31,112 - MainThread - aws_encryption_sdk_cli - WARNING - Operation failed: deleting output file: ./server.crt.encrypted
Encountered unexpected error: increase verbosity to see details.
GenerateKeyError("Master Key <my_kms_key_arn> unable to generate data key")
$

Verbose

aws-encryption-cli --encrypt \
                      --input server.crt \
                      --master-keys key=$cmkArn \
                      --metadata-output ~/metadata \
                      --encryption-context purpose=test \
                      --output . -vvv
2018-03-08 00:14:15,190 - MainThread - aws_encryption_sdk_cli - DEBUG - Encryption mode: encrypt
2018-03-08 00:14:15,190 - MainThread - aws_encryption_sdk_cli - DEBUG - Encryption source: server.crt
2018-03-08 00:14:15,191 - MainThread - aws_encryption_sdk_cli - DEBUG - Encryption destination: .
2018-03-08 00:14:15,191 - MainThread - aws_encryption_sdk_cli - DEBUG - Master key provider configuration: [{'key': ['<my_kms_key_arn>'], 'provider': 'aws-encryption-sdk-cli::aws-kms'}]
2018-03-08 00:14:15,191 - MainThread - aws_encryption_sdk_cli - DEBUG - Suffix requested: None
2018-03-08 00:14:15,191 - MainThread - aws_encryption_sdk_cli - DEBUG - Loading provider: aws-encryption-sdk-cli::aws-kms
2018-03-08 00:14:15,191 - MainThread - aws_encryption_sdk_cli - DEBUG - Discovering master key provider plugins
2018-03-08 00:14:15,222 - MainThread - aws_encryption_sdk_cli - INFO - Collecting plugin "aws-kms" registered by "aws-encryption-sdk-cli 1.1.4"
2018-03-08 00:14:15,223 - MainThread - aws_encryption_sdk_cli - DEBUG - Plugin details: {'module_name': 'aws_encryption_sdk_cli.key_providers', 'extras': (), 'dist': aws-encryption-sdk-cli 1.1.4 (/usr/lib/python2.7/site-packages), 'attrs': ('aws_kms_master_key_provider',), 'name': 'aws-kms'}
2018-03-08 00:14:15,324 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2018-03-08 00:14:15,530 - MainThread - aws_encryption_sdk_cli - DEBUG - Requested source: server.crt
2018-03-08 00:14:15,531 - MainThread - aws_encryption_sdk_cli - DEBUG - Expanded source: ['server.crt']
2018-03-08 00:14:15,531 - MainThread - aws_encryption_sdk_cli - DEBUG - Duplicating filename server.crt into .
2018-03-08 00:14:15,545 - MainThread - aws_encryption_sdk_cli - INFO - encrypting file server.crt to ./server.crt.encrypted
2018-03-08 00:14:15,668 - MainThread - aws_encryption_sdk.key_providers.base - INFO - generating data key with encryption context: {'aws-crypto-public-key': u'<some_random_key>', 'purpose': 'test'}
2018-03-08 00:14:15,695 - MainThread - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): kms.ap-southeast-2.amazonaws.com
Traceback (most recent call last):
  File "/usr/lib64/python2.7/logging/__init__.py", line 851, in emit
    msg = self.format(record)
  File "/usr/lib64/python2.7/logging/__init__.py", line 724, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/site-packages/aws_encryption_sdk_cli/internal/logging_utils.py", line 137, in format
    _record = self.__redact_record(record)
  File "/usr/lib/python2.7/site-packages/aws_encryption_sdk_cli/internal/logging_utils.py", line 123, in __redact_record
    _record = copy.deepcopy(record)
  File "/usr/lib64/python2.7/copy.py", line 190, in deepcopy
    y = _reconstruct(x, rv, 1, memo)
  File "/usr/lib64/python2.7/copy.py", line 334, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib64/python2.7/copy.py", line 163, in deepcopy
    y = copier(x, memo)
  File "/usr/lib64/python2.7/copy.py", line 257, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib64/python2.7/copy.py", line 163, in deepcopy
    y = copier(x, memo)
  File "/usr/lib64/python2.7/copy.py", line 237, in _deepcopy_tuple
    y.append(deepcopy(a, memo))
  File "/usr/lib64/python2.7/copy.py", line 190, in deepcopy
    y = _reconstruct(x, rv, 1, memo)
  File "/usr/lib64/python2.7/copy.py", line 329, in _reconstruct
    y = callable(*args)
TypeError: __init__() takes exactly 3 arguments (2 given)
Logged from file kms.py, line 233
2018-03-08 00:14:15,841 - MainThread - aws_encryption_sdk_cli - WARNING - Operation failed: deleting output file: ./server.crt.encrypted
2018-03-08 00:14:15,842 - MainThread - aws_encryption_sdk_cli - DEBUG - Encountered unexpected error: increase verbosity to see details.
GenerateKeyError("Master Key <my_kms_key_arn> unable to generate data key")
2018-03-08 00:14:15,843 - MainThread - aws_encryption_sdk_cli - DEBUG - Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/aws_encryption_sdk_cli/__init__.py", line 273, in cli
    process_cli_request(stream_args, args)
  File "/usr/lib/python2.7/site-packages/aws_encryption_sdk_cli/__init__.py", line 216, in process_cli_request
    destination=_destination
  File "/usr/lib/python2.7/site-packages/aws_encryption_sdk_cli/internal/io_handling.py", line 336, in process_single_file
    destination=destination
  File "/usr/lib/python2.7/site-packages/aws_encryption_sdk_cli/internal/io_handling.py", line 266, in process_single_operation
    destination_writer=destination_writer
  File "/usr/lib/python2.7/site-packages/aws_encryption_sdk_cli/internal/io_handling.py", line 207, in _single_io_write
    header=json_ready_header(handler.header)
  File "/usr/lib/python2.7/site-packages/aws_encryption_sdk/streaming_client.py", line 175, in header
    self._prep_message()
  File "/usr/lib/python2.7/site-packages/aws_encryption_sdk/streaming_client.py", line 405, in _prep_message
    request=encryption_materials_request
  File "/usr/lib/python2.7/site-packages/aws_encryption_sdk/materials_managers/default.py", line 94, in get_encryption_materials
    encryption_context=encryption_context
  File "/usr/lib/python2.7/site-packages/aws_encryption_sdk/internal/utils/__init__.py", line 111, in prepare_data_keys
    data_encryption_key = primary_master_key.generate_data_key(algorithm, encryption_context)
  File "/usr/lib/python2.7/site-packages/aws_encryption_sdk/key_providers/base.py", line 433, in generate_data_key
    encryption_context=encryption_context
  File "/usr/lib/python2.7/site-packages/aws_encryption_sdk/key_providers/kms.py", line 234, in _generate_data_key
    raise GenerateKeyError(error_message)
GenerateKeyError: Master Key <my_kms_key_arn> unable to generate data key

Encountered unexpected error: increase verbosity to see details.
GenerateKeyError("Master Key <my_kms_key_arn> unable to generate data key")

[mattsb42-aws]

It looks like there's an odd error with our log handler, which I'll look into in #145 , but that wouldn't trigger the GenerateKeyError.

My first thought is that the AWS principal (IAM user or role) that you are using does not have the kms:GenerateDataKey permission for the CMK you requested.

Are you using the same principal on both hosts that you tested, and if not, have you verified that the principal used on the CentOS host has the necessary permissions for the CMK?

[vikas027]

Hey @mattsb42-aws ,

Yeah, I have too found that and was about to close the issue with the same findings. My IAM policy needed "kms:GenerateDataKey*" in Action.

Sorry for the noise.

Cheers,
Vikas