Skip to content

Latest commit

 

History

History
148 lines (127 loc) · 11.9 KB

README.md

File metadata and controls

148 lines (127 loc) · 11.9 KB
Raw

Compatibility

As of Module version v1.8.0, Plaintext Ports are no longer allowed through the module's security groups for both the broker and zookeeper

This means that from module v1.8.0 onwards the minimum supported Kafka version is 2.5.1

Should you require an older version of Kafka than you should use module version v1.7.x. However, the downside is that plaintext ports will be allowed on the older module version

Usage

MSK Cluster

module "msk_cluster" {
  source = "git::https://github.com/UKHomeOffice/acp-tf-msk-cluster?ref=master"

  name                   = "msktestcluster"
  msk_instance_type      = "kafka.m5.large"
  kafka_version          = "2.8.1"
  environment            = var.environment
  number_of_broker_nodes = "3"
  subnet_ids             = data.aws_subnet_ids.compute.ids
  vpc_id                 = var.vpc_id
  ebs_volume_size        = "50"
  cidr_blocks            = values(var.compute_cidrs)
  # certificateauthority = true (This will fail on merge the first time it's executed, this is expected. Install the CA in the AWS console then restart the merge.)
  # or
  # ca_arn               = [module.<existing_cert>.ca_certificate_arn]
}

MSK Cluster with config

module "msk_cluster_with_config" {
  source = "git::https://github.com/UKHomeOffice/acp-tf-msk-cluster?ref=master"

  name                        = "msktestclusterwithconfig"
  msk_instance_type           = "kafka.m5.large"
  kafka_version               = "2.8.1"
  environment                 = var.environment
  number_of_broker_nodes      = "3"
  subnet_ids                  = data.aws_subnet_ids.compute.ids
  vpc_id                      = var.vpc_id
  ebs_volume_size             = "50"
  cidr_blocks                 = values(var.compute_cidrs)
  # certificateauthority      = true (This will fail on merge the first time it's executed, this is expected. Install the CA in the AWS console then restart the merge.)
  # or
  # ca_arn                    = [module.<existing_cert>.ca_certificate_arn]
  config_name                 = "test-msk-config"
  config_kafka_versions       = ["2.8.1"]
  config_description          = "Test MSK configuration"

  config_server_properties = <<PROPERTIES
 auto.create.topics.enable = true
 delete.topic.enable = true
 PROPERTIES
}

Requirements

Name Version
terraform >= 1.0
aws ~> 3.0

Providers

Name Version
aws 3.76.1

Modules

Name Source Version
self_serve_access_keys git::https://github.com/UKHomeOffice/acp-tf-self-serve-access-keys v0.1.0

Resources

Name Type
aws_acmpca_certificate_authority.msk_kafka_ca_with_config resource
aws_acmpca_certificate_authority.msk_kafka_with_ca resource
aws_appautoscaling_policy.msk_appautoscaling_policy resource
aws_appautoscaling_target.msk_appautoscaling_target resource
aws_iam_policy.acmpca_policy_with_msk_policy resource
aws_iam_policy.msk_iam_policy resource
aws_iam_policy_attachment.msk_acmpca_iam_policy_attachment resource
aws_iam_policy_attachment.msk_iam_policy_attachment resource
aws_iam_user.msk_acmpca_iam_user resource
aws_iam_user.msk_iam_user resource
aws_iam_policy.msk_iam_authentication resource
aws_iam_policy_attachment.msk_iam_authentication_policy resource
aws_kms_alias.msk_cluster_kms_alias resource
aws_kms_key.kms resource
aws_msk_cluster.msk_kafka resource
aws_msk_cluster.msk_kafka_with_config resource
aws_msk_configuration.msk_kafka_config resource
aws_security_group.sg_msk resource
aws_caller_identity.current data source
aws_iam_policy_document.kms_key_policy_document data source

Inputs

Name Description Type Default Required
acmpca_iam_user_name The name of the IAM user assigned to the created AWS Private CA string "" no
ca_arn ARN of the AWS managed CA to attach to the MSK cluster list(string) [] no
certificateauthority Should a CA be created with the MSK cluster? bool false no
cidr_blocks The CIDR blocks that the MSK cluster allows ingress connections from list(string) 
[
  "0.0.0.0/0"
]
 no
config_arn ARN of the MSK configuration to attach to the MSK cluster string "" no
config_description The description of the MSK configuration string "" no
config_kafka_versions A list of Kafka versions that the configuration supports list(string) [] no
config_name Name of the MSK configuration to attach to the MSK cluster string "" no
config_revision The revision of the MSK configuration to use string "" no
config_server_properties The properties to set on the MSK cluster. Omitted properties are set to a default value string "" no
ebs_volume_size The MSK cluster EBS volume size for each broker any n/a yes
email_addresses A list of email addresses for key rotation notifications. list(string) [] no
encryption_at_rest_kms_key_arn Use to set custom KMS key to encrypt data written to EBS volume any null no
enhanced_monitoring The desired enhanced MSK CloudWatch monitoring level string "DEFAULT" no
environment The environment the MSK cluster is running in i.e. dev, prod etc any n/a yes
iam_authentication Enables IAM client authentication. bool "false" no
kafka_version The Kafka version for the AWS MSK cluster string "2.2.1" no
key_rotation Enable email notifications for old IAM keys. string "true" no
logging_broker_s3 Configuration block for Broker Logs settings for s3. 
object({
    enabled = bool
    bucket  = string
    prefix  = string
  })
 null no
msk_instance_type The MSK cluster instance type any n/a yes
name Name of the MSK cluster any n/a yes
number_of_broker_nodes The number of broker nodes running in the MSK cluster any n/a yes
prometheus_jmx_exporter_enabled Enable Prometheus open monitoring for the JMX exporter bool false no
prometheus_node_exporter_enabled Enable Prometheus open monitoring for the node exporter bool false no
storage_autoscaling_max_capacity The MSK cluster EBS maximum volume size for each broker. Value between 1 and 16384. number 1 no
storage_autoscaling_threshold The percentage threshold that needs to be exceeded to trigger a scale up. Value between 10 and 80. number 65 no
subnet_ids A list of subnets that the MSK cluster should run in list(string) n/a yes
tags A map of tags to add to all resources map(string) {} no
type The type of the certificate authority string "" no
vpc_id The MSK cluster's VPC ID any n/a yes

Outputs

Name Description
bootstrap_brokers Plaintext connection host:port pairs
bootstrap_brokers_tls TLS connection host:port pairs
msk_cluster_arn The MSK cluster arn
msk_sg_id The MSK security group ID
zookeeper_connect_string A comma separated list of one or more IP:port pairs to use to connect to the Apache Zookeeper cluster