Sync devDeploy with recent Replayer auth changes (opensearch-project#279

) * Sync devDeploy with recent Replayer auth changes Signed-off-by: Tanner Lewis <[email protected]>
lewijacn · Aug 22, 2023 · 4165dc4 · 4165dc4
1 parent 3fd28dd
commit 4165dc4
Show file tree

Hide file tree

Showing 4 changed files with 32 additions and 78 deletions.
diff --git a/TrafficCapture/trafficReplayer/README.md b/TrafficCapture/trafficReplayer/README.md
@@ -83,3 +83,12 @@ this class uses [JOLT](https://github.com/bazaarvoice/jolt) to perform transform
 operations that are defined in the [resources](../trafficReplayer/src/main/resources/jolt/operations) associated with
 the package. Future work will include adding more JSON transformations and other potential JSON transformation tools
 (like [JMESPath](https://jmespath.org/)).
+
+
+## Authorization Header for Replayed Requests
+
+There is a level of precedence that will determine which or if any Auth header should be added to outgoing Replayer requests, which is listed below.
+1. If the user provides an explicit auth header option to the Replayer, such as providing a static value auth header or using a user and secret arn pair, this mechanism will be used for the auth header of outgoing requests. The options can be found as Parameters [here](src/main/java/org/opensearch/migrations/replay/TrafficReplayer.java)
+2. If the devDeploy [script](../../deployment/copilot/devDeploy.sh) is used and its Replayer command has not been altered (as in the case of 1.) and CDK deploys a target cluster with a configured FGAC user (see `fineGrainedManagerUserName` and `fineGrainedManagerUserSecretManagerKeyARN` CDK context options [here](../../deployment/cdk/opensearch-service-migration/README.md#configuration-options)) or is running in demo mode (see `enableDemoAdmin` CDK context option), this user and secret arn pair will be provided in the Replay command for the Auth header
+3. If the user provides no auth header option and incoming captured requests have an auth header, this auth header will be reused for outgoing requests
+4. If the user provides no auth header option and incoming captured requests have no auth header, then no auth header will be used for outgoing requests
diff --git a/deployment/cdk/opensearch-service-migration/lib/opensearch-service-domain-cdk-stack.ts b/deployment/cdk/opensearch-service-migration/lib/opensearch-service-domain-cdk-stack.ts
@@ -125,11 +125,10 @@ export class OpensearchServiceDomainCdkStack extends Stack {
       `export MIGRATION_DOMAIN_ENDPOINT=${this.domainEndpoint}`
     ]
     if (domain.masterUserPassword && !adminUserSecret) {
-      console.log("A master user was configured without an existing Secrets Manager secret, will not export MIGRATION_DOMAIN_USER_NAME and MIGRATION_DOMAIN_USER_SECRET_ARN for Copilot")
+      console.log("A master user was configured without an existing Secrets Manager secret, will not export MIGRATION_DOMAIN_USER_AND_SECRET_ARN for Copilot")
     }
     else if (domain.masterUserPassword && adminUserSecret) {
-      exports.push(`export MIGRATION_DOMAIN_USER_NAME=${adminUserName}`)
-      exports.push(`export MIGRATION_DOMAIN_USER_SECRET_ARN=${adminUserSecret.secretArn}`)
+      exports.push(`export MIGRATION_DOMAIN_USER_AND_SECRET_ARN=${adminUserName} ${adminUserSecret.secretArn}`)
     }
     new CfnOutput(this, 'CopilotDomainExports', {
       value: exports.join(";"),

diff --git a/deployment/copilot/README.md b/deployment/copilot/README.md
@@ -39,16 +39,13 @@ Options:
 Deploy migration solution infrastructure composed of resources deployed by CDK and Copilot
 
 Options:
-  --skip-bootstrap          Skip one-time setup of installing npm package, bootstrapping CDK, and building Docker images.
-  --skip-copilot-init       Skip one-time Copilot initialization of app, environments, and services
-  --copilot-app-name        [string, default: migration-copilot] Specify the Copilot application name to use for deployment
-  --destroy-env             Destroy all CDK and Copilot CloudFormation stacks deployed, excluding the Copilot app level stack, for the given env/stage and return to a clean state.
-  --destroy-all-copilot     Destroy Copilot app and all Copilot CloudFormation stacks deployed for the given app across all regions.
-  --auth-header-value       [string, default: null] Prepared "authorization" header to provide the Replayer, i.e. Basic YWRtaW46QWRtaW4xMjMh. This will override a CDK configured FGAC master user auth header if setup
-  --aws-auth-header-user    [string, default: null] Plaintext username to provide the Replayer to construct an "authorization" header. Used in conjunction with --aws-auth-header-secret. This will override a CDK configured FGAC master user auth header if setup
-  --aws-auth-header-secret  [string, default: null] Secret ARN or Secret name from AWS Secrets Manager to provide the Replayer to construct an "authorization" header. Used in conjunction with --aws-auth-header-user. This will override a CDK configured FGAC master user auth header if setup
-  -r, --region              [string, default: us-east-1] Specify the AWS region to deploy the CloudFormation stacks and resources.
-  -s, --stage               [string, default: dev] Specify the stage name to associate with the deployed resources
+  --skip-bootstrap                      Skip one-time setup of installing npm package, bootstrapping CDK, and building Docker images.
+  --skip-copilot-init                   Skip one-time Copilot initialization of app, environments, and services
+  --copilot-app-name                    [string, default: migration-copilot] Specify the Copilot application name to use for deployment
+  --destroy-env                         Destroy all CDK and Copilot CloudFormation stacks deployed, excluding the Copilot app level stack, for the given env/stage and return to a clean state.
+  --destroy-all-copilot                 Destroy Copilot app and all Copilot CloudFormation stacks deployed for the given app across all regions.
+  -r, --region                          [string, default: us-east-1] Specify the AWS region to deploy the CloudFormation stacks and resources.
+  -s, --stage                           [string, default: dev] Specify the stage name to associate with the deployed resources
 
 ```
 
@@ -58,10 +55,7 @@ Requirements:
 
 #### How is an Authorization header set for requests from the Replayer to the target cluster?
 
-There is a level of precedence that will determine which or if any Auth header should be added to outgoing Replayer requests, which is listed below:
-1. `[--auth-header-value]` or `[--aws-auth-header-user, --aws-auth-header-secret]` are provided to the deployment script and will be used
-2. If the CDK deploys a target cluster with a configured FGAC user (see `fineGrainedManagerUserName` and `fineGrainedManagerUserSecretManagerKeyARN` CDK context options [here](../cdk/opensearch-service-migration/README.md)) or is running in demo mode (see `enableDemoAdmin` CDK context option), this username and secret key will be used for the Auth header
-3. Lastly, the Replayer will not use an explicit Auth header and instead use the same Auth header from the capture source cluster request, if one exists
+See Replayer explanation [here](../../TrafficCapture/trafficReplayer/README.md#authorization-header-for-replayed-requests)
 
 ### Deploy commands one at a time
 
@@ -74,8 +68,7 @@ The provided CDK will output export commands once deployed that can be ran on a
 ```
 export MIGRATION_DOMAIN_SG_ID=sg-123;
 export MIGRATION_DOMAIN_ENDPOINT=vpc-aos-domain-123.us-east-1.es.amazonaws.com;
-export MIGRATION_DOMAIN_USER_NAME=admin
-export MIGRATION_DOMAIN_USER_SECRET_ARN=arn:aws:secretsmanager:us-east-1:123456789123:secret:demo-user-secret-123abc
+export MIGRATION_DOMAIN_USER_AND_SECRET_ARN=admin arn:aws:secretsmanager:us-east-1:123456789123:secret:demo-user-secret-123abc
 export MIGRATION_VPC_ID=vpc-123;
 export MIGRATION_CAPTURE_MSK_SG_ID=sg-123;
 export MIGRATION_COMPARATOR_EFS_ID=fs-123;
@@ -88,7 +81,7 @@ export MIGRATION_KAFKA_BROKER_ENDPOINTS=b-1-public.loggingmskcluster.123.45.kafk
 ```
 Additionally, if not using the deploy script, the following export is needed for the Replayer service:
 ```
-export MIGRATION_REPLAYER_COMMAND=/bin/sh -c "/runJavaWithClasspath.sh org.opensearch.migrations.replay.TrafficReplayer $MIGRATION_DOMAIN_ENDPOINT --insecure --kafka-traffic-brokers $MIGRATION_KAFKA_BROKER_ENDPOINTS --kafka-traffic-topic logging-traffic-topic --kafka-traffic-group-id default-logging-group --kafka-traffic-enable-msk-auth --aws-auth-header-user $MIGRATION_DOMAIN_USER_NAME --aws-auth-header-secret $MIGRATION_DOMAIN_USER_SECRET_ARN | nc traffic-comparator 9220"
+export MIGRATION_REPLAYER_COMMAND=/bin/sh -c "/runJavaWithClasspath.sh org.opensearch.migrations.replay.TrafficReplayer $MIGRATION_DOMAIN_ENDPOINT --insecure --kafka-traffic-brokers $MIGRATION_KAFKA_BROKER_ENDPOINTS --kafka-traffic-topic logging-traffic-topic --kafka-traffic-group-id default-logging-group --kafka-traffic-enable-msk-auth --auth-header-user-and-secret $MIGRATION_DOMAIN_USER_AND_SECRET_ARN | nc traffic-comparator 9220"
 ```
 
 #### Setting up existing Copilot infrastructure

diff --git a/deployment/copilot/devDeploy.sh b/deployment/copilot/devDeploy.sh
@@ -18,16 +18,13 @@ usage() {
   echo "Deploy migration solution infrastructure composed of resources deployed by CDK and Copilot"
   echo ""
   echo "Options:"
-  echo "  --skip-bootstrap          Skip one-time setup of installing npm package, bootstrapping CDK, and building Docker images."
-  echo "  --skip-copilot-init       Skip one-time Copilot initialization of app, environments, and services"
-  echo "  --copilot-app-name        [string, default: migration-copilot] Specify the Copilot application name to use for deployment"
-  echo "  --destroy-env             Destroy all CDK and Copilot CloudFormation stacks deployed, excluding the Copilot app level stack, for the given env/stage and return to a clean state."
-  echo "  --destroy-all-copilot     Destroy Copilot app and all Copilot CloudFormation stacks deployed for the given app across all regions."
-  echo "  --auth-header-value       [string, default: null] Prepared \"authorization\" header to provide the Replayer, i.e. Basic YWRtaW46QWRtaW4xMjMh. This will override a CDK configured FGAC master user auth header if setup"
-  echo "  --aws-auth-header-user    [string, default: null] Plaintext username to provide the Replayer to construct an \"authorization\" header. Used in conjunction with --aws-auth-header-secret. This will override a CDK configured FGAC master user auth header if setup"
-  echo "  --aws-auth-header-secret  [string, default: null] Secret ARN or Secret name from AWS Secrets Manager to provide the Replayer to construct an \"authorization\" header. Used in conjunction with --aws-auth-header-user. This will override a CDK configured FGAC master user auth header if setup"
-  echo "  -r, --region              [string, default: us-east-1] Specify the AWS region to deploy the CloudFormation stacks and resources."
-  echo "  -s, --stage               [string, default: dev] Specify the stage name to associate with the deployed resources"
+  echo "  --skip-bootstrap                      Skip one-time setup of installing npm package, bootstrapping CDK, and building Docker images."
+  echo "  --skip-copilot-init                   Skip one-time Copilot initialization of app, environments, and services"
+  echo "  --copilot-app-name                    [string, default: migration-copilot] Specify the Copilot application name to use for deployment"
+  echo "  --destroy-env                         Destroy all CDK and Copilot CloudFormation stacks deployed, excluding the Copilot app level stack, for the given env/stage and return to a clean state."
+  echo "  --destroy-all-copilot                 Destroy Copilot app and all Copilot CloudFormation stacks deployed for the given app across all regions."
+  echo "  -r, --region                          [string, default: us-east-1] Specify the AWS region to deploy the CloudFormation stacks and resources."
+  echo "  -s, --stage                           [string, default: dev] Specify the stage name to associate with the deployed resources"
   exit 1
 }
 
@@ -38,10 +35,6 @@ DESTROY_ENV=false
 DESTROY_ALL_COPILOT=false
 REGION=us-east-1
 STAGE=dev
-
-REPLAY_AUTH_HEADER=""
-REPLAY_AWS_AUTH_HEADER_USER=""
-REPLAY_AWS_AUTH_HEADER_SECRET=""
 while [[ $# -gt 0 ]]; do
   case $1 in
     --skip-bootstrap)
@@ -65,21 +58,6 @@ while [[ $# -gt 0 ]]; do
       DESTROY_ALL_COPILOT=true
       shift # past argument
       ;;
-    --auth-header-value)
-      REPLAY_AUTH_HEADER="$2"
-      shift # past argument
-      shift # past value
-      ;;
-    --aws-auth-header-user)
-      REPLAY_AWS_AUTH_HEADER_USER="$2"
-      shift # past argument
-      shift # past value
-      ;;
-    --aws-auth-header-secret)
-      REPLAY_AWS_AUTH_HEADER_SECRET="$2"
-      shift # past argument
-      shift # past value
-      ;;
     -r|--region)
       REGION="$2"
       shift # past argument
@@ -103,16 +81,6 @@ while [[ $# -gt 0 ]]; do
   esac
 done
 
-if [[ -n "${REPLAY_AUTH_HEADER}" && ( -n "${REPLAY_AWS_AUTH_HEADER_USER}" || -n "${REPLAY_AWS_AUTH_HEADER_SECRET}" ) ]]; then
-    echo "[--auth-header-value] and [--aws-auth-header-user, --aws-auth-header-secret] are mutually exclusive sets of authorization header parameters"
-    exit 1
-fi
-
-if [[ ( -n "${REPLAY_AWS_AUTH_HEADER_USER}" || -n "${REPLAY_AWS_AUTH_HEADER_SECRET}" ) && (  -z "${REPLAY_AWS_AUTH_HEADER_USER}" || -z "${REPLAY_AWS_AUTH_HEADER_SECRET}" ) ]]; then
-    echo "[--aws-auth-header-user, --aws-auth-header-secret] must both be provided if specified"
-    exit 1
-fi
-
 COPILOT_DEPLOYMENT_STAGE=$STAGE
 export AWS_DEFAULT_REGION=$REGION
 export CDK_DEPLOYMENT_STAGE=$STAGE
@@ -164,30 +132,15 @@ fi
 # More details on the CDK used here can be found at opensearch-migrations/deployment/cdk/opensearch-service-migration/README.md
 cdk deploy "*" --c domainName="aos-domain" --c engineVersion="OS_1.3" --c  dataNodeCount=2 --c vpcEnabled=true --c availabilityZoneCount=2 --c openAccessPolicyEnabled=true --c domainRemovalPolicy="DESTROY" --c migrationAssistanceEnabled=true --c enableDemoAdmin=true -O cdkOutput.json --require-approval never --concurrency 3
 
-# Gather CDK output which includes export commands needed by Copilot, and make them available to the environment
-found_exports=$(grep -o "export [a-zA-Z0-9_]*=[^\\;\"]*" cdkOutput.json)
-eval "$(grep -o "export [a-zA-Z0-9_]*=[^\\;\"]*" cdkOutput.json)"
+# Collect export commands from CDK output, which are needed by Copilot, wrap the commands in double quotes and make them available to the environment
+found_exports=$(grep -o "export [a-zA-Z0-9_]*=[^\\;\"]*" cdkOutput.json | sed 's/=/="/' | sed 's/.*/&"/')
+eval "$(grep -o "export [a-zA-Z0-9_]*=[^\\;\"]*" cdkOutput.json | sed 's/=/="/' | sed 's/.*/&"/')"
 printf "The following exports were added from CDK:\n%s\n" "$found_exports"
 
 # Future enhancement needed here to make our Copilot deployment able to be reran without error even if no changes are deployed
 # === Copilot Deployment ===
 
-# Construct Replayer image docker command
-auth_header_args=""
-# Use arguments passed to script to override default auth header values
-if [ -n "${REPLAY_AUTH_HEADER}" ]; then
-  auth_header_args="--auth-header-value ${REPLAY_AUTH_HEADER}"
-elif [ -n "${REPLAY_AWS_AUTH_HEADER_USER}" ]; then
-  auth_header_args="--aws-auth-header-user ${REPLAY_AWS_AUTH_HEADER_USER} --aws-auth-header-secret ${REPLAY_AWS_AUTH_HEADER_SECRET}"
-# Use CDK environment variables for a target cluster user if detected
-elif [ -n "${MIGRATION_DOMAIN_USER_NAME}" ]; then
-  auth_header_args="--aws-auth-header-user ${MIGRATION_DOMAIN_USER_NAME} --aws-auth-header-secret ${MIGRATION_DOMAIN_USER_SECRET_ARN}"
-else
-  echo "No auth header options detected for Replayer, defaulting to not specifying an explicit auth header"
-fi
-replay_command_base="/bin/sh -c \"/runJavaWithClasspath.sh org.opensearch.migrations.replay.TrafficReplayer https://${MIGRATION_DOMAIN_ENDPOINT}:443 --insecure --kafka-traffic-brokers ${MIGRATION_KAFKA_BROKER_ENDPOINTS} --kafka-traffic-topic logging-traffic-topic --kafka-traffic-group-id default-logging-group --kafka-traffic-enable-msk-auth "
-replay_command_end=" | nc traffic-comparator 9220\""
-replay_command="${replay_command_base}${auth_header_args}${replay_command_end}"
+replay_command="/bin/sh -c \"/runJavaWithClasspath.sh org.opensearch.migrations.replay.TrafficReplayer https://${MIGRATION_DOMAIN_ENDPOINT}:443 --insecure --kafka-traffic-brokers ${MIGRATION_KAFKA_BROKER_ENDPOINTS} --kafka-traffic-topic logging-traffic-topic --kafka-traffic-group-id default-logging-group --kafka-traffic-enable-msk-auth --auth-header-user-and-secret ${MIGRATION_DOMAIN_USER_AND_SECRET_ARN} | nc traffic-comparator 9220\""
 echo "Constructed replay command: ${replay_command}"
 export MIGRATION_REPLAYER_COMMAND="${replay_command}"