Community Signup Serverless Service with Lambda, Python, StepFunction and DynamoDB

Blog post: Exploring Serverless with Python, StepFunctions, and Web Front-end Serverless

This time, with Serverless framework

This is a 'take two' on Slack Signup with AWS Lambda. It was simple and functional, but working with AWS raw is such a pain in the butt...

Here, I use 1) serverless.com to make working with lambda enjoyable and 2) AWS step-functions for multi-step sign-up workflow that touches multiple services. In this particular example, the workflow does what StackStorm signtup needs:

• call Slack API to create an invitation with Slack's undocumented API
• add a user to ActiveCampaign CRM
• record a user in DynamoDB

It is easy to modify to add your custom steps, have fun!

Install, configure, deploy, test

1. Clone this repository.

2. Install serverless framework, and the required plugins:

   npm install -g serverless
   npm install
   

   I use serverless-stepfunction plugin to write stepfunction as YAML in serverless.yaml and expose it as an API endpoint, and serverless-apig-s3 for convenient front-end deployment to the same URI (no CORS messing).

3. Configure credentials

   1. For AWS credentials, follow setup docs. I prefer using [AWS CLI with named profiles](http://docs.aws.amazon.com/cli/latest/userguide/ cli-multiple-profiles.html). To use an AWS profile, export AWS_DEFAULT_PROFILE="profileName". Test AWS CLI settings: aws lambda list-functions.

      To use an AWS profile in Serverless, export AWS_PROFILE=$AWS_DEFAULT_PROFILE, to match aws CLI profile and avoid confusion. You MUST also specify the AWS region in AWS_REGION for serverless, as it won't take it from AWS profile. So, three env vars in total:

      export AWS_DEFAULT_PROFILE="profileName"
      export AWS_PROFILE=$AWS_DEFAULT_PROFILE
      export AWS_REGION=us-west-2
      

      NOTE: Till the bug #3947 fixed,the way to set a region is by --region in sls CLI.

      Or use --aws-profile profileName when invoking sls CLI. Unfortunately there's no way to test it before you try to deploy.

      Create ./private.yml file and set up the accountID and region there, see private.yml.example. Note that region must match

   2. Slack and ActiveCampaign: place your credentials and config in ./env.yml, use env.yml.example. To find your Slack credentials for production or testing, use this hint from Stackstorm-Exchange.

4. Deploy

   1. Install Python dependencies:

      ./build.sh
      

   2. Deploy serverless stack

      sls deploy -v
      

   3. Deploy web client

      sls client deploy
      

5. Run, with curl:

   curl -X POST -H "Content-Type: application/json" -d '{"email":"[email protected]", "first_name":"Donald", "last_name":"Trump"}'  https://wqftmz3m97.execute-api.us-east-1.amazonaws.com/dev/signup
   

   Or, with serverless, for convenience:

   sls invoke stepf --name signup --data '{"email":"[email protected]", "first_name":"Donald", "last_name":"Trump"}' 
   

Unit testing and local development

1. Run tox (dah, tox and all your Python stack needs to be installed)

   virtualenv .venv
   source .venv/bin/activate
   pip install -r tests/test_requirements.txt
   

2. To run unit tests for local testing/development:

   # Activate virtualenv from tox
   source ./tox/py27/bin/activate
   # Run all tests
   python -m pytest
   # Run a single test and print the output ( -s flag ) 
   python -m pytest -s tests/invite_slack_test.py::InviteSlackTest::test_handler_ok 
   

Tips and tricks

• Call individual lambda functions with serverless

  sls invoke -f RecordAC --data '{"email":"[email protected]", "first_name":"Donald", "last_name":"Trump"}' 
  

• Watch the logs:

  sls logs -f RecordAC
  

• On subsequent web client updates, just run sls client deploy. However when changes touch API Gateway settings, the full deploy is needed.

• Web updates doesn't [always, ever] deploy the API Gateway changes. Need to deploy manually (Console->Resources->Actions-Deploy API).

• Due to apig-s3 bug, remove the web s3 bucket manually when removing a stack. aws s3 rb s3://bucketname --force

• Removing the stack does not remove the DynamoDB table. If you really want to start from scratch, delete it manually (export the data first): aws dynamodb delete-table --table-name slack-signup-dev