Skip to content

AWS CodeBuild CI

Jump to bottom
Steven Hahn edited this page Mar 3, 2023 · 8 revisions

Notes on automated testing setup with webhooks and AWS CodeBuild.

1) push containers to AWS Elastic Cloud Repository (ECR) to provide testing context

aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin 598751968580.dkr.ecr.us-east-2.amazonaws.com

  • define Dockerfile and build each image

note: I had to update docker to enable swap to build successfully

also: asgard-containers is my ECR repo name

from directory containing Dockerfile: docker build -t [image_name] .

  • tag image

docker tag [image_name]:latest 598751968580.dkr.ecr.us-east-2.amazonaws.com/asgard-containers:[image_name]

  • push to ECR

docker push 598751968580.dkr.ecr.us-east-2.amazonaws.com/asgard-containers:[image_name]

2) get access token from github

  • navigate to https://github.com/settings/tokens

  • click developer settings on left sidebar

  • click personal access tokens on left sidebar

  • create new personal access token with repo and repo hook scopes enabled

3) create build projects in codebuild

note: started from example at: https://docs.aws.amazon.com/codebuild/latest/userguide/sample-ecr.html

  • create build project: on CodeBuild landing page, click "Create Build Project".

under source section: Github, select repo in my github account radio. ensure report build status radio also selected.

under primary source webhook, select rebuild on change radio.

for our project, used PULL_REQUEST_CREATED or _CHANGED event types.

under start a build under conditions, add github user ids (not usernames, see https://api.github.com/users/[your_user_name]). this will prevent users from opening PRs/running builds that run up bills. may not need if repo access is carefully managed.

under environment, select custom image, and choose the ECR repo/Docker image you want to build. under additional config, select needed hardware.

under buildspec, choose insert build commands, and enter the commands you need to run, e.g.:

version: 0.2
run-as: mpiuser

phases:
  #install:
    #If you use the Ubuntu standard image 2.0 or later, you must specify runtime-versions.
    #If you specify runtime-versions and use an image other than Ubuntu standard image 2.0, the build fails.
    #runtime-versions:
      # name: version
      # name: version
    #commands:
      # - command
      # - command
  #pre_build:
    #commands:
      # - command
      # - command
  build:
    run-as: mpiuser
    commands:
      - mkdir build && cd build
      - cmake ../ -DCMAKE_CXX_COMPILER=g++ -DASGARD_USE_MPI=1
      - make -j 8
      - ctest
      # - command
      # - command
  #post_build:
    #commands:
      # - command
      # - command
#reports:
  #report-name-or-arn:
    #files:
      # - location
      # - location
    #base-directory: location
    #discard-paths: yes
    #file-format: JunitXml | CucumberJson
#artifacts:
  #files:
    # - location
    # - location
  #name: $(date +%Y-%m-%d)
  #discard-paths: yes
  #base-directory: location
#cache:
  #paths:
    # - paths

you can click on webhooks to view the payload/response for triggering events, helpful for debugging.

clicking edit will allow you to change triggering events - at least pull request and push should be enabled.

click the checkboxes for any builds you want to require before merge.

How to add or modify automated tests

editing a build project

From CodeBuild landing page, you can click on an existing build project name (e.g., asgard-unit-gxx) to edit its configuration or view its history.

The edit button on this page has several options, but the most important are:

  • Configuration - allows for naming the project. also, checkbox "enable build badges" must be enabled to add as CI badge in Github.

  • Source - where AWS will fetch code from (Github repository). an access badge has been generated from Github and added to CodeBuild for authentication. also defines the triggering events for the build project. For us, these events are PULL_REQUEST_CREATED and _UPDATED, and merge into develop branch. This section also contains whitelist rules that disable all builds unless the developer's Github userid is listed in the rules.

  • Environment - test context, as defined by a container in Elastic Cloud Registry (ECR). adding packages/dependencies requires updating container files in ECR (accessible from the management console.

  • Buildspec - this is where test actions to run after the environment has been deployed are scripted.

  • Logs - option to output logs to CloudWatch should be checked so that build logs are stored.

adding a build project

You can click on the orange "Create Build Project" button on the CodeBuild landing page to define a new project.

See (https://github.com/project-asgard/asgard/wiki/AWS-CodeBuild-CI#3-create-build-projects-in-codebuild) above for notes on how to configure a new project. If the build project requires additional dependencies, our ECR containers will need to be updated. there are notes for how to build, tag, and push an image above (https://github.com/project-asgard/asgard/wiki/AWS-CodeBuild-CI#1-push-containers-to-aws-elastic-cloud-repository-ecr-to-provide-testing-context). please open an issue if additional information is needed.

Clone this wiki locally