chore: Migrate CI/CD to GitHub Actions (#11638)

* feat: Actions for pr testing (#12)

* feat: Add working e2e example (#14)

* chore: Add configurable e2e tests to GH workflows (#15)

* feat(e2e): add integ test runner and test

* fix(e2e): retain existing workflows

* fix(e2e): remove old e2e example

---------

Co-authored-by: Sridhar <[email protected]>

* chore: add e2e browser auth tests (#16)

* chore: add e2e browser auth tests

* chore: code cleanup

---------

Co-authored-by: Sridhar <[email protected]>

* chore: add e2e browser predictions interactions tests (#17)

* chore: add e2e integ_react_predictions

* chore: add browser e2e interactions

---------

Co-authored-by: Sridhar <[email protected]>
Co-authored-by: Aaron S <[email protected]>

* chore: limit PR concurrency (#19)

* chore: add build_type and enable retry (#18)

* prod & dev

* -n 3

* turn off fail-fast

---------

Co-authored-by: Aaron S <[email protected]>

* chore: add e2e browser datastore tests (#20)

Co-authored-by: Sridhar <[email protected]>

* chore: add e2e browser api,storage tests (#21)

Co-authored-by: Sridhar <[email protected]>

* chore: fix single browser provided in integ-config (#22)

Co-authored-by: Sridhar <[email protected]>

* chore: Run dev and prod on the same runner

* chore: Add detox testing (#26)

* Feat/GitHub actions/test ashwinkumar6 (#27)

* chore: Re-arrange integ test configurations and improve prebuild (#29)

* chore: Re-arrange integ test configurations and inprove preflight

* Fix is-preflight flag

* Change preflight to prebuild for consistencu

* Disable fast fail on unit tests

* feat: Change yaml load output pattern

* chore: fix e2e integ_datastore_auth_v2 test

* chore: disable playwright tests (#33)

Co-authored-by: Sridhar <[email protected]>

* chore: add google env variables for e2e datastore tests (#35)

Co-authored-by: Sridhar <[email protected]>

* chore: add support for uploading artifacts  (#34)

* chore: test upload artifacts

* chore: test uploading artifact

* chore: Detox fix and step cleanup

* chore: Increase detox time limit

* chore: Turn warn to ignore for upload artifact misses

* chore: Disable broken detox tests

* chore: Disable PN testing and remove datastore test not running in circle

* chore: e2e add configurable time_out and retry_count (#36)

Co-authored-by: Sridhar <[email protected]>

* chore: Increase deviceTracking timeout

* chore: add deploy workflow (#37)

* chore: add deploy workflow

* chore: gh actions publish code clean up

---------

Co-authored-by: Sridhar <[email protected]>

* chore: Model full CI/CD flow - improve preid release (#38)

* fix: Cache on the single yarn.lock instead of all

* fix: Sync with main

* chore: Revert to relative path use (#40)

* chore: Revert to relative path use

* fix: Remove path from workflows

* chore: Add post-release merge to main

* chore: Remove orphaned workflow files

* chore: e2e add integ_auth_test_cypress_no_ui test (#39)

* chore: e2e add integ_auth_test_cypress_no_ui test

* chore: code cleanup

---------

Co-authored-by: Sridhar <[email protected]>

* chore: change github name and email

* chore: update yarn.log

* chore: move additional e2e commented tests (#43)

* chore: move additional e2e commented tests

* chore: add required permissions to script file

---------

Co-authored-by: Sridhar <[email protected]>

* chore: e2e keep test_name same as CCI (#45)

* chore: e2e keep test_name same as CCI
* chore: e2e re-arrange integ config
---------
Co-authored-by: Sridhar <[email protected]>

* chore: update actions typo

Co-authored-by: William Lee <[email protected]>

* chore: update actions typo

Co-authored-by: William Lee <[email protected]>

* chore: cleanup github_email and github_name (#46)

Co-authored-by: Sridhar <[email protected]>

* chore: cleanup load verdaccio action (#47)

Co-authored-by: Sridhar <[email protected]>

* chore: actions code cleanup (#48)

Co-authored-by: Sridhar <[email protected]>

* chore: Remove unused actions and use env vars instead of injection in run commands (#49)

* chore: add retention period and timeout for cache (#50)

* chore: add retention period and timeout for cache

* chore: code cleanup

---------

Co-authored-by: Sridhar <[email protected]>

* chore: Chanke js token name

* chore: Filter preid content

* chore: Move file names around and consume env vars

* fix: Rest bundle size increase

* chore: Compactify on syntax

* chore: dynamically fetch the repo_owner_name (#52)

* chore: dynamically fetch the repo_owner_name

* chore: code cleanup

---------

Co-authored-by: Sridhar <[email protected]>

* chore: Configure staging to use corresponding branch name

* chore: Naming and cleanup

* chore: Naming and dependency workflow improvements

* chore: Fix vars use for gh user/email

* chore: run e2e Rollup + DataStore test on just chrome (#54)

Co-authored-by: Sridhar <[email protected]>

* chore: Add preid protection (#57)

* chore: Only publish to npm from the aws-amplify org

* chore: add concurrency to push workflows (#56)

* chore: add concurrency to push workflows

* chore: code cleanup

---------

Co-authored-by: Sridhar <[email protected]>

* chore: Prepare for merge to main

* chore: Fix circle build

* chore: Okay, this will fix circle!

* chore: Integrate license checking into pr check

* chore: Add github action static tests

* chore: Fix test command

* Update .github/actions/npm-publish/action.yml

Co-authored-by: Jim Blanchard <[email protected]>

* chore: Fix names and descriptions

---------

Co-authored-by: Ashwin Kumar <[email protected]>
Co-authored-by: Sridhar <[email protected]>
Co-authored-by: Lucas Yingshyan Ku <[email protected]>
Co-authored-by: William Lee <[email protected]>
Co-authored-by: Jim Blanchard <[email protected]>

.circleci/config.yml

@@ -364,21 +364,10 @@ jobs:
       - checkout
       - run:
           name: yarn install --non-interactive
-          # temporarily rename .yarnrc before installing so that we can generate a yarn.lock artifact
           command: |
-            mv .yarnrc ._yarnrc
             yarn
       - run: yarn bootstrap
       - run: yarn build
-      # storing yarn.lock as an artifact, so that we can quickly get a dependency diff
-      # with the last working build in the event that some upstream deps break the build in the future
-      - store_artifacts:
-          path: yarn.lock
-      - run:
-          name: delete lockfile after persisting
-          command: |
-            mv ._yarnrc .yarnrc
-            rm yarn.lock
       - save_cache:
           key: amplify-ssh-deps-{{ .Branch }}
           paths:

.github/actions/load-verdaccio-with-amplify-js/action.yml

@@ -0,0 +1,50 @@
+name: Load Verdaccio with AmplifyJs
+description: Turn on Verdaccio and load up all of the AmplifyJS build artifacts
+
+runs:
+  using: 'composite'
+  steps:
+    - name: Start verdaccio
+      run: |
+        npx [email protected] &
+        while ! nc -z localhost 4873; do
+          echo "Verdaccio not running yet"
+          sleep 1
+        done
+
+        # Run your commands after verdaccio is up and running
+        echo "Verdaccio is up and running, proceeding with the script..."
+      shell: bash
+    - name: Install and run npm-cli-login
+      shell: bash
+      env:
+        NPM_REGISTRY: http://localhost:4873/
+        NPM_USER: verdaccio
+        NPM_PASS: verdaccio
+        NPM_EMAIL: [email protected]
+      run: |
+        npm i -g npm-cli-adduser
+        npm-cli-adduser
+        sleep 1
+    - name: Configure registry and git
+      shell: bash
+      working-directory: ./amplify-js
+      env:
+        NPM_REGISTRY: http://localhost:4873/
+        NPM_USER: verdaccio
+        NPM_PASS: verdaccio
+        NPM_EMAIL: [email protected]
+      run: |
+        yarn config set registry $NPM_REGISTRY
+        npm set registry $NPM_REGISTRY
+        git config --global user.email $NPM_EMAIL
+        git config --global user.name $NPM_USER
+        git status
+        git --no-pager diff
+    - name: Publish to verdaccio
+      shell: bash
+      working-directory: ./amplify-js
+      run: |
+        scripts/retry-yarn-script.sh -s publish:verdaccio -n 5 -r true
+        yarn info aws-amplify@unstable description
+        npm info aws-amplify@unstable version

.github/actions/node-and-build/action.yml

@@ -0,0 +1,48 @@
+name: Set Node and Build
+description: Checks out Amplify and builds the package
+inputs:
+  is-prebuild:
+    required: false
+    default: false
+runs:
+  using: 'composite'
+  steps:
+    - name: Setup Node.js 16
+      uses: actions/setup-node@64ed1c7eab4cce3362f8c340dee64e5eaeef8f7c # v3.6.0 https://github.com/actions/setup-node/commit/64ed1c7eab4cce3362f8c340dee64e5eaeef8f7c
+      with:
+        node-version: 16
+      env:
+        SEGMENT_DOWNLOAD_TIMEOUT_MINS: 2
+    - uses: actions/cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 # v3.3.1
+      id: cache-build-artifacts
+      with:
+        path: |
+          **/node_modules
+          **/dist
+          **/lib
+          **/lib-esm/
+          **/es/
+          **/esm/
+          **/cjs/
+          **/packages/core/src/Platform/version.ts
+        key: ${{ runner.os }}-build-artifacts-${{ hashFiles('./amplify-js/yarn.lock') }}
+        restore-keys: |
+          ${{ runner.os }}-build-artifacts-
+      env:
+        SEGMENT_DOWNLOAD_TIMEOUT_MINS: 2
+    # TODO We should be able to skip yarn / bootstrap if we cache enough things. Leaving because skipping causes issues.
+    - name: Install
+      if: inputs.is-prebuild != 'true' || steps.cache-build-artifacts.outputs.cache-hit != 'true'
+      run: yarn
+      shell: bash
+      working-directory: ./amplify-js
+    - name: Bootstrap
+      if: inputs.is-prebuild != 'true' || steps.cache-build-artifacts.outputs.cache-hit != 'true'
+      run: yarn bootstrap
+      shell: bash
+      working-directory: ./amplify-js
+    - name: Build packages
+      if: steps.cache-build-artifacts.outputs.cache-hit != 'true'
+      run: yarn build
+      shell: bash
+      working-directory: ./amplify-js

.github/actions/npm-publish/action.yml

@@ -0,0 +1,47 @@
+name: Publish to NPM
+description: Publish changes from the current commit to npm using the target release process.
+
+inputs:
+  github_user:
+    description: The git user to make commits with
+    required: true
+  github_email:
+    description: The git email to make commits with
+    required: true
+  target:
+    description: The release process target (either release or preid)
+    required: true
+  preid:
+    description: The preid to release to when the target is preid. The preid process will fail if this isn't provided.
+    required: false
+  npm_token:
+    description: The npm token that gives the job npm permissions
+    required: true
+
+runs:
+  using: 'composite'
+  steps:
+    - name: Set github commit user
+      shell: bash
+      env:
+        GITHUB_EMAIL: ${{ inputs.github_email }}
+        GITHUB_USER: ${{ inputs.github_user }}
+      run: |
+        git config --global user.email $GITHUB_EMAIL
+        git config --global user.name $GITHUB_USER
+
+    - name: Run npm publish
+      if: github.repository_owner == 'aws-amplify'
+      shell: bash
+      working-directory: ./amplify-js
+      env:
+        NPM_TOKEN: ${{ inputs.npm_token }}
+        GH_SHA: ${{ github.sha }}
+        TARGET: ${{ inputs.target }}
+        PREID_PREFIX: ${{ inputs.preid || '' }}
+      run: |
+        echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" >> ~/.npmrc
+        export PREID_HASH_SUFFIX=$(echo $GH_SHA | cut -c -7 | sed 's/^/\./')
+        git status
+        git --no-pager diff
+        yarn publish:$TARGET

.github/actions/setup-samples-staging/action.yml

@@ -0,0 +1,51 @@
+name: Setup Amplify Integration Test Package
+description: Checks out "amplify-js-samples-staging" and builds the package with caching
+
+inputs:
+  GH_TOKEN_STAGING_READ:
+    description: The token that grants read access to the sample staging repo
+    required: true
+
+runs:
+  using: 'composite'
+  steps:
+    - name: Create cache
+      uses: actions/cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 # v3.3.1 https://github.com/actions/cache/commit/88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8
+      id: cache-samples-staging-build
+      with:
+        key: aws-amplify-js-samples-staging-build-${{ github.sha }}
+        path: |
+          ./amplify-js-samples-staging/node_modules
+          /home/runner/.cache/Cypress
+      env:
+        SEGMENT_DOWNLOAD_TIMEOUT_MINS: 2
+    - name: Checkout staging repo
+      uses: actions/checkout@24cb9080177205b6e8c946b17badbe402adc938f # v3.4.0 https://github.com/actions/checkout/commit/24cb9080177205b6e8c946b17badbe402adc938f
+      with:
+        repository: ${{ github.repository_owner }}/amplify-js-samples-staging
+        path: amplify-js-samples-staging
+        token: ${{ inputs.GH_TOKEN_STAGING_READ }}
+
+    # We test on the staging branch that corresponds to the amplify-js branch
+    #   when it exists and test on the default branch `main` otherwise
+    - name: Staging repo branch selection
+      shell: bash
+      working-directory: ./amplify-js-samples-staging
+      env:
+        BRANCH: ${{ github.ref_name }}
+      run: |
+        if git ls-remote --exit-code --heads origin $BRANCH >/dev/null 2>&1; then
+            # Branch exists, checkout and echo success message
+            git fetch origin $BRANCH
+            git checkout $BRANCH
+            echo "Checked out branch: $BRANCH"
+        else
+            # Branch doesn't exist, echo error message
+            echo "Branch '$BRANCH' does not exist"
+        fi
+
+    - name: Install
+      run: |
+        yarn
+      shell: bash
+      working-directory: ./amplify-js-samples-staging