Skip to content

Commit

Permalink
ci: measure and report CLI load time, and require subsecond
Browse files Browse the repository at this point in the history
Also, revert this commit:
 2023-09-13 acd724e Eric Joanis ci: suppress redundant tests runs on PR and push
It turns out doing CI on review_requested is not great, because:
 - if you request 2 or 3 reviews, it triggers the action 2 or 3 times
 - when the branch needs a real merge, we do want the result of the merge
   tested in CI, so we lost important validation
  • Loading branch information
joanise committed Jan 5, 2024
1 parent baffdd4 commit 17868c3
Showing 1 changed file with 28 additions and 3 deletions.
31 changes: 28 additions & 3 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
name: Run Tests
on:
pull_request:
types: [review_requested]
push:
- pull_request
- push

jobs:
test:
Expand Down Expand Up @@ -53,6 +52,32 @@ jobs:
token: ${{ secrets.CODECOV_TOKEN }} # optional but apparently makes upload more reliable
fail_ci_if_error: false # too many upload errors to keep "true"

- name: Make sure the CLI stays fast
id: cli-load-time
run: |
PYTHONPROFILEIMPORTTIME=1 readalongs -h 2> importtime.txt > /dev/null
CLI_LOAD_TIME="$((/usr/bin/time --format=%E readalongs -h > /dev/null) 2>&1)"
echo "CLI load time: $CLI_LOAD_TIME"
echo "CLI load time: $CLI_LOAD_TIME" > import-message.txt
echo "PR head ${{ github.event.pull_request.head.sha }}" >> import-message.txt
echo "Imports that take more than 0.1 s:" >> import-message.txt
grep -E 'cumulative|[0-9]{6} ' importtime.txt >> import-message.txt
cat import-message.txt
echo "Full import time log:"
cat importtime.txt
if [[ "$CLI_LOAD_TIME" > "0:01.00" ]]; then \
echo "ERROR: readalongs --help is too slow."; \
echo "Please run 'PYTHONPROFILEIMPORTTIME=1 readalongs -h 2> importtime.txt; tuna importtime.txt' and tuck away expensive imports so that the CLI doesn't load them until it uses them."; \
false; \
fi
- name: Report help speed in a PR comment
if: github.event_name == 'pull_request'
uses: mshick/add-pr-comment@v2
with:
preformatted: true
message-path: import-message.txt

test-on-windows:
runs-on: windows-latest
if: ${{ !contains(github.event.head_commit.message, '#no-ci') }}
Expand Down

0 comments on commit 17868c3

Please sign in to comment.