Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

COSI-53: Optimize CI Workflows, Mono-Repo Migration, and Enhancements #82

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

anurag4DSB
Copy link
Collaborator

@anurag4DSB anurag4DSB commented Jan 2, 2025

This PR introduces multiple improvements to CI workflows and project organization:

Commit 1. Parallelized CI Steps: Optimized workflow execution by parallelizing steps and adding if: always where necessary.
Commit 2. Renamed Workflows files and updated CI workflow names for clarity:

  • ci-build-and-unit-tests.yml → build-and-unit-tests.yml
  • ci-e2e-tests.yml → e2e-feature-tests.yml
  • ci-smoke-tests.yml → helm-validation.yml]

Commit 3. CNCF Mono-Repo Migration: use mono-repo as others have been archived; updated workflows and scripts.
Commit 4. Improved Coverage: Set GOCOVERDIR to reduce Codecov failures.
Commit 5. Enhanced Logging: Updated commands to improve output visibility in GitHub UI and logs.

@anurag4DSB anurag4DSB changed the title COSI-53: reduce-ci-time-parallelize-docker-ops COSI-53: CI Improvements - Reduce CI time, Jan 2, 2025
Copy link

codecov bot commented Jan 2, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.40%. Comparing base (a9cf51d) to head (9c4e7f0).

Additional details and impacted files

Impacted file tree graph

Components Coverage Δ
🏠 Main Package ∅ <ø> (∅)
🚗 Driver Package 92.22% <ø> (ø)
📡 gRPC Factory Package 81.65% <ø> (ø)
🔐 IAM Client Package 100.00% <ø> (ø)
🌐 S3 Client Package 100.00% <ø> (ø)
🔧 Util Package 100.00% <ø> (ø)
🔖 Constants Package ∅ <ø> (∅)
@@           Coverage Diff           @@
##             main      #82   +/-   ##
=======================================
  Coverage   93.40%   93.40%           
=======================================
  Files           9        9           
  Lines         637      637           
=======================================
  Hits          595      595           
  Misses         36       36           
  Partials        6        6           

@anurag4DSB anurag4DSB force-pushed the improvement/COSI-53-misc-ci-improvements branch from 1e7c78a to 0bbfed6 Compare January 2, 2025 09:35
This commit parallelizes two steps in the CI workflow,
adds if: always where needed, and includes minor lint changes.
@anurag4DSB anurag4DSB force-pushed the improvement/COSI-53-misc-ci-improvements branch from 0bbfed6 to 4242967 Compare January 2, 2025 09:39
renamed in .github/workflows:
- ci-build-and-unit-tests.yml -> build-and-unit-tests.yml
- ci-e2e-tests.yml -> e2e-feature-tests.yml
- ci-smoke-tests.yml -> helm-validation.yml
- The old repos have been archived and everything has been moved to a
  mono-repo
- Updated CI workflows and scripts to reflect the same
@anurag4DSB anurag4DSB force-pushed the improvement/COSI-53-misc-ci-improvements branch from 1adf835 to 31d4874 Compare January 2, 2025 10:04
@anurag4DSB anurag4DSB force-pushed the improvement/COSI-53-misc-ci-improvements branch from 31d4874 to 9c4e7f0 Compare January 2, 2025 10:06
@anurag4DSB anurag4DSB marked this pull request as ready for review January 2, 2025 10:14
@anurag4DSB anurag4DSB changed the title COSI-53: CI Improvements - Reduce CI time, COSI-53: Optimize CI Workflows, Mono-Repo Migration, and Enhancements Jan 2, 2025
echo "=== Loading cached S3 and IAM Docker images ==="
if [ -d /tmp/.docker_cache ] && [ "$(ls -A /tmp/.docker_cache 2>/dev/null)" ]; then
for image in /tmp/.docker_cache/*.tar; do
docker load -i "$image" || true # continue on failure

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't there an action that can load from the cache or save images to cache ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there is, but I can't use them in a script when doing things in parallel.
Ideally, we would want to use the action, but GitHub doesn't allow parallel actions on the same machine yet.
What do you think?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh that's unlucky

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants