Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store build results and discard old build records #4

Merged
merged 1 commit into from
Mar 20, 2024

Conversation

mkaapu
Copy link
Contributor

@mkaapu mkaapu commented Mar 15, 2024

Archives the build results as post step of pipeline.
Discard old builds and artifacts:

  • build age: no older than 70 days
  • build count: maximum 100 builds
  • artifact age: no older than 7 days
  • artifact count: maximum 10 builds

@mkaapu mkaapu requested a review from henrirosten March 15, 2024 16:14
Copy link

@mnokka mnokka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

Copy link
Collaborator

@henrirosten henrirosten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the proposed configuration would not work due to:

archiveArtifacts step copies the build images to /var/lib/jenkins on the jenkins-controller VM. In the current ghaf-infra configuration, /var/lib/jenkins is a 10GiB jenkins state disk (configured here and mounted here) so as it is now, there is simply no disk space to store the ghaf images (artifacts).

If we want to distribute the artifacts the way proposed here, how much disk space would we need?
One build generates roughly 100GiB of build artifacts, so storing build results from 10 builds would need around 1000GiB of disk space. You could reduce the disk usage by compressing the stored images.

Short term, if we must, we can have PoCs like this for the artifact distribution. Longer term, I believe we should think of better ways of doing this.

@mkaapu
Copy link
Contributor Author

mkaapu commented Mar 20, 2024

Short term, if we must, we can have PoCs like this for the artifact distribution. Longer term, I believe we should think of better ways of doing this.

As a workaround, jenkins home/state disk size was increased with
tiiuae/ghaf-infra#103

@mkaapu mkaapu merged commit bdc7474 into tiiuae:main Mar 20, 2024
@mkaapu mkaapu deleted the pr-archive-artifacts branch March 20, 2024 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants