Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Garbage Collector on azure a little too aggresive #4445

Closed
smerle33 opened this issue Dec 10, 2024 · 1 comment
Closed

Garbage Collector on azure a little too aggresive #4445

smerle33 opened this issue Dec 10, 2024 · 1 comment
Assignees
@smerle33
Labels
azure packer
Milestone
infra-team-sync-2...

Comments

@smerle33
Copy link
Contributor

smerle33 commented Dec 10, 2024 •
edited by dduportal
Loading

By checking this PR by @slide: jenkins-infra/packer-images#1585,
I found out that 2 VMs have been removed by the garbage collector while still being used:
pkrvmpjlaxxlp3c and pkrvmfxg5ljcler by this run https://infra.ci.jenkins.io/job/infra-tools/job/packer-images-gc/job/main/575/pipeline-console/log?nodeId=39

[2024-12-09T21:29:34.969Z] command executed: 
[2024-12-09T21:29:34.969Z] az resource delete --ids /subscriptions/****/resourceGroups/dev-packer-builds/providers/Microsoft.Compute/virtualMachines/pkrvmfxg5ljcler --resource-group dev-packer-builds
[2024-12-09T21:30:42.621Z] command executed: 
[2024-12-09T21:30:42.621Z] az resource delete --ids /subscriptions/****/resourceGroups/dev-packer-builds/providers/Microsoft.Compute/virtualMachines/pkrvmpjlaxxlp3c --resource-group dev-packer-builds

we need to provide either more delay or a check on opened pullrequests using those VM before garbaging them
@smerle33 smerle33 added triage Incoming issues that need review azure packer labels Dec 10, 2024
@dduportal dduportal mentioned this issue Dec 10, 2024
Merged
@dduportal dduportal added this to the infra-team-sync-2024-12-17 milestone Dec 10, 2024
@dduportal dduportal removed the triage Incoming issues that need review label Dec 10, 2024
@smerle33 smerle33 self-assigned this Dec 11, 2024
smerle33 added a commit to jenkins-infra/packer-images that referenced this issue Dec 11, 2024
@smerle33
chore(GC/azure) add debug
b474a6b 
ref: jenkins-infra/helpdesk#4445

adding some debug temporarily
This was referenced Dec 11, 2024
Merged
Merged
@dduportal
Copy link
Contributor

Fixed by jenkins-infra/packer-images#1596

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@smerle33 smerle33

Labels
azure packer
Projects
None yet
Milestone
infra-team-sync-2024-12-17
Development

No branches or pull requests
2 participants
@dduportal @smerle33