Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Release-1.28] - S3 snapshots missing from etcdsnapshotfile list / k3s-etcd-snapshots configmap when multiple nodes save at the same time #11053

Closed
brandond opened this issue Oct 9, 2024 · 1 comment
Closed

[Release-1.28] - S3 snapshots missing from etcdsnapshotfile list / k3s-etcd-snapshots configmap when multiple nodes save at the same time #11053

brandond opened this issue Oct 9, 2024 · 1 comment
Assignees
@brandond @VestigeJ
Labels
waiting-for-RC Issue is available to test only after we have an RC
Milestone
v1.28.15+k3s1

Comments

@brandond
Copy link
Member

brandond commented Oct 9, 2024

Backport fix for S3 snapshots missing from etcdsnapshotfile list / k3s-etcd-snapshots configmap when multiple nodes save at the same time
@brandond brandond self-assigned this Oct 9, 2024
@brandond brandond mentioned this issue Oct 10, 2024
Merged
@brandond brandond moved this from New to To Test in K3s Development Oct 10, 2024
@brandond brandond added this to the v1.28.15+k3s1 milestone Oct 10, 2024
@aganesh-suse aganesh-suse added the waiting-for-RC Issue is available to test only after we have an RC label Oct 24, 2024
@VestigeJ
Copy link

Previous behavior established here - rancher/rke2#7027 (comment)

Validated against v1.28.15-rc1+k3s1

$ sudo kubectl get cm -n kube-system k3s-etcd-snapshots

NAME                 DATA   AGE
k3s-etcd-snapshots   6      19m

$ sudo k3s etcd-snapshot ls

Name                                                 Location                                                                                             Size     Created
etcd-snapshot-v12815rc1-pool1-bw8gq-lcs4g-1729886400 s3://justin-snapshot-buckets/v12815rc1/etcd-snapshot-v12815rc1-pool1-bw8gq-lcs4g-1729886400          10330144 2024-10-25T20:00:00Z
etcd-snapshot-v12815rc1-pool1-bw8gq-lcs4g-1729886400 file:///var/lib/rancher/k3s/server/db/snapshots/etcd-snapshot-v12815rc1-pool1-bw8gq-lcs4g-1729886400 10330144 2024-10-25T20:00:00Z
etcd-snapshot-v12815rc1-pool1-bw8gq-fjmq9-1729886401 s3://justin-snapshot-buckets/v12815rc1/etcd-snapshot-v12815rc1-pool1-bw8gq-fjmq9-1729886401          10346528 2024-10-25T20:00:01Z
etcd-snapshot-v12815rc1-pool1-bw8gq-4bpt7-1729886403 s3://justin-snapshot-buckets/v12815rc1/etcd-snapshot-v12815rc1-pool1-bw8gq-4bpt7-1729886403          10453024 2024-10-25T20:00:03Z

$ sudo kubectl get etcdsnapshotfile

NAME                                                                SNAPSHOTNAME                                           NODE                          LOCATION                                                                                               SIZE       CREATIONTIME
local-etcd-snapshot-v12815rc1-pool1-bw8gq-4bpt7-1729886403-e9d6fd   etcd-snapshot-v12815rc1-pool1-bw8gq-4bpt7-1729886403   v12815rc1-pool1-bw8gq-4bpt7   file:///var/lib/rancher/k3s/server/db/snapshots/etcd-snapshot-v12815rc1-pool1-bw8gq-4bpt7-1729886403   10453024   2024-10-25T20:00:03Z
local-etcd-snapshot-v12815rc1-pool1-bw8gq-fjmq9-1729886401-a5d696   etcd-snapshot-v12815rc1-pool1-bw8gq-fjmq9-1729886401   v12815rc1-pool1-bw8gq-fjmq9   file:///var/lib/rancher/k3s/server/db/snapshots/etcd-snapshot-v12815rc1-pool1-bw8gq-fjmq9-1729886401   10346528   2024-10-25T20:00:01Z
local-etcd-snapshot-v12815rc1-pool1-bw8gq-lcs4g-1729886400-c98117   etcd-snapshot-v12815rc1-pool1-bw8gq-lcs4g-1729886400   v12815rc1-pool1-bw8gq-lcs4g   file:///var/lib/rancher/k3s/server/db/snapshots/etcd-snapshot-v12815rc1-pool1-bw8gq-lcs4g-1729886400   10330144   2024-10-25T20:00:00Z
s3-etcd-snapshot-v12815rc1-pool1-bw8gq-4bpt7-1729886403-3639e4      etcd-snapshot-v12815rc1-pool1-bw8gq-4bpt7-1729886403   s3                            s3://justin-snapshot-buckets/v12815rc1/etcd-snapshot-v12815rc1-pool1-bw8gq-4bpt7-1729886403            10453024   2024-10-25T20:00:03Z
s3-etcd-snapshot-v12815rc1-pool1-bw8gq-fjmq9-1729886401-131bdb      etcd-snapshot-v12815rc1-pool1-bw8gq-fjmq9-1729886401   s3                            s3://justin-snapshot-buckets/v12815rc1/etcd-snapshot-v12815rc1-pool1-bw8gq-fjmq9-1729886401            10346528   2024-10-25T20:00:01Z
s3-etcd-snapshot-v12815rc1-pool1-bw8gq-lcs4g-1729886400-800900      etcd-snapshot-v12815rc1-pool1-bw8gq-lcs4g-1729886400   s3                            s3://justin-snapshot-buckets/v12815rc1/etcd-snapshot-v12815rc1-pool1-bw8gq-lcs4g-1729886400            10330144   2024-10-25T20:00:00Z

Screenshot 2024-10-25 at 1 26 42 PM

@github-project-automation github-project-automation bot moved this from To Test to Done Issue in K3s Development Oct 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@brandond brandond

@VestigeJ VestigeJ

Labels
waiting-for-RC Issue is available to test only after we have an RC
Projects
K3s Development
Status: Done Issue
Milestone
v1.28.15+k3s1
Development

No branches or pull requests
3 participants
@brandond @VestigeJ @aganesh-suse