Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data upload is been canceled #8488

Open
SCLogo opened this issue Dec 5, 2024 · 7 comments
Open

Data upload is been canceled #8488

SCLogo opened this issue Dec 5, 2024 · 7 comments
Assignees

Comments

@SCLogo
Copy link

SCLogo commented Dec 5, 2024

What steps did you take and what happened:
We have velero 1.15.0 with kopia. Data mover enabled. Back is aws. Sometimes I see dataupload is been cancelled

apiVersion: velero.io/v2alpha1
kind: DataUpload
metadata:
  creationTimestamp: "2024-12-05T06:09:05Z"
  generateName: velero-hourly-20241205060051-
  generation: 4
  labels:
    velero.io/accepted-by: i-048b5b25b16cae984
    velero.io/async-operation-id: du-3fe2dd8f-13e4-48e0-86c9-69d3dfaad582.13499861-b33e-4c339bd32
    velero.io/backup-name: velero-hourly-20241205060051
    velero.io/backup-uid: 3fe2dd8f-13e4-48e0-86c9-69d3dfaad582
    velero.io/pvc-uid: 13499861-b33e-4c38-ac84-f3a1f0368cb1
  name: velero-hourly-20241205060051-x6nd7
  namespace: velero
  ownerReferences:
  - apiVersion: velero.io/v1
    controller: true
    kind: Backup
    name: velero-hourly-20241205060051
    uid: 3fe2dd8f-13e4-48e0-86c9-69d3dfaad582
  resourceVersion: "373513216"
  uid: 223ae86b-c492-409e-89b5-71cddb65091b
spec:
  backupStorageLocation: default
  cancel: true
  csiSnapshot:
    snapshotClass: backup
    storageClass: storageclass
    volumeSnapshot: pod_to_be_backed
  dataMoverConfig:
    ParallelFilesUpload: "10"
  datamover: velero
  operationTimeout: 10m0s
  snapshotType: CSI
  sourceNamespace: test
  sourcePVC: sourcens
  status:
  completionTimestamp: "2024-12-05T06:09:25Z"
  message: Dataupload is in Accepted status during the node-agent starting, mark it
    as cancel
  phase: Canceled
  progress: {}
  startTimestamp: "2024-12-05T06:09:25Z"

What did you expect to happen:
able to do a backup without issues
The following information will help us better understand what's going on:

If you are using velero v1.7.0+:
Please use velero debug --backup <backupname> --restore <restorename> to generate the support bundle, and attach to this issue, more options please refer to velero debug --help

Environment:

  • Velero version (use velero version): 1.15.0
  • Velero features (use velero client config get features):
  • Kubernetes version (use kubectl version):
Client Version: v1.31.1
Kustomize Version: v5.4.2
Server Version: v1.30.4
  • Kubernetes installer & version: kops 1.30.0
  • Cloud provider or hardware configuration: aws
  • OS (e.g. from /etc/os-release): ubuntu

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

  • 👍 for "I would like to see this bug fixed as soon as possible"
  • 👎 for "There are more important bugs to focus on right now"
@ywk253100
Copy link
Contributor

This is the expected behavior that the DataUpload is canceled when node agent restarting

@ywk253100 ywk253100 self-assigned this Dec 9, 2024
@SCLogo
Copy link
Author

SCLogo commented Dec 9, 2024

I did not see any restart on that time

@SCLogo
Copy link
Author

SCLogo commented Dec 9, 2024

any of the node agents restart / come up? or the node agent which accepted the upload ?

@ywk253100
Copy link
Contributor

Th node agent which accepted the upload

@dharanui
Copy link

dharanui commented Dec 18, 2024

Same issue observed with any node-agent restart. Not only the one which accepted du.
(Not always)

@ywk253100
Copy link
Contributor

@dharanui Sorry for the mistake. The DataUpload is canceled if any node-agent restart.
Maybe we can do some improvement for this part. @Lyndon-Li

@Lyndon-Li
Copy link
Contributor

@dharanui Sorry for the mistake. The DataUpload is canceled if any node-agent restart. Maybe we can do some improvement for this part. @Lyndon-Li

Opened a separate issue #8534 to track this requirement

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants