Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[close #138] br return backup ts #156

Merged
merged 6 commits into from
Jul 16, 2022
Merged

Conversation

haojinming
Copy link
Contributor

@haojinming haojinming commented Jul 16, 2022

What problem does this PR solve?

Issue Number: close #138

Problem Description: TBD

What is changed and how does it work?

BR return the backup-ts, which is ts described in issue #138 .

Code changes

  • Has exported function/method change

Check List for Tests

This PR has been tested by at least one of the following methods:

  • Unit test
  • Manual test (add detailed scripts or steps below)

image

image

Side effects

  • No side effects

Related changes

@haojinming haojinming requested review from pingyu and zeminzhou July 16, 2022 02:10
@haojinming haojinming changed the title [clode #138] backup return backup ts [close #138] backup return backup ts Jul 16, 2022
Signed-off-by: haojinming <[email protected]>
@haojinming haojinming changed the title [close #138] backup return backup ts [close #138] br return backup ts Jul 16, 2022
Copy link
Collaborator

@pingyu pingyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest to add related integration test, which extracts and verifies backup-ts from result of backup.

@@ -52,7 +54,7 @@ func (r *testBackup) SetUpSuite(c *C) {
defer httpmock.DeactivateAndReset()
// Exact URL match
httpmock.RegisterResponder("GET", `=~^/config`,
httpmock.NewStringResponder(200, `{"storage":{"api-version":1, "enable-ttl":false}}`))
httpmock.NewStringResponder(200, `{"storage":{"api-version":2, "enable-ttl":false}}`))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"enable-ttl: true" would be reasonable when api-version is 2.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -49,6 +51,10 @@ func DefineRawBackupFlags(command *cobra.Command) {
command.Flags().Bool(flagRemoveSchedulers, false,
"disable the balance, shuffle and region-merge schedulers in PD to speed up backup.")

command.Flags().Duration(flagSafeInterval, utils.DefaultBRSafeInterval,
"The interval between backup-ts and current tso.")
command.Flags().Int64(flagGCTTL, utils.DefaultBRGCSafePointTTL, "the TTL (in seconds) that PD holds for BR's GC safepoint")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest to be a duration argument.
TTL would be expected to be e.g, 30 minutes, or 3 days. It is too big to be in secondes.

Suggested change
command.Flags().Int64(flagGCTTL, utils.DefaultBRGCSafePointTTL, "the TTL (in seconds) that PD holds for BR's GC safepoint")
command.Flags().Int64(flagGCTTL, utils.DefaultBRGCSafePointTTL, "the TTL of BR's GC safepoint")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -49,6 +51,10 @@ func DefineRawBackupFlags(command *cobra.Command) {
command.Flags().Bool(flagRemoveSchedulers, false,
"disable the balance, shuffle and region-merge schedulers in PD to speed up backup.")

command.Flags().Duration(flagSafeInterval, utils.DefaultBRSafeInterval,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest to hide safe interval. It's difficult for common users to set this argument.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

// DefaultBRGCSafePointTTL means PD keep safePoint limit at least 5min.
DefaultBRGCSafePointTTL = 5 * 60
// DefaultBRGCSafePointTTL means PD keep safePoint limit at least 72h.
DefaultBRGCSafePointTTL = 72 * 60 * 60 // 72h
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defaults to 72 hours may be too long, and waste disk space if there is no following CDC.
Maybe we can keep 5 minutes, but tell CDC users to set a suitable value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

@zeminzhou zeminzhou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM~

Signed-off-by: haojinming <[email protected]>
@haojinming
Copy link
Contributor Author

Suggest to add related integration test, which extracts and verifies backup-ts from result of backup.

I would like to add integration test for apiv2 and backup-ts in next pr. May need another new github action for apiv2 and need some more time to let it run correctly.

Copy link
Collaborator

@pingyu pingyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM~

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

br: Return the timestamp of backup to cooperate with TiKV-CDC
3 participants