Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TxWithRetries and test cases #109

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

jeremy-babylonlabs
Copy link
Contributor

#63

Features:

  • Conditions retries on the type of error using the shouldRetry function, targeting network failures, timeouts, write conflicts, and transaction aborts.
  • Default settings for maxAttempts, backoffFactors and initialBackoff for the exponential backoff algo.
  • Implements startSession in dbTransactionClient to initiate MongoDB sessions and end with endSession method.

Tests:

  • test exponential algo
  • test max attempts
  • mock transient or nonretryable errors

CleanShot 2024-05-10 at 19 21 37@2x
CleanShot 2024-05-10 at 19 21 29@2x
CleanShot 2024-05-10 at 19 21 19@2x

return cmdErr.Code == 112
}

log.Println("Error does not conform to CommandError")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we using "github.com/rs/zerolog/log" in the service. i'm not sure if using log package directly would be a good idea.

Copy link
Collaborator

@jrwbabylonlab jrwbabylonlab May 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just adding on top of this. i don't think we actually need any logs in this file. it's the calling methods responsibility to log errors.
That means u can simply remove all logs in this file

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all the methods here looks very similar, they all try to check the cmdErr code.
shall we have a private generic method that does that and compare the code instead? then this generic method can be called by IsWriteConflictError etc by passing in the 122 code into the method

dbTransactionClient DBTransactionClient,
txnFunc func(sessCtx mongo.SessionContext) (interface{}, error),
) (interface{}, error) {
maxAttempts := DefaultMaxAttempts
Copy link
Collaborator

@jrwbabylonlab jrwbabylonlab May 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maxAttempts is not overwritten anywhere in this method, why not use the pre-defined variable directly in this case?
same as initialBackoff and backoffFactor

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes I can remove that

@@ -54,3 +56,10 @@ type DBClient interface {
) error
FindTopStakersByTvl(ctx context.Context, paginationToken string) (*DbResultMap[model.StakerStatsDocument], error)
}
type DBSession interface {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you remind me why we need to define this here?
I'm thinking twice around this, feel like we shall not expose internal implementation of the db client here as the interface here are used for defining API contract, it's not mongodb specific.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was trying to make it an interface so that the function could consume both test db and mongo db

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in go, unit tests are placed under the same file directory as the methods you are trying to test with. this would avoid u making those internal implementation methods being public

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok agree

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: why do we need this implementation instead of using the time.sleep method directly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im abstracted the sleep func to mainly use it in tests for mock sleep

backoffFactor := DefaultBackoffFactor

var (
result interface{}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there might be too many tabs here

for attempt := 1; attempt <= maxAttempts; attempt++ {
session, sessionErr := dbTransactionClient.StartSession();


Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra space

if mongo.IsNetworkError(err) {
return true
}
if mongo.IsTimeout(err) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think the network error and timeout error from mongo is already retry handled by the mongo driver implementation of WithTransaction

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants