You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When heartbeating after some task (that may be writing data in DB) in fakeProgress activity when there is a failure the activity retries from a previous successful progress which may lead to data inconsistency(when dealing with data insertion in DB).
Describe the bug
With the existing heartbeat in the fakeProgress activity it whem retried it have to perform an extra iteration, which when the heartbeat is sent immediately after entering the loop saves this extra iteration. When some data insertion is a step in this loop, this may lead to data duplication.
Minimal Reproduction
I have identified this issue on my personal project and this is quite self explanatory.
Example case:
when progress = 4
heartbeat details -> 3 (previous progress)
if activity fails at this progress it will be retried from the point where progress == 3, and the iteration with progress == 3 is a successful iteration which should not be the retrying point.
PR - #363
Environment/Versions
temporal sdk version - 1.9.3
The text was updated successfully, but these errors were encountered:
You cannot trust the progress, as the heartbeat(progress) could be throttled. The progress in the last heartbeat is probably less than the actual progress.
It is better to always to do some checks before inserting to database (or catch the unique key constraint violations).
@johnson-lu-opus I agree with you that there should be checks before inserting to DB, but it doesn't makes sense when an activity gets retried from a successful iteration. @mjameswh can you please review this issue and the PR attached.
What are you really trying to do?
When heartbeating after some task (that may be writing data in DB) in
fakeProgress
activity when there is a failure the activity retries from a previous successful progress which may lead to data inconsistency(when dealing with data insertion in DB).Describe the bug
With the existing heartbeat in the
fakeProgress
activity it whem retried it have to perform an extra iteration, which when the heartbeat is sent immediately after entering the loop saves this extra iteration. When some data insertion is a step in this loop, this may lead to data duplication.Minimal Reproduction
I have identified this issue on my personal project and this is quite self explanatory.
Example case:
when progress = 4
heartbeat details -> 3 (previous progress)
if activity fails at this progress it will be retried from the point where
progress == 3
, and the iteration with progress == 3 is a successful iteration which should not be the retrying point.PR - #363
Environment/Versions
temporal sdk version - 1.9.3
The text was updated successfully, but these errors were encountered: