Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using "WriteThroughputBudget" makes the spark job to drag writing records in cosmos container and job runs longer #447

Open
txl013 opened this issue Apr 6, 2021 · 1 comment

Comments

@txl013
Copy link

txl013 commented Apr 6, 2021

I started to use, 3.6.1 version and "WriteThroughputBudget" is working. But i am experiencing a different issue, that the spark job is running very long and writing only records in 100 or even less per second at the end of the job(identified by querying the cosmos container manually with time interveal). Which is kind of drags the spark job to run very longer. Even though there are available RUs, the spark job is writing data verly slowly. This is not the case when the "WriteThroughputBudget" is not put in the configuration.
Below is the Screen shot of how the RUs usage and request to cosmosdb drops and drags at the end of job.

image

@aaronS7
Copy link

aaronS7 commented Apr 27, 2024

Were you able to find a solution to this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants