-
Notifications
You must be signed in to change notification settings - Fork 212
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix ValueError for Empty DataFrames: Ensure Process Count is at Least 1 #245
Conversation
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #245 +/- ##
==========================================
+ Coverage 86.72% 91.28% +4.55%
==========================================
Files 12 12
Lines 580 585 +5
==========================================
+ Hits 503 534 +31
+ Misses 77 51 -26 ☔ View full report in Codecov by Sentry. |
Hi @Mithil467, thanks for the PR. Could you please add a test for this case? |
Sure. I noticed that when progress_bar is True, we get ZeroDivisionError. In order to fix that, I would need to know what we expect UI wise. I personally like [2] more than [1] as it gives a sense of success, but would like to know your opinions or if you want to do things differently. Hence, I have added [2] for now, let me know if it needs changes. Consoles: Notebook: |
Personally, I prefer option 1.: The grey bar is neither failure (red) nor success (green), similarly, the processing of the empty DataFrame didn't really succeed or fail. |
Hi dev team, any updates on this one? |
My apologies, I am not maintaining this project anymore, hence me not responding. But I can make an exception and see this PR through. I would suggest going with [1]. @Mithil467 could you kindly ping me when/if you've implemented that? |
Pandaral·lel is looking for a maintainer! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM - checked that it works in a small script and in a notebook on my machine. Thanks for the PR + also the tests! Will merge soon.
Thanks for the contribution :) |
Thanks for the help! @till-m |
Since nb_item comes out as 0 for empty dataframes and series, we were returning an empty list from the
chunk
function.Hence, we were yielding nothing from our
DataType.get_chunks
method which caused ourchunks
list being empty and nb_workers = len(chunks) = 0.Let me know if this fix seems good enough, and also if we need to add any new tests.
Fixes #115, fixes #141.