ENH: Add case_when method #56059

samukweku · 2023-11-19T07:23:38Z

closes ENH: Dedicated method for creating conditional columns #39154
Tests added and passed if fixing a bug or adding a new feature
All code checks passed.
Added type annotations to new arguments/methods/functions.
Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

Continues the work started by @ELHoussineT(#50343)

uses pd.Series.mask under the hood
function implementation, as well as a Series method

samukweku · 2023-11-19T07:27:28Z

pre-commit.ci autofix

samukweku · 2023-11-19T08:24:12Z

@rhshadrach had issues with the previous pr #55390 and created this one. If there is a way to reconnect to the old one, kindly advise me on the approach. Thanks

rhshadrach

I think the implementation is looking pretty good - but tests can be expanded on. Would it be okay if I pushed some tests to your branch?

pandas/core/case_when.py

pandas/tests/test_case_when.py

…_function

samukweku · 2023-11-25T21:10:31Z

@rhshadrach made changes based on your feedback. Looking forward to your reviews

rhshadrach · 2023-11-28T21:59:25Z

Thanks for the changes. I think this is looking good - but want to add more tests when I get a chance. In the meantime, cc @pandas-dev/pandas-core for any thoughts/feedback here.

pandas/core/case_when.py

Dr-Irv

Just added a comment with respect to typing, and also another check to validate arguments.

jbrockmendel · 2023-11-29T16:04:29Z

Thoughts:

@samukweku will you have bandwidth/willingness to be responsible for maintenance of this going forward?
id prefer either a top-level function or a method rather than both
in the same direction as Irv: strong-as-possible typing will make me minimally cranky

…_function

samukweku · 2023-11-30T12:07:26Z

@jbrockmendel in response to your comments:

yes i am willing to be responsible for the maintenance going forward.
i'd lean more in favour of a method - it would certainly make the code simpler, since it would be a method on a Series. if the team agrees on this, that would be great. The reason behind providing a function was to provide more flexibility - the conversation gives an idea of where the function could be helpful, but generally a method would be my preference.
made some changes based on @Dr-Irv feedback. Thanks @Dr-Irv

pandas/core/case_when.py

…_function

pandas/core/case_when.py

…_function

Dr-Irv

I'm fine with the typing changes and changes to the API, but I will let other maintainers take care of the rest of this one.

phofl · 2023-12-02T18:28:16Z

Strong preference for method

phofl · 2023-12-02T18:29:15Z

pandas/core/case_when.py

+            updated_replacements.append(replacement)
+        replacements = updated_replacements
+        if (default is not None) and isinstance(default, ABCSeries):
+            default = default.astype(common_dtypes)


cc @MarcoGorelli

This might upcast, thoughts related to PDEP6?

I don't think this is related to PDEP6, we are creating a new Series and not doing something like setitem.

…ails (#56589) Add McKinney (2010) reference, DOI, team website, & more keywords

* REF: fix can_use_libjoin check * DOC: docstring for can_use_libjoin * Make can_use_libjoin checks more-correct * avoid allocating mapping in monotonic cases * fix categorical memory usage tests * catch decimal.InvalidOperation --------- Co-authored-by: Luke Manley <[email protected]>

* TYP: fix some annotations * pyupgrade

* BUG: Series.to_numpy raising for arrow floats to numpy floats * Fixup

jbrockmendel · 2023-12-28T01:58:49Z

How does this handle attrs/_metadata/flags when given mismatched args with mismatched attrs/_metadat/flags?

samukweku · 2023-12-28T02:10:18Z

How does this handle attrs/_metadata/flags when given mismatched args with mismatched attrs/_metadat/flags?

@jbrockmendel i am unfamiliar with your question. Kindly share an example to help me understand.

phofl · 2023-12-28T15:24:34Z

@samukweku your latest rebase threw something off here, the diff is too large. Could you take a look?

phofl · 2023-12-28T15:25:16Z

@jbrockmendel

The handling off metadata is dispatched to mask in this case, so we should be good

Fix PR05 error

samukweku · 2024-01-05T09:01:20Z

I think the implementation is looking pretty good - but tests can be expanded on. Would it be okay if I pushed some tests to your branch?

@rhshadrach yes pls feel free to add more tests

phofl · 2024-01-09T12:54:39Z

merging this, we can always add tests after if necessary

outdated

phofl · 2024-01-09T12:55:40Z

thx for staying with us @samukweku

lumberbot-app · 2024-01-09T12:55:49Z

Something went wrong ... Please have a look at my logs.

(cherry picked from commit e3a55a4)

ENH: Add case_when method (#56059) (cherry picked from commit e3a55a4) Co-authored-by: Samuel Oranyeli <[email protected]>

samukweku added 2 commits November 19, 2023 18:08

updates

f48502f

add test for default if Series

40057c7

rhshadrach previously requested changes Nov 20, 2023

View reviewed changes

pandas/core/case_when.py Outdated Show resolved Hide resolved

pandas/core/case_when.py Outdated Show resolved Hide resolved

pandas/core/case_when.py Outdated Show resolved Hide resolved

pandas/tests/test_case_when.py Outdated Show resolved Hide resolved

samukweku added 2 commits November 23, 2023 13:27

Merge remote-tracking branch 'upstream/main' into samukweku/case_when…

4a8be16

…_function

updates based on feedback

089bbe6

samukweku requested a review from rhshadrach November 23, 2023 03:21

Dr-Irv reviewed Nov 28, 2023

View reviewed changes

pandas/core/case_when.py Outdated Show resolved Hide resolved

Dr-Irv reviewed Nov 28, 2023

View reviewed changes

pandas/core/case_when.py Outdated Show resolved Hide resolved

Dr-Irv reviewed Nov 28, 2023

View reviewed changes

Merge remote-tracking branch 'upstream/main' into samukweku/case_when…

bcfd458

…_function

updates based on feedback

b95ce55

Dr-Irv requested changes Nov 30, 2023

View reviewed changes

pandas/core/case_when.py Outdated Show resolved Hide resolved

samukweku added 2 commits December 1, 2023 09:17

Merge remote-tracking branch 'upstream/main' into samukweku/case_when…

acc3fdb

…_function

update typing hints for *args, based on feedback

8be4349

samukweku requested a review from Dr-Irv November 30, 2023 22:20

Dr-Irv reviewed Nov 30, 2023

View reviewed changes

pandas/core/case_when.py Outdated Show resolved Hide resolved

samukweku added 3 commits December 1, 2023 17:04

Merge remote-tracking branch 'upstream/main' into samukweku/case_when…

8d08458

…_function

update typehints; add caselist argument - based on feedback

ec18086

cleanup docstrings

0b72fbb

samukweku requested review from mroeschke and Dr-Irv December 1, 2023 06:41

Dr-Irv reviewed Dec 1, 2023

View reviewed changes

phofl reviewed Dec 2, 2023

View reviewed changes

cgobat and others added 7 commits December 28, 2023 11:19

ENH: Update CFF with publication reference, Zenodo DOI, and other det…

3e404fa

…ails (#56589) Add McKinney (2010) reference, DOI, team website, & more keywords

DOC: Fixup CoW userguide (#56636)

21659bc

DOC: Minor fixups for 2.2.0 whatsnew (#56632)

becc626

TYP: Fix some PythonParser and Plotting types (#56643)

918a19e

* TYP: fix some annotations * pyupgrade

BUG: Series.to_numpy raising for arrow floats to numpy floats (#56644)

5744df2

* BUG: Series.to_numpy raising for arrow floats to numpy floats * Fixup

updates based on feedback

bc6ba0e

samukweku requested review from attack68, MarcoGorelli and WillAyd as code owners December 28, 2023 00:29

add to API reference

a0f4797

samukweku requested a review from mroeschke December 28, 2023 00:37

fix whitespace

cb7d6e3

samukweku added 2 commits December 29, 2023 09:26

updates

c8f0e2e

Update series.py

9679b9e

Fix PR05 error

phofl approved these changes Jan 9, 2024

View reviewed changes

phofl merged commit e3a55a4 into pandas-dev:main Jan 9, 2024
50 checks passed

phofl pushed a commit to phofl/pandas that referenced this pull request Jan 9, 2024

ENH: Add case_when method (pandas-dev#56059)

38cbce4

(cherry picked from commit e3a55a4)

phofl mentioned this pull request Jan 9, 2024

Backport PR #56059 on branch 2.2.x (ENH: Add case_when method) #56800

Merged

phofl added a commit that referenced this pull request Jan 9, 2024

Backport PR #56059 on branch 2.2.x (ENH: Add case_when method) (#56800)

66df0bd

ENH: Add case_when method (#56059) (cherry picked from commit e3a55a4) Co-authored-by: Samuel Oranyeli <[email protected]>

pmhatre1 pushed a commit to pmhatre1/pandas-pmhatre1 that referenced this pull request May 7, 2024

ENH: Add case_when method (pandas-dev#56059)

aeaf086

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Add case_when method #56059

ENH: Add case_when method #56059

samukweku commented Nov 19, 2023

samukweku commented Nov 19, 2023

samukweku commented Nov 19, 2023

rhshadrach left a comment

samukweku commented Nov 25, 2023

rhshadrach commented Nov 28, 2023

Dr-Irv left a comment

jbrockmendel commented Nov 29, 2023

samukweku commented Nov 30, 2023

Dr-Irv left a comment

phofl commented Dec 2, 2023

phofl Dec 2, 2023

rhshadrach Dec 12, 2023

jbrockmendel commented Dec 28, 2023

samukweku commented Dec 28, 2023

phofl commented Dec 28, 2023

phofl commented Dec 28, 2023

samukweku commented Jan 5, 2024

phofl commented Jan 9, 2024

phofl commented Jan 9, 2024

lumberbot-app bot commented Jan 9, 2024

ENH: Add case_when method #56059

ENH: Add case_when method #56059

Conversation

samukweku commented Nov 19, 2023

samukweku commented Nov 19, 2023

samukweku commented Nov 19, 2023

rhshadrach left a comment

Choose a reason for hiding this comment

samukweku commented Nov 25, 2023

rhshadrach commented Nov 28, 2023

Dr-Irv left a comment

Choose a reason for hiding this comment

jbrockmendel commented Nov 29, 2023

samukweku commented Nov 30, 2023

Dr-Irv left a comment

Choose a reason for hiding this comment

phofl commented Dec 2, 2023

phofl Dec 2, 2023

Choose a reason for hiding this comment

rhshadrach Dec 12, 2023

Choose a reason for hiding this comment

jbrockmendel commented Dec 28, 2023

samukweku commented Dec 28, 2023

phofl commented Dec 28, 2023

phofl commented Dec 28, 2023

samukweku commented Jan 5, 2024

phofl commented Jan 9, 2024

phofl commented Jan 9, 2024

lumberbot-app bot commented Jan 9, 2024