BUG: incompatible dtype when creating string column with loc

[5j9]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

df = pd.DataFrame({"x": [0, 1, 2, 3]})

new_cols = pd.DataFrame(
    {"y": ["a", "b"], "z": ["c", "d"]}, index=[1, 2]
)

df.loc[slice(1, 3), ["y", "z"]] = new_cols

print(df)

Issue Description

Produces the following warnings:

....\~test2.py:9: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '['a' 'b' nan]' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  df.loc[slice(1, 3), ["y", "z"]] = new_cols
....\~test2.py:9: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '['c' 'd' nan]' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  df.loc[slice(1, 3), ["y", "z"]] = new_cols

Expected Behavior

No warning.

Installed Versions

INSTALLED VERSIONS

commit : b7ab856
python : 3.11.6.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19045
machine : AMD64
processor : Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United States.1252

pandas : 0+untagged.1.gb7ab856
numpy : 1.26.0
pytz : 2023.3.post1
dateutil : 2.8.2
setuptools : 68.2.2
pip : 23.3.1
Cython : None
pytest : 7.4.2
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.9.3
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.1.2
IPython : 8.16.1
pandas_datareader : None
bs4 : 4.12.2
bottleneck : None
dataframe-api-compat: None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.8.0
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 13.0.0
pyreadstat : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2023.3
qtpy : None
pyqt5 : None

[mroeschke]

cc @MarcoGorelli looks like assignment that expands (filling with NaN) should ignore the PDEP 6 warning?

[hvsesha]

@mroeschke
Instead of starting the slice position from 1 if we start from zero we are not getting the warnings

df.loc[slice(0, 3), ["y", "z"]] = new_cols

Kindly advise what other way need to fix the issue .

If you try slice with 0,3 it should work

[MarcoGorelli]

The issue is that, before setting the new values, a new float block is being created

pandas/pandas/core/internals/managers.py

Lines 810 to 816 in 192a4e3

	blocks.append(
	self._make_na_block(
	placement=mgr_locs,
	fill_value=fill_value,
	use_na_proxy=use_na_proxy,
	)
	)

Not really sure what to do about this with adding a tonne of complexity (to already too complex code). I'm inclined to say it's OK to raise here, and that you should set your whole column:

df = pd.DataFrame({"x": [0, 1, 2, 3]})

new_cols = pd.DataFrame(
    {"y": ["a", "b"], "z": ["c", "d"]}, index=[1, 2]
)

df.loc[:, ["y", "z"]] = new_cols

pandas will already align on the index anyway, and you'll get the same result

[5j9]

Thanks for investigating this issue.

I'm inclined to say it's OK to raise here, and that you should set your whole column

That's OK with me. It's just that the warning was a little vague.

pandas will already align on the index anyway, and you'll get the same result

If that's always the case, maybe pandas could raise another warning for assigning a dataframe using loc with specific rows specified in the indexer?