Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sc.pl.heatmap var_names function KeyError #3363

Open
3 tasks done
josenimo opened this issue Nov 13, 2024 · 1 comment
Open
3 tasks done

sc.pl.heatmap var_names function KeyError #3363

josenimo opened this issue Nov 13, 2024 · 1 comment
Labels

Comments

@josenimo
Copy link

Please make sure these conditions are met

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of scanpy.
  • (optional) I have confirmed this bug exists on the main branch of scanpy.

What happened?

I pass a adata.var column header with my gene name that I want displayed, sc.pl.heatmap runs into a key error where my index values are being searched in the genename column

Minimal code sample

sc.pl.heatmap(
    adata=adata,
    var_names=top_25,
    gene_symbols=adata.var.columns.tolist()[2],
    groupby='slide_number',
    swap_axes=True,
    standard_scale='var',
)

Error output

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[86], line 1
----> 1 sc.pl.heatmap(
      2     adata=adata,
      3     var_names=top_25,
      4     gene_symbols=adata.var.columns.tolist()[2],
      5     groupby='slide_number',
      6     swap_axes=True,
      7     standard_scale='var',
      8 )

File /opt/homebrew/Caskroom/mambaforge/base/envs/proteomics/lib/python3.12/site-packages/legacy_api_wrap/__init__.py:80, in legacy_api.<locals>.wrapper.<locals>.fn_compatible(*args_all, **kw)
     77 @wraps(fn)
     78 def fn_compatible(*args_all: P.args, **kw: P.kwargs) -> R:
     79     if len(args_all) <= n_positional:
---> 80         return fn(*args_all, **kw)
     82     args_pos: P.args
     83     args_pos, args_rest = args_all[:n_positional], args_all[n_positional:]

File /opt/homebrew/Caskroom/mambaforge/base/envs/proteomics/lib/python3.12/site-packages/scanpy/plotting/_anndata.py:1120, in heatmap(adata, var_names, groupby, use_raw, log, num_categories, dendrogram, gene_symbols, var_group_positions, var_group_labels, var_group_rotation, layer, standard_scale, swap_axes, show_gene_labels, show, save, figsize, vmin, vmax, vcenter, norm, **kwds)
   1070 """\
   1071 Heatmap of the expression values of genes.
   1072 
   (...)
   1114 tl.rank_genes_groups
   1115 """
   1116 var_names, var_group_labels, var_group_positions = _check_var_names_type(
   1117     var_names, var_group_labels, var_group_positions
   1118 )
-> 1120 categories, obs_tidy = _prepare_dataframe(
   1121     adata,
   1122     var_names,
   1123     groupby,
   1124     use_raw=use_raw,
   1125     log=log,
   1126     num_categories=num_categories,
   1127     gene_symbols=gene_symbols,
   1128     layer=layer,
   1129 )
   1131 # check if var_group_labels are a subset of categories:
   1132 if var_group_labels is not None:

File /opt/homebrew/Caskroom/mambaforge/base/envs/proteomics/lib/python3.12/site-packages/scanpy/plotting/_anndata.py:2057, in _prepare_dataframe(adata, var_names, groupby, use_raw, log, num_categories, layer, gene_symbols)
   2055     groupby.remove(groupby_index)
   2056 keys = list(groupby) + list(np.unique(var_names))
-> 2057 obs_tidy = get.obs_df(
   2058     adata, keys=keys, layer=layer, use_raw=use_raw, gene_symbols=gene_symbols
   2059 )
   2060 assert np.all(np.array(keys) == np.array(obs_tidy.columns))
   2062 if groupby_index is not None:
   2063     # reset index to treat all columns the same way.

File /opt/homebrew/Caskroom/mambaforge/base/envs/proteomics/lib/python3.12/site-packages/scanpy/get/get.py:296, in obs_df(adata, keys, obsm_keys, layer, gene_symbols, use_raw)
    293 else:
    294     alias_index = None
--> 296 obs_cols, var_idx_keys, var_symbols = _check_indices(
    297     adata.obs,
    298     var.index,
    299     dim="obs",
    300     keys=keys,
    301     alias_index=alias_index,
    302     use_raw=use_raw,
    303 )
    305 # Make df
    306 df = pd.DataFrame(index=adata.obs_names)

File /opt/homebrew/Caskroom/mambaforge/base/envs/proteomics/lib/python3.12/site-packages/scanpy/get/get.py:186, in _check_indices(dim_df, alt_index, dim, keys, alias_index, use_raw)
    184         not_found.append(key)
    185 if len(not_found) > 0:
--> 186     raise KeyError(
    187         f"Could not find keys '{not_found}' in columns of `adata.{dim}` or in"
    188         f" {alt_repr}.{alt_search_repr}."
    189     )
    191 return col_keys, index_keys, index_aliases

KeyError: "Could not find keys '[np.str_('O00154'), np.str_('O00515'), np.str_('O94886'), np.str_('P09493'), np.str_('P20338'), np.str_('P23141'), np.str_('P49419'), np.str_('P55268'), np.str_('P98095'), np.str_('P98160'), np.str_('Q01995'), np.str_('Q05682'), np.str_('Q09666'), np.str_('Q13751'), np.str_('Q14195'), np.str_('Q16795'), np.str_('Q8IUX7'), np.str_('Q8WX93'), np.str_('Q96AC1'), np.str_('Q96FZ7'), np.str_('Q96G03'), np.str_('Q9NSE4'), np.str_('Q9NZM1'), np.str_('Q9UN36'), np.str_('Q9Y4K1')]' in columns of `adata.obs` or in adata.var['Genes']."

Versions

# packages in environment at /opt/homebrew/Caskroom/mambaforge/base/envs/proteomics:
#
# Name                    Version                   Build  Channel
adjusttext                1.2.0              pyhd8ed1ab_0    conda-forge
anndata                   0.10.9             pyhd8ed1ab_0    conda-forge
anyio                     4.6.2.post1        pyhd8ed1ab_0    conda-forge
appnope                   0.1.4              pyhd8ed1ab_0    conda-forge
argon2-cffi               23.1.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0          py312h024a12e_5    conda-forge
array-api-compat          1.9                pyhd8ed1ab_0    conda-forge
arrow                     1.3.0              pyhd8ed1ab_0    conda-forge
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
async-lru                 2.0.4              pyhd8ed1ab_0    conda-forge
attrs                     24.2.0             pyh71513ae_0    conda-forge
babel                     2.14.0             pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.12.3             pyha770c72_0    conda-forge
bleach                    6.1.0              pyhd8ed1ab_0    conda-forge
brotli                    1.1.0                hd74edd7_2    conda-forge
brotli-bin                1.1.0                hd74edd7_2    conda-forge
brotli-python             1.1.0           py312hde4cb15_2    conda-forge
bzip2                     1.0.8                h99b78c6_7    conda-forge
c-ares                    1.34.2               h7ab814d_0    conda-forge
ca-certificates           2024.8.30            hf0a4a13_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
certifi                   2024.8.30          pyhd8ed1ab_0    conda-forge
cffi                      1.17.1          py312h0fad829_0    conda-forge
charset-normalizer        3.4.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
comm                      0.2.2              pyhd8ed1ab_0    conda-forge
contourpy                 1.3.0           py312h6142ec9_2    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
debugpy                   1.8.7           py312hde4cb15_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
exceptiongroup            1.2.2              pyhd8ed1ab_0    conda-forge
executing                 2.1.0              pyhd8ed1ab_0    conda-forge
fonttools                 4.54.1          py312h024a12e_0    conda-forge
fqdn                      1.5.1              pyhd8ed1ab_0    conda-forge
freetype                  2.12.1               hadb7bae_2    conda-forge
get-annotations           0.1.2              pyhd8ed1ab_0    conda-forge
h11                       0.14.0             pyhd8ed1ab_0    conda-forge
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
h5py                      3.11.0          nompi_py312h3cd0824_103    conda-forge
hdf5                      1.14.3          nompi_hec07895_105    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
httpcore                  1.0.6              pyhd8ed1ab_0    conda-forge
httpx                     0.27.2             pyhd8ed1ab_0    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
icu                       75.1                 hfee45f7_0    conda-forge
idna                      3.10               pyhd8ed1ab_0    conda-forge
importlib-metadata        8.5.0              pyha770c72_0    conda-forge
importlib_metadata        8.5.0                hd8ed1ab_0    conda-forge
importlib_resources       6.4.5              pyhd8ed1ab_0    conda-forge
ipykernel                 6.29.5             pyh57ce528_0    conda-forge
ipython                   8.28.0             pyh707e725_0    conda-forge
ipywidgets                8.1.5              pyhd8ed1ab_0    conda-forge
isoduration               20.11.0            pyhd8ed1ab_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
joblib                    1.4.2              pyhd8ed1ab_0    conda-forge
json5                     0.9.25             pyhd8ed1ab_0    conda-forge
jsonpointer               3.0.0           py312h81bd7bf_1    conda-forge
jsonschema                4.23.0             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2024.10.1          pyhd8ed1ab_0    conda-forge
jsonschema-with-format-nongpl 4.23.0               hd8ed1ab_0    conda-forge
jupyter                   1.1.1              pyhd8ed1ab_0    conda-forge
jupyter-lsp               2.2.5              pyhd8ed1ab_0    conda-forge
jupyter_client            8.6.3              pyhd8ed1ab_0    conda-forge
jupyter_console           6.6.3              pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.2              pyh31011fe_1    conda-forge
jupyter_events            0.10.0             pyhd8ed1ab_0    conda-forge
jupyter_server            2.14.2             pyhd8ed1ab_0    conda-forge
jupyter_server_terminals  0.5.3              pyhd8ed1ab_0    conda-forge
jupyterlab                4.2.5              pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.3.0              pyhd8ed1ab_1    conda-forge
jupyterlab_server         2.27.3             pyhd8ed1ab_0    conda-forge
jupyterlab_widgets        3.0.13             pyhd8ed1ab_0    conda-forge
kaleido                   0.2.1                    pypi_0    pypi
kiwisolver                1.4.7           py312h6142ec9_0    conda-forge
krb5                      1.21.3               h237132a_0    conda-forge
lcms2                     2.16                 ha0e7c42_0    conda-forge
legacy-api-wrap           1.4                pyhd8ed1ab_1    conda-forge
lerc                      4.0.0                h9a09cb3_0    conda-forge
libaec                    1.1.3                hebf3989_0    conda-forge
libblas                   3.9.0           24_osxarm64_openblas    conda-forge
libbrotlicommon           1.1.0                hd74edd7_2    conda-forge
libbrotlidec              1.1.0                hd74edd7_2    conda-forge
libbrotlienc              1.1.0                hd74edd7_2    conda-forge
libcblas                  3.9.0           24_osxarm64_openblas    conda-forge
libcurl                   8.10.1               h13a7ad3_0    conda-forge
libcxx                    19.1.2               ha82da77_0    conda-forge
libdeflate                1.22                 hd74edd7_0    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h93a5062_2    conda-forge
libexpat                  2.6.3                hf9b8971_0    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgfortran               5.0.0           13_2_0_hd922786_3    conda-forge
libgfortran5              13.2.0               hf226fd6_3    conda-forge
libhwloc                  2.11.1          default_h7685b71_1000    conda-forge
libiconv                  1.17                 h0d3ecfb_2    conda-forge
libjpeg-turbo             3.0.0                hb547adb_1    conda-forge
liblapack                 3.9.0           24_osxarm64_openblas    conda-forge
libllvm14                 14.0.6               hd1a9a77_4    conda-forge
libnghttp2                1.58.0               ha4dd798_1    conda-forge
libopenblas               0.3.27          openmp_h517c56d_1    conda-forge
libpng                    1.6.44               hc14010f_0    conda-forge
libsodium                 1.0.20               h99b78c6_0    conda-forge
libsqlite                 3.46.1               hc14010f_0    conda-forge
libssh2                   1.11.0               h7a5bd25_0    conda-forge
libtiff                   4.7.0                hfce79cd_1    conda-forge
libwebp-base              1.4.0                h93a5062_0    conda-forge
libxcb                    1.17.0               hdb1d25a_0    conda-forge
libxml2                   2.12.7               h01dff8b_4    conda-forge
libzlib                   1.3.1                h8359307_2    conda-forge
littleutils               0.2.4              pyhd8ed1ab_0    conda-forge
llvm-openmp               19.1.1               hb52a8e5_1    conda-forge
llvmlite                  0.43.0          py312ha9ca408_1    conda-forge
loguru                    0.7.2           py312h81bd7bf_2    conda-forge
markupsafe                3.0.1           py312h906988d_1    conda-forge
matplotlib                3.9.2           py312h1f38498_1    conda-forge
matplotlib-base           3.9.2           py312h9bd0bc6_1    conda-forge
matplotlib-inline         0.1.7              pyhd8ed1ab_0    conda-forge
mistune                   3.0.2              pyhd8ed1ab_0    conda-forge
mpmath                    1.3.0              pyhd8ed1ab_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
natsort                   8.4.0              pyhd8ed1ab_0    conda-forge
nbclient                  0.10.0             pyhd8ed1ab_0    conda-forge
nbconvert-core            7.16.4             pyhd8ed1ab_1    conda-forge
nbformat                  5.10.4             pyhd8ed1ab_0    conda-forge
ncurses                   6.5                  h7bae524_1    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
networkx                  3.4.1              pyhd8ed1ab_0    conda-forge
notebook                  7.2.2              pyhd8ed1ab_0    conda-forge
notebook-shim             0.2.4              pyhd8ed1ab_0    conda-forge
numba                     0.60.0          py312h41cea2d_0    conda-forge
numpy                     2.0.2           py312h801f5e3_0    conda-forge
openjpeg                  2.5.2                h9f1df11_0    conda-forge
openssl                   3.3.2                h8359307_0    conda-forge
outdated                  0.2.2              pyhd8ed1ab_1    conda-forge
overrides                 7.7.0              pyhd8ed1ab_0    conda-forge
packaging                 24.1               pyhd8ed1ab_0    conda-forge
pandas                    2.2.3           py312hcd31e36_1    conda-forge
pandas-flavor             0.6.0              pyhd8ed1ab_2    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
parso                     0.8.4              pyhd8ed1ab_0    conda-forge
patsy                     0.5.6              pyhd8ed1ab_0    conda-forge
pexpect                   4.9.0              pyhd8ed1ab_0    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    11.0.0          py312haf37ca6_0    conda-forge
pingouin                  0.5.5              pyhd8ed1ab_0    conda-forge
pip                       24.2               pyh8b19718_1    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              4.3.6              pyhd8ed1ab_0    conda-forge
plotly                    5.24.1             pyhd8ed1ab_0    conda-forge
prometheus_client         0.21.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.48             pyha770c72_0    conda-forge
prompt_toolkit            3.0.48               hd8ed1ab_0    conda-forge
psutil                    6.0.0           py312h0bf5046_2    conda-forge
pthread-stubs             0.4               hd74edd7_1002    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.3              pyhd8ed1ab_0    conda-forge
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
pynndescent               0.5.13             pyhff2d567_0    conda-forge
pyobjc-core               10.3.1          py312hd24fc31_1    conda-forge
pyobjc-framework-cocoa    10.3.1          py312hd24fc31_1    conda-forge
pyparsing                 3.2.0              pyhd8ed1ab_1    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.12.7          h739c21a_0_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.20.0             pyhd8ed1ab_0    conda-forge
python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
python-tzdata             2024.2             pyhd8ed1ab_0    conda-forge
python_abi                3.12                    5_cp312    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.2           py312h024a12e_1    conda-forge
pyzmq                     26.2.0          py312hf8a1cbd_3    conda-forge
qhull                     2020.2               h420ef59_5    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
referencing               0.35.1             pyhd8ed1ab_0    conda-forge
requests                  2.32.3             pyhd8ed1ab_0    conda-forge
rfc3339-validator         0.1.4              pyhd8ed1ab_0    conda-forge
rfc3986-validator         0.1.1              pyh9f0ad1d_0    conda-forge
rpds-py                   0.20.0          py312he431725_1    conda-forge
scanpy                    1.10.3             pyhd8ed1ab_0    conda-forge
scikit-learn              1.5.2           py312h387f99c_1    conda-forge
scipy                     1.14.1          py312heb3a901_0    conda-forge
seaborn                   0.13.2               hd8ed1ab_2    conda-forge
seaborn-base              0.13.2             pyhd8ed1ab_2    conda-forge
send2trash                1.8.3              pyh31c8845_0    conda-forge
session-info              1.0.0              pyhd8ed1ab_0    conda-forge
setuptools                75.1.0             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sniffio                   1.3.1              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
statsmodels               0.14.4          py312h755e627_0    conda-forge
stdlib-list               0.11.0             pyhd8ed1ab_0    conda-forge
tabulate                  0.9.0              pyhd8ed1ab_1    conda-forge
tbb                       2021.13.0            h7b3277c_0    conda-forge
tenacity                  9.0.0              pyhd8ed1ab_0    conda-forge
terminado                 0.18.1             pyh31c8845_0    conda-forge
threadpoolctl             3.5.0              pyhc1e730c_0    conda-forge
tinycss2                  1.3.0              pyhd8ed1ab_0    conda-forge
tk                        8.6.13               h5083fa2_1    conda-forge
tomli                     2.0.2              pyhd8ed1ab_0    conda-forge
tornado                   6.4.1           py312h024a12e_1    conda-forge
tqdm                      4.66.5             pyhd8ed1ab_0    conda-forge
traitlets                 5.14.3             pyhd8ed1ab_0    conda-forge
types-python-dateutil     2.9.0.20241003     pyhff2d567_0    conda-forge
typing-extensions         4.12.2               hd8ed1ab_0    conda-forge
typing_extensions         4.12.2             pyha770c72_0    conda-forge
typing_utils              0.1.0              pyhd8ed1ab_0    conda-forge
tzdata                    2024b                hc8b5060_0    conda-forge
umap-learn                0.5.6           py312h81bd7bf_1    conda-forge
uri-template              1.3.0              pyhd8ed1ab_0    conda-forge
urllib3                   2.2.3              pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
webcolors                 24.8.0             pyhd8ed1ab_0    conda-forge
webencodings              0.5.1              pyhd8ed1ab_2    conda-forge
websocket-client          1.8.0              pyhd8ed1ab_0    conda-forge
wheel                     0.44.0             pyhd8ed1ab_0    conda-forge
widgetsnbextension        4.0.13             pyhd8ed1ab_0    conda-forge
xarray                    2024.9.0           pyhd8ed1ab_1    conda-forge
xorg-libxau               1.0.11               hd74edd7_1    conda-forge
xorg-libxdmcp             1.1.5                hd74edd7_0    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
yaml                      0.2.5                h3422bc3_2    conda-forge
zeromq                    4.3.5                h9f5b81c_6    conda-forge
zipp                      3.20.2             pyhd8ed1ab_0    conda-forge
zstandard                 0.23.0          py312h15fbf35_1    conda-forge
zstd                      1.5.6                hb46c0d2_0    conda-forge
@josenimo josenimo added Bug 🐛 Triage 🩺 This issue needs to be triaged by a maintainer labels Nov 13, 2024
@ilan-gold
Copy link
Contributor

@josenimo Judging by our tests, the var_names argument needs to correspond to gene_symbols i.e., it doesn't come from the adata.var_names. I think that's a failing of the docs more than anything because there is somewhat contradictory documentation:

var_names
[str](https://docs.python.org/3/library/stdtypes.html#str) | [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence)[[str](https://docs.python.org/3/library/stdtypes.html#str)] | [Mapping](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping)[[str](https://docs.python.org/3/library/stdtypes.html#str), [str](https://docs.python.org/3/library/stdtypes.html#str) | [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence)[[str](https://docs.python.org/3/library/stdtypes.html#str)]]

    var_names should be a valid subset of adata.var_names.

and

gene_symbols
[str](https://docs.python.org/3/library/stdtypes.html#str) | [None](https://docs.python.org/3/library/constants.html#None) (default: None)

    Column name in .var DataFrame that stores gene symbols. By default var_names refer to the index column of the .var DataFrame. Setting this option allows alternative names to be used.

@flying-sheep thoughts?

@ilan-gold ilan-gold removed the Triage 🩺 This issue needs to be triaged by a maintainer label Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants