Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/main' into series-generic
Browse files Browse the repository at this point in the history
  • Loading branch information
EdAbati committed Nov 26, 2024
2 parents eea19c6 + 73e67c7 commit 3a14e1d
Show file tree
Hide file tree
Showing 58 changed files with 2,065 additions and 728 deletions.
4 changes: 3 additions & 1 deletion .github/workflows/check_docs_build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,9 @@ jobs:
- name: local-install
run: uv pip install -e . --system
- name: check-no-errors
run: python -m mkdocs build > output.txt 2>&1
run: |
python -m mkdocs build 2>&1 | tee output.txt
cat output.txt
- name: assert-no-errors
run: python utils/check_for_no_build_errors.py
- name: strict-docs-build
Expand Down
6 changes: 4 additions & 2 deletions .github/workflows/downstream_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,8 @@ jobs:
. .venv/bin/activate
uv pip uninstall narwhals
uv pip install -e ./..
# temporary pin
uv pip install "altair<5.5"
- name: show-deps
run: |
cd marimo
Expand Down Expand Up @@ -184,8 +186,8 @@ jobs:
run: |
uv pip uninstall narwhals --system
uv pip install -e . --system
# temporarily pin websockets to get CI green
uv pip install "websockets<14.0" --system
# temporarily pin ollama to get CI green
uv pip install "ollama<0.4.0" --system
- name: show-deps
run: uv pip freeze
- name: Run `make narwhals-test-integration`
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ Join the party!
- [altair](https://github.com/vega/altair/)
- [marimo](https://github.com/marimo-team/marimo)
- [panel-graphic-walker](https://github.com/panel-extensions/panel-graphic-walker)
- [plotly](https://plotly.com)
- [pymarginaleffects](https://github.com/vincentarelbundock/pymarginaleffects)
- [py-shiny](https://github.com/posit-dev/py-shiny)
- [rio](https://github.com/rio-labs/rio)
Expand Down
2 changes: 2 additions & 0 deletions docs/api-reference/expr.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,13 @@
- pipe
- quantile
- replace_strict
- rolling_mean
- rolling_sum
- round
- sample
- shift
- sort
- skew
- std
- sum
- tail
Expand Down
1 change: 1 addition & 0 deletions docs/api-reference/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
- [narwhals.Expr.name](expr_name.md)
- [narwhals.Expr.str](expr_str.md)
- [narwhals.GroupBy](group_by.md)
- [narwhals.LazyGroupBy](lazy_group_by.md)
- [narwhals.LazyFrame](lazyframe.md)
- [narwhals.Schema](schema.md)
- [narwhals.Series](series.md)
Expand Down
9 changes: 9 additions & 0 deletions docs/api-reference/lazy_group_by.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# `narwhals.LazyGroupBy`

::: narwhals.group_by.LazyGroupBy
handler: python
options:
members:
- agg
show_source: false
show_bases: false
2 changes: 2 additions & 0 deletions docs/api-reference/series.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,13 +52,15 @@
- quantile
- rename
- replace_strict
- rolling_mean
- rolling_sum
- round
- sample
- scatter
- shape
- shift
- sort
- skew
- std
- sum
- tail
Expand Down
130 changes: 18 additions & 112 deletions docs/api-reference/typing.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,120 +3,26 @@
Narwhals comes fully statically typed. In addition to `nw.DataFrame`, `nw.Expr`,
`nw.Series`, `nw.LazyFrame`, we also provide the following type hints:

## `DataFrameT`
A `TypeVar` bound to `nw.DataFrame`. Use this when you have a function which
accepts a `nw.DataFrame` and returns a `nw.DataFrame` backed by the same backend, for example:

```python
import narwhals as nw
from narwhals.typing import DataFrameT


@nw.narwhalify
def func(df: DataFrameT) -> DataFrameT:
return df.with_columns(c=df["a"] + 1)
```

## `Frame`

Either a `nw.DataFrame` or `nw.LazyFrame`. Use this if your function can work on
either and your function doesn't care about its backend, for example:

```python
import narwhals as nw
from narwhals.typing import Frame


@nw.narwhalify
def func(df: Frame) -> list[str]:
return df.columns
```

## `FrameT`
A `TypeVar` bound to `Frame`. Use this if your function accepts either `nw.DataFrame`
or `nw.LazyFrame` and returns an object backed by the same backend, for example:

```python
import narwhals as nw
from narwhals.typing import FrameT


@nw.narwhalify
def func(df: FrameT) -> FrameT:
return df.with_columns(c=nw.col("a") + 1)
```

## `IntoDataFrame`
An object which can be converted to `nw.DataFrame` (e.g. `pd.DataFrame`, `pl.DataFrame`).
Use this if your function accepts a narwhalifiable object but doesn't care about its backend:

```python
from __future__ import annotations

import narwhals as nw
from narwhals.typing import IntoDataFrame


def func(df_native: IntoDataFrame) -> tuple[int, int]:
df = nw.from_native(df_native, eager_only=True)
return df.shape
```

## `IntoDataFrameT`
A `TypeVar` bound to `IntoDataFrame`. Use this if your function accepts
a function which can be converted to `nw.DataFrame` and returns an object of the same
class:

```python
import narwhals as nw
from narwhals.typing import IntoDataFrameT


def func(df_native: IntoDataFrameT) -> IntoDataFrameT:
df = nw.from_native(df_native, eager_only=True)
return nw.to_native(df.with_columns(c=df["a"] + 1))
```

## `IntoExpr`
Use this to mean "either a Narwhals expression, or something
which can be converted into one". For example, `exprs` in `DataFrame.select` is
typed to accept `IntoExpr`, as it can either accept a `nw.Expr` (e.g. `df.select(nw.col('a'))`)
or a string which will be interpreted as a `nw.Expr`, e.g. `df.select('a')`.

## `IntoFrame`
An object which can be converted to `nw.DataFrame` or `nw.LazyFrame`
(e.g. `pd.DataFrame`, `pl.DataFrame`, `pl.LazyFrame`). Use this if your function can accept
an object which can be converted to either `nw.DataFrame` or `nw.LazyFrame` and it doesn't
care about its backend:

```python
import narwhals as nw
from narwhals.typing import IntoFrame


def func(df_native: IntoFrame) -> list[str]:
df = nw.from_native(df_native)
return df.columns
```

## `IntoFrameT`
A `TypeVar` bound to `IntoFrame`. Use this if your function accepts an
object which is convertible to `nw.DataFrame` or `nw.LazyFrame` and returns an object
of the same type:

```python
import narwhals as nw
from narwhals.typing import IntoFrameT


def func(df_native: IntoFrameT) -> IntoFrameT:
df = nw.from_native(df_native)
return nw.to_native(df.with_columns(c=nw.col("a") + 1))
```
::: narwhals.typing
handler: python
options:
members:
- DataFrameT
- Frame
- FrameT
- IntoDataFrame
- IntoDataFrameT
- IntoExpr
- IntoFrame
- IntoFrameT
- IntoSeries
- IntoSeriesT
show_source: false
show_bases: false

## `nw.narwhalify`, or `nw.from_native`?

Although the former is more readable, the latter is better at preserving type hints.
Although some people find the former more readable, the latter is better at preserving type hints.

Here's an example:
```python
Expand Down Expand Up @@ -154,5 +60,5 @@ Success: no issues found in 1 source file
In the first case, mypy can infer that `df` is a `polars.DataFrame`. In the second case, it can't.

If you want to make the most out of type hints and preserve them as much as possible, we recommend
`nw.from_native` and `nw.to_native` - otherwise, `nw.narwhalify`. Type hints will still be respected
`nw.from_native` and `nw.to_native`. Type hints will still be respected
inside the function body if you type the arguments.
2 changes: 1 addition & 1 deletion docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ To verify the installation, start the Python REPL and execute:
```python
>>> import narwhals
>>> narwhals.__version__
'1.14.1'
'1.14.2'
```
If you see the version number, then the installation was successful!

Expand Down
2 changes: 2 additions & 0 deletions docs/requirements-docs.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
black # required by mkdocstrings_handlers
jinja2
duckdb
# Hopefully temporary until https://github.com/mkdocstrings/mkdocstrings/issues/716
git+https://github.com/MarcoGorelli/griffe.git@no-overloads
markdown-exec[ansi]
mkdocs
mkdocs-autorefs
Expand Down
8 changes: 4 additions & 4 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ nav:
- api-reference/expr_name.md
- api-reference/expr_str.md
- api-reference/group_by.md
- api-reference/lazy_group_by.md
- api-reference/lazyframe.md
- api-reference/schema.md
- api-reference/series.md
Expand Down Expand Up @@ -95,9 +96,6 @@ plugins:
members_order: alphabetical
enable_inventory: true

extra_javascript:
- javascripts/extra.js

hooks:
- utils/generate_backend_completeness.py
- utils/generate_zen_content.py
Expand All @@ -122,8 +120,10 @@ markdown_extensions:
generic: true
extra_javascript:
- javascripts/katex.js
- javascripts/extra.js
- https://unpkg.com/katex@0/dist/katex.min.js
- https://unpkg.com/katex@0/dist/contrib/auto-render.min.js


extra_css:
- https://unpkg.com/katex@0/dist/katex.min.css
- https://unpkg.com/katex@0/dist/katex.min.css
2 changes: 1 addition & 1 deletion narwhals/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@
from narwhals.utils import maybe_reset_index
from narwhals.utils import maybe_set_index

__version__ = "1.14.1"
__version__ = "1.14.2"

__all__ = [
"Array",
Expand Down
13 changes: 8 additions & 5 deletions narwhals/_arrow/dataframe.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from __future__ import annotations

from itertools import chain
from typing import TYPE_CHECKING
from typing import Any
from typing import Iterable
Expand Down Expand Up @@ -465,19 +466,21 @@ def with_row_index(self, name: str) -> Self:
row_indices = pa.array(range(df.num_rows))
return self._from_native_frame(df.append_column(name, row_indices))

def filter(
self,
*predicates: IntoArrowExpr,
) -> Self:
def filter(self, *predicates: IntoArrowExpr, **constraints: Any) -> Self:
if (
len(predicates) == 1
and isinstance(predicates[0], list)
and all(isinstance(x, bool) for x in predicates[0])
and not constraints
):
mask = predicates[0]
else:
plx = self.__narwhals_namespace__()
expr = plx.all_horizontal(*predicates)
expr = plx.all_horizontal(
*chain(
predicates, (plx.col(name) == v for name, v in constraints.items())
)
)
# Safety: all_horizontal's expression only returns a single column.
mask = expr._call(self)[0]._native_series
return self._from_native_frame(self._native_frame.filter(mask))
Expand Down
18 changes: 18 additions & 0 deletions narwhals/_arrow/expr.py
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,9 @@ def n_unique(self) -> Self:
def std(self, ddof: int = 1) -> Self:
return reuse_series_implementation(self, "std", ddof=ddof, returns_scalar=True)

def skew(self: Self) -> Self:
return reuse_series_implementation(self, "skew", returns_scalar=True)

def cast(self, dtype: DType) -> Self:
return reuse_series_implementation(self, "cast", dtype)

Expand Down Expand Up @@ -468,6 +471,21 @@ def rolling_sum(
center=center,
)

def rolling_mean(
self: Self,
window_size: int,
*,
min_periods: int | None,
center: bool,
) -> Self:
return reuse_series_implementation(
self,
"rolling_mean",
window_size=window_size,
min_periods=min_periods,
center=center,
)

@property
def dt(self: Self) -> ArrowExprDateTimeNamespace:
return ArrowExprDateTimeNamespace(self)
Expand Down
Loading

0 comments on commit 3a14e1d

Please sign in to comment.