Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nc/1oct/perf #1057

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Nc/1oct/perf #1057

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
cb5b958
Simplify path for handling utf8 errors
nfcampos Oct 2, 2024
32014aa
Remove unnecessary lock created for every BaseRun
nfcampos Oct 2, 2024
b3fd952
Fix
nfcampos Oct 2, 2024
e689e6d
Rm 20,000 benchmark
nfcampos Oct 2, 2024
41cacde
Merge branch 'main' into nc/1oct/perf
hinthornw Oct 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions python/bench/__main__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from pyperf._runner import Runner

Check notice on line 1 in python/bench/__main__.py

View workflow job for this annotation

GitHub Actions / benchmark

Benchmark results 

.........................................
create_5_000_run_trees: Mean +- std dev: 570 ms +- 40 ms
.........................................
create_10_000_run_trees: Mean +- std dev: 1.14 sec +- 0.05 sec
.........................................
dumps_class_nested_py_branch_and_leaf_200x400: Mean +- std dev: 774 us +- 14 us
.........................................
dumps_class_nested_py_leaf_50x100: Mean +- std dev: 27.2 ms +- 0.4 ms
.........................................
dumps_class_nested_py_leaf_100x200: Mean +- std dev: 112 ms +- 2 ms
.........................................
dumps_dataclass_nested_50x100: Mean +- std dev: 27.4 ms +- 0.4 ms
.........................................
WARNING: the benchmark result may be unstable
* the standard deviation (6.20 ms) is 11% of the mean (57.3 ms)

Try to rerun the benchmark with more runs, values and/or loops.
Run 'python -m pyperf system tune' command to reduce the system jitter.
Use pyperf stats, pyperf dump and pyperf hist to analyze results.
Use --quiet option to hide these warnings.

dumps_pydantic_nested_50x100: Mean +- std dev: 57.3 ms +- 6.2 ms
.........................................
WARNING: the benchmark result may be unstable
* the standard deviation (27.4 ms) is 14% of the mean (198 ms)

Try to rerun the benchmark with more runs, values and/or loops.
Run 'python -m pyperf system tune' command to reduce the system jitter.
Use pyperf stats, pyperf dump and pyperf hist to analyze results.
Use --quiet option to hide these warnings.

dumps_pydanticv1_nested_50x100: Mean +- std dev: 198 ms +- 27 ms

Check notice on line 1 in python/bench/__main__.py

View workflow job for this annotation

GitHub Actions / benchmark

Comparison against main 

+------------------------------------+---------+-----------------------+
| Benchmark                          | main    | changes               |
+====================================+=========+=======================+
| dumps_pydanticv1_nested_50x100     | 215 ms  | 198 ms: 1.09x faster  |
+------------------------------------+---------+-----------------------+
| dumps_pydantic_nested_50x100       | 61.9 ms | 57.3 ms: 1.08x faster |
+------------------------------------+---------+-----------------------+
| dumps_dataclass_nested_50x100      | 28.0 ms | 27.4 ms: 1.02x faster |
+------------------------------------+---------+-----------------------+
| dumps_class_nested_py_leaf_100x200 | 114 ms  | 112 ms: 1.02x faster  |
+------------------------------------+---------+-----------------------+
| dumps_class_nested_py_leaf_50x100  | 27.6 ms | 27.2 ms: 1.01x faster |
+------------------------------------+---------+-----------------------+
| Geometric mean                     | (ref)   | 1.03x faster          |
+------------------------------------+---------+-----------------------+

Benchmark hidden because not significant (3): create_5_000_run_trees, create_10_000_run_trees, dumps_class_nested_py_branch_and_leaf_200x400
Ignored benchmarks (1) of out/main.json: create_20_000_run_trees

from bench.create_run_tree import create_run_trees
from bench.dumps_json import (
Expand All @@ -25,11 +25,6 @@
create_run_trees,
10_000,
),
(
"create_20_000_run_trees",
create_run_trees,
10_000,
),
(
"dumps_class_nested_py_branch_and_leaf_200x400",
lambda x: _dumps_json({"input": x}),
Expand Down
17 changes: 2 additions & 15 deletions python/langsmith/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -268,12 +268,6 @@ def _serialize_json(obj: Any) -> Any:
return str(obj)


def _elide_surrogates(s: bytes) -> bytes:
pattern = re.compile(rb"\\ud[89a-f][0-9a-f]{2}", re.IGNORECASE)
result = pattern.sub(b"", s)
return result


def _dumps_json_single(
obj: Any, default: Optional[Callable[[Any], Any]] = None
) -> bytes:
Expand All @@ -289,18 +283,11 @@ def _dumps_json_single(
except TypeError as e:
# Usually caused by UTF surrogate characters
logger.debug(f"Orjson serialization failed: {repr(e)}. Falling back to json.")
result = json.dumps(
return json.dumps(
obj,
default=_simple_default,
ensure_ascii=True,
).encode("utf-8")
try:
result = orjson.dumps(
orjson.loads(result.decode("utf-8", errors="surrogateescape"))
)
except orjson.JSONDecodeError:
result = _elide_surrogates(result)
return result
).encode("utf-8", "ignore")


def _dumps_json(obj: Any, depth: int = 0) -> bytes:
Expand Down
Loading