Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(perf): reduce parser size #356

Merged
merged 10 commits into from
Nov 23, 2024
Merged

refactor(perf): reduce parser size #356

merged 10 commits into from
Nov 23, 2024

Conversation

tris203
Copy link
Contributor

@tris203 tris203 commented Nov 23, 2024

Starting Stats

cat src/parser.c | rg "#define.*STATE"
#define STATE_COUNT 10002
#define LARGE_STATE_COUNT 4750

binary_expression 2040
member_access_expression 1694
range_expression 1614
conditional_expression 1608
is_expression 1596
is_pattern_expression 1584
switch_expression 1578
conditional_access_expression 1573
with_expression 1554
as_expression 1544
element_access_expression 1518
postfix_unary_expression 1517
invocation_expression 1517
lambda_expression 1416
record_declaration 399
struct_declaration 282
for_statement 260
qualified_name 238
recursive_pattern 236
ref_type 228

33836 src/parser.c

After Stats

#define STATE_COUNT 7556
#define LARGE_STATE_COUNT 3816

binary_expression 1752
member_access_expression 1406
range_expression 1326
conditional_expression 1320
is_expression 1308
is_pattern_expression 1296
conditional_access_expression 1285
as_expression 1256
with_expression 1236
switch_expression 1236
element_access_expression 1230
postfix_unary_expression 1229
invocation_expression 1229
qualified_name 241
recursive_pattern 233
cast_expression 218
ref_type 177
_lambda_expression_init 171
assignment_expression 157
lambda_expression 150

26832 src/parser.c

20.7% reduction in size
from 33mb to 26.8mb

@tris203 tris203 marked this pull request as ready for review November 23, 2024 10:52
@tris203 tris203 changed the title perf: refactor to reduce parser size refactor(perf): reduce parser size Nov 23, 2024
Copy link
Member

@amaanq amaanq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work @tris203!

grammar.js Outdated Show resolved Hide resolved
@tris203
Copy link
Contributor Author

tris203 commented Nov 23, 2024

Do I need to address the ci failure? @amaanq

@amaanq
Copy link
Member

amaanq commented Nov 23, 2024

Do I need to address the ci failure? @amaanq

No, it's just 1 failing file. I should probably pin those repos to test on, since this grammar is a little more fragile than others. The issue is also not too hard to fix (just preprocs in attribute list), so I'll follow up with a fix for it.

@amaanq amaanq merged commit ec73575 into tree-sitter:master Nov 23, 2024
1 of 4 checks passed
@amaanq
Copy link
Member

amaanq commented Nov 23, 2024

thanks a ton @tris203! I think once we handle reserved identifiers better upstream, this parser will shrink another substantial amount 🙂

@tris203 tris203 deleted the perf branch November 23, 2024 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants