Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Prefix filter optimizations #7309

Merged
merged 4 commits into from
Oct 6, 2023
Merged

Prefix filter optimizations #7309

merged 4 commits into from
Oct 6, 2023

Conversation

Cali0707
Copy link
Member

Fixes #7305
Fixes #7308

Proposed Changes

  • Only copy the event attribute into a string with Sprintf if that attribute needs to be converted to a string
  • Only loop over the event attributes once, instead of twice
  • use int instead of string for eventfilter result types

Performance improvements after these changes:

benchmark                                                                                 old ns/op     new ns/op     delta
BenchmarkPrefixFilter/Creation:_Pass_with_prefix_match_of_id-8                            53.5          54.1          +1.07%
BenchmarkPrefixFilter/Run:_Pass_with_prefix_match_of_id-8                                 558           439           -21.34%
BenchmarkPrefixFilter/Creation:_Pass_with_prefix_match_of_all_context_attributes-8        89.6          89.7          +0.11%
BenchmarkPrefixFilter/Run:_Pass_with_prefix_match_of_all_context_attributes-8             1923          1194          -37.91%
BenchmarkPrefixFilter/Creation:_Pass_with_prefix_match_of_all_context_attributes#01-8     89.6          89.7          +0.11%
BenchmarkPrefixFilter/Run:_Pass_with_prefix_match_of_all_context_attributes#01-8          1929          1167          -39.50%
BenchmarkPrefixFilter/Creation:_No_pass_with_prefix_match_of_id_and_source-8              61.7          61.9          +0.26%
BenchmarkPrefixFilter/Run:_No_pass_with_prefix_match_of_id_and_source-8                   639           428           -32.99%

benchmark                                                                                 old allocs     new allocs     delta
BenchmarkPrefixFilter/Creation:_Pass_with_prefix_match_of_id-8                            1              1              +0.00%
BenchmarkPrefixFilter/Run:_Pass_with_prefix_match_of_id-8                                 5              4              -20.00%
BenchmarkPrefixFilter/Creation:_Pass_with_prefix_match_of_all_context_attributes-8        1              1              +0.00%
BenchmarkPrefixFilter/Run:_Pass_with_prefix_match_of_all_context_attributes-8             21             15             -28.57%
BenchmarkPrefixFilter/Creation:_Pass_with_prefix_match_of_all_context_attributes#01-8     1              1              +0.00%
BenchmarkPrefixFilter/Run:_Pass_with_prefix_match_of_all_context_attributes#01-8          21             15             -28.57%
BenchmarkPrefixFilter/Creation:_No_pass_with_prefix_match_of_id_and_source-8              1              1              +0.00%
BenchmarkPrefixFilter/Run:_No_pass_with_prefix_match_of_id_and_source-8                   5              4              -20.00%

benchmark                                                                                 old bytes     new bytes     delta
BenchmarkPrefixFilter/Creation:_Pass_with_prefix_match_of_id-8                            8             8             +0.00%
BenchmarkPrefixFilter/Run:_Pass_with_prefix_match_of_id-8                                 224           208           -7.14%
BenchmarkPrefixFilter/Creation:_Pass_with_prefix_match_of_all_context_attributes-8        8             8             +0.00%
BenchmarkPrefixFilter/Run:_Pass_with_prefix_match_of_all_context_attributes-8             571           448           -21.54%
BenchmarkPrefixFilter/Creation:_Pass_with_prefix_match_of_all_context_attributes#01-8     8             8             +0.00%
BenchmarkPrefixFilter/Run:_Pass_with_prefix_match_of_all_context_attributes#01-8          571           448           -21.54%
BenchmarkPrefixFilter/Creation:_No_pass_with_prefix_match_of_id_and_source-8              8             8             +0.00%
BenchmarkPrefixFilter/Run:_No_pass_with_prefix_match_of_id_and_source-8                   236           218           -7.63%

Pre-review Checklist

  • At least 80% unit test coverage
  • E2E tests for any new behavior
  • Docs PR for any user-facing impact
  • Spec PR for any new API feature
  • Conformance test for any change to the spec

Release Note

The prefix filter just got a whole lot faster!

Docs

@knative-prow knative-prow bot requested review from aliok and creydr September 27, 2023 17:54
@knative-prow knative-prow bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Sep 27, 2023
@Cali0707
Copy link
Member Author

Cali0707 commented Sep 27, 2023 •
edited
Loading

/cc @pierDipi @creydr @matzew

By the way, there is a behaviour change, however I think it is now more "correct". Before, if any of the attributes or values was an empty string, we would return that there was "NoFilter". However, I think that the more correct behaviour is to only return this "NoFilter" value if all of the attribute/value pairs have one or both be an empty string. WDYT? I can revert this behaviour change pretty easily if needed

As an example, before if I had a filter that looked like:

filters:
  - prefix:
    - type: "com"
    - id:

Then in this case, we would return "NoFilter" on every event. However, I think it makes more sense to actually check the result of filtering type and then return that instead.

@knative-prow knative-prow bot requested review from matzew and pierDipi September 27, 2023 17:55
@knative-prow knative-prow bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Sep 27, 2023
Cali0707
Cali0707 commented Sep 27, 2023
View reviewed changes
pkg/eventfilter/subscriptionsapi/prefix_filter.go Outdated
logger := logging.FromContext(ctx)
logger.Debugw("Performing a prefix match ", zap.Any("filters", filter.filters), zap.Any("event", event))
for k, v := range filter.filters {
if k == "" || v == "" {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we actually even need this check, as we validate that this is the case when we initialize the filter

@codecov
Copy link

codecov bot commented Sep 27, 2023 •
edited
Loading

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (ff9444d) 77.66% compared to head (8331495) 77.52%.
Report is 14 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #7309      +/-   ##
==========================================
- Coverage   77.66%   77.52%   -0.15%     
==========================================
  Files         250      250              
  Lines       13436    13529      +93     
==========================================
+ Hits        10435    10488      +53     
- Misses       2478     2516      +38     
- Partials      523      525       +2
Files Coverage Δ
pkg/apis/duck/v1/subscribable_types.go 100.00% <ø> (ø)
pkg/apis/eventing/v1/trigger_types.go 100.00% <ø> (ø)
pkg/apis/flows/v1/parallel_types.go 100.00% <ø> (ø)
pkg/apis/flows/v1/sequence_types.go 100.00% <ø> (ø)
pkg/apis/messaging/v1/subscription_types.go 66.66% <ø> (ø)
pkg/eventfilter/subscriptionsapi/prefix_filter.go 70.00% <40.00%> (ø)

... and 8 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Cali0707
Copy link
Member Author

/retest-required

@Cali0707
Make closer to suffix filter
8331495 
Signed-off-by: Calum Murray <[email protected]>
@Leo6Leo
Copy link
Member

Leo6Leo commented Sep 28, 2023

/lgtm

@knative-prow knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Sep 28, 2023
pierDipi
pierDipi approved these changes Oct 6, 2023
View reviewed changes
Copy link
Member

@pierDipi pierDipi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@knative-prow
Copy link

knative-prow bot commented Oct 6, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Cali0707, pierDipi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prow knative-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 6, 2023
@knative-prow knative-prow bot merged commit e40037b into knative:main Oct 6, 2023
27 of 32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pierDipi pierDipi pierDipi approved these changes

@aliok aliok Awaiting requested review from aliok

@creydr creydr Awaiting requested review from creydr

@matzew matzew Awaiting requested review from matzew

Assignees

@pierDipi pierDipi

@Leo6Leo Leo6Leo

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
3 participants
@Cali0707 @Leo6Leo @pierDipi