Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow global resync & reconcilerImpl informers to be filtered via filterfunc #1909

Closed
wants to merge 6 commits into from

Conversation

lberk
Copy link
Member

@lberk lberk commented Nov 16, 2020

related to #1764

There were two missing places where the informers from the shared cache we're not being filtered.

  1. in the globalresync functions (where the filterFunc being passed through was ~= return true
  2. the informer in the LeaderAwareFuncs.PromoteFunc which was marked with a TODO to allow filtering

The former has been changed to allow for a controllerImpl.GlobalResyncFilterFunc to be set (and if not, the former default of return true is passed to allow objects.

The latter was complicated to pass as a function, if done as a controller.Option, it would be too late for the closure in the initial reconcilerImpl. I've opted instead for a pkg/controller/controller.go variable that will store via the shared context and again -- if not set -- pass a default filter of return true filtering over the objects.

/cc @vaikas @n3wscott @matzew

@knative-prow-robot knative-prow-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Nov 16, 2020
@knative-prow-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: lberk
To complete the pull request process, please assign markusthoemmes after the PR has been reviewed.
You can assign the PR to them by writing /assign @markusthoemmes in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-cla google-cla bot added the cla: yes Indicates the PR's author has signed the CLA. label Nov 16, 2020
@codecov
Copy link

codecov bot commented Nov 16, 2020

Codecov Report

Merging #1909 (ca3d2a9) into main (615f17a) will decrease coverage by 0.33%.
The diff coverage is 52.63%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1909      +/-   ##
==========================================
- Coverage   69.42%   69.08%   -0.34%     
==========================================
  Files         207      209       +2     
  Lines        8739     8819      +80     
==========================================
+ Hits         6067     6093      +26     
- Misses       2402     2454      +52     
- Partials      270      272       +2     
Impacted Files Coverage Δ
.../injection-gen/generators/reconciler_controller.go 0.00% <0.00%> (ø)
controller/controller.go 87.19% <66.66%> (-1.70%) ⬇️
network/handlers/drain.go 82.50% <0.00%> (-5.74%) ⬇️
websocket/connection.go 92.91% <0.00%> (-3.81%) ⬇️
webhook/env.go 100.00% <0.00%> (ø)
network/network.go 100.00% <0.00%> (ø)
test/spoof/spoof.go 0.00% <0.00%> (ø)
reconciler/events.go 100.00% <0.00%> (ø)
test/upgrade/functions.go 100.00% <0.00%> (ø)
controller/testing/fake_stats_reporter.go 100.00% <0.00%> (ø)
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 615f17a...ca3d2a9. Read the comment docs.

controller/controller.go Outdated Show resolved Hide resolved
lberk added a commit to lberk/eventing that referenced this pull request Nov 16, 2020
controller/controller.go Outdated Show resolved Hide resolved
controller/controller.go Outdated Show resolved Hide resolved
controller/controller.go Outdated Show resolved Hide resolved
Impl.globalResyncFilterFunc -> do not export (as is set by controller)
Ensure FilteredGlobalResync actually runs filterFunc over passed
objects
Fix descriptions/comments
Change passing func(interfcae{}) bool, to filterFunc
@knative-prow-robot knative-prow-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 24, 2020
Copy link
Member

@julz julz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm apart from a couple comment typos

controller/controller.go Outdated Show resolved Hide resolved
controller/controller.go Outdated Show resolved Hide resolved
@lberk
Copy link
Member Author

lberk commented Nov 25, 2020

thanks for the reviews @julz, should be fixed at this point.

Copy link
Member

@julz julz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@knative-prow-robot knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label Nov 25, 2020
@@ -203,6 +207,7 @@ func NewImpl(ctx {{.contextContext|raw}}, r Interface{{if .hasClass}}, classValu

lister := {{.type|lowercaseSingular}}Informer.Lister()

filterFunc := {{.controllerGetFilterFunc|raw}}(ctx)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather see this come through the controller.Options, but the ordering and capture make this awkward.

I think that if you have filterFunc := noFilter up here, and optionally overwrite it below the capture should work out correctly though?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in fact the rest of the things already seem to be doing this, so you are pretty close 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what @benmoss did in #1909 too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#2170 rather 😄 . Just clicked your link and momentarily was very confused

@knative-prow-robot
Copy link
Contributor

@lberk: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@knative-prow-robot knative-prow-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 16, 2021
Base automatically changed from master to main March 8, 2021 17:40
@knative-prow-robot
Copy link
Contributor

@lberk: The following test failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
pull-knative-pkg-build-tests ca3d2a9 link /test pull-knative-pkg-build-tests

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@markusthoemmes
Copy link
Contributor

Which state is this one in? Seems like it fell between the cracks @lberk ?

@lberk
Copy link
Member Author

lberk commented Apr 8, 2021

@markusthoemmes this is still a problem in eventing. I was asked for further justification that this solves the root cause of the issue, I haven't had time to return to it since.

@dprotaso
Copy link
Member

I'm going to close this out since it's been a number of months - feel free reopen

@dprotaso dprotaso closed this Aug 30, 2021
@mattmoor
Copy link
Member

I’m pretty sure Options now has PromoteFilterFunc…

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes Indicates the PR's author has signed the CLA. lgtm Indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants