Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor/OIDC custos #16497

Merged
merged 1 commit into from
Oct 21, 2023
Merged

Refactor/OIDC custos #16497

merged 1 commit into from
Oct 21, 2023

Conversation

uwwint
Copy link
Contributor

@uwwint uwwint commented Aug 1, 2023

Refactoring of the whole OIDC code:

  • move IDP specifics into subclasses
  • create a factory
  • build a cache into the factory to reduce server load

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. Configure various OIDC providers
    2. Test the login and refresh token exchange still works

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

@github-actions github-actions bot added the area/auth Authentication and authorization label Aug 1, 2023
@github-actions github-actions bot added this to the 23.2 milestone Aug 1, 2023
Copy link
Member

@nuwang nuwang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a big improvement! Thanks for working on this.

lib/galaxy/authnz/custos_authnz.py Outdated Show resolved Hide resolved
lib/galaxy/authnz/custos_authnz.py Show resolved Hide resolved

def _fetch_well_known_oidc_config(self, well_known_uri):
def _load_config(self,headers : dict = None, params : dict = None):
self.config.well_known_oidc_config_uri = self._get_well_known_uri_from_url(self.config.provider)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sample config file says that the well known url can be overridden, but that does not appear to have been the case for a while. In fact, I'm wondering whether we need the url parameter at all, and it would be better to deprecate the url parameter in favour of the well_known_oidc_config_uri parameter, with some backward compatible code for converting the url to a well-known one.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lib/galaxy/authnz/custos_authnz.py Outdated Show resolved Hide resolved
@staticmethod
def GetCustosBasedAuthProvider (provider, oidc_config, oidc_backend_config, idphint=None):
# remove old entries (older than 15 minutes)
_CustosAuthBasedProvidersCache = filter(lambda x: datetime.now() - x.created_at < timedelta(minutes=15),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this to cache well-known info? If so, I think it changes rarely enough, if ever, that we can afford a restart if that info changes, so my personal preference would be to simplify further by removing the cache.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this will cache the well known config for 15 minutes. If you don't want the cache to expire, just remove line 512. However having a cache in here makes galaxy 'repair' itself after 15 minutes of a oidc well-known change.

Not caching at all is IMHO a bad idea. Due to the design of the code the well-known is fetched every request to galaxy (by a user with an OIDC account). As a result every request to galaxy causes a request to the .well-known... That should cause quite some server load and also stall the transaction while the request of .well-known is in progress.
We could further refactor to only fetch the .well-known upon refresh/authenticate/callback but I believe that is a more fragile design than simply caching it.

Copy link
Member

@nuwang nuwang Aug 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not suggesting reloading on every request. I'm suggesting that it be loaded once permanently (using a module level variable). 15 minute caching can be removed. Afaik, OIDC well known info does not change, and on the remote chance that it does, an admin can just trigger a process level restart.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To put it differently, if cache expiry has minimal benefit, it's better to remove it and simplify. If it does serve a purpose, it would be better to use a library like TTLCache from cachetools.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed the TTL

@nuwang
Copy link
Member

nuwang commented Aug 8, 2023

Please also look at linting failures.

@uwwint
Copy link
Contributor Author

uwwint commented Aug 11, 2023

Please also look at linting failures.

sorted.

@nuwang nuwang force-pushed the refactor/oidc-custos branch 2 times, most recently from 8826b21 to 13b450a Compare October 20, 2023 14:27
@nuwang nuwang force-pushed the refactor/oidc-custos branch from 13b450a to 0ae47b7 Compare October 20, 2023 20:01
Copy link
Member

@nuwang nuwang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these changes. Have also run the latest tests against this and they've passed, so it's looking good! Toolshed test failures should be unrelated.

@nuwang nuwang added the kind/refactoring cleanup or refactoring of existing code, no functional changes label Oct 21, 2023
@nuwang nuwang merged commit f9cecae into galaxyproject:dev Oct 21, 2023
47 of 49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/auth Authentication and authorization kind/refactoring cleanup or refactoring of existing code, no functional changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants