caching strategy

[juliangruber]

@MarshallOfSound very well analyzed:

From the looks of it, the cache expires every 15 minutes, that means we have a maximum number of apps as 1250 (5000 rate limit / 4 requests per hour). According to apps-using-electron at the time of that packes generation (a while ago) there were 13455 repos on github that used Electron as a dependency. That number is probably way up now so we're probably gonna need a SUPER TOKEN from GH 😉 @zeke

[juliangruber]

we might be able to slightly improve the number by at the end of each token quota period fetching as many extra repos as possible, and pre fill the cache. That's not going to be of help though if we don't have any requests left anyway.

[juliangruber]

Can we use the GraphQL api to update many repos at once?

[mathieudutour]

How about asking for the dev to add a webhook for the release event pointing to update.electronjs.org and be a lot more aggressive for the caching? Like 2 or 3 hours?

[zeke]

That's an interesting idea, @mathieudutour. More work on the part of the app developer, but could be worth it for scalability.

[mathieudutour]

Indeed, might be a little bit more work (even tho it takes literally 30s to add a webhook) but at the same time, the cache would be invalidated instantly (instead of potentially having to wait 15min) so it’s a win-win

[juliangruber]

I like that! Seems like we could add it next, and then make this simpler by turning this into a GitHub app maybe? Also, with webhooks, the cache can expire never I think.

[mathieudutour]

If it’s a requirement to use this service, yes it could never expire. But we could also have a mix: if you want the cache to update directly when there is an update, then you need to add a webhook. But maybe we want to make it super simple to start and only have it as an optional (but strongly recommended) step

[MarshallOfSound]

We could also make a small FE for users to set up their repo's.

GH API has a webhook endpoint so if we oAuth in we can set it all up for them:

https://developer.github.com/v3/repos/hooks/