A github repo slows "update all packages"

[reagle]

Following the directions to install a dev plugin from github results in very slow package control.

In STealthy-and-haSTy/EnhancedSnippets/issues/7 @OdatNurd suggests its a PC issue.

macOS 13.4 22F66 arm64

[FichteFoll]

It's likely that you end up being rate-limited and there's nothing that PC can do about it by itself. However, you could provide PC with some credentials for GitHub's API in your preferences.

package_control/Package Control.sublime-settings

Lines 91 to 103 in cfaaeb5

	// Username and password support for private repositories, or for
	// bypassing anonymous user download limits on services.
	//
	// Each entry should be a key of the domain name, with a 2-element list
	// as the value, first the username and then the password. Examples of
	// common hosting platforms below:
	//
	// {
	// "api.github.com": ["username", "personal access token"],
	// "gitlab.com": ["username", "personal access token"],
	// "api.bitbucket.org": ["username", "app password"],
	// }
	"http_basic_auth": {},

[deathaxe]

FWIW: IIRC, basic auth doesn't work properly in PC 3.x (see: #1622).

That said, resolving release information from VCS based releases requires various API calls per package, which are performed one after another. Even if rate-limit is not hit, it just takes some time.

PC4.0 takes some meassures to reduce amount of API calls and runs some stuff in parallel to reduce delays, but finally it would require a complete rewrite applying asyncio or apply other complicated Promise architectures to solve the problem.

I don't think any of that will happen anytime soon, especially as PC4.0 claims support for ST3143+ and needs to stay fully compatible with python 3.3 for the forseeable time. The other reason is likelyness of hitting rate limits. Even authenticated users can only perform 5000 API calls per hour, which is not enough to resolve the whole default channel's repositories atm.

Such kind of performance issues is the main reason for existence of channels as those contain a cache for resolved packages. Unfortunatelly packagecontrol.io is currently the only channel provider, which implements the background crawler to fill those caches.

The only other solution would be to provide explicit releases via repository.json, but this would also require to either hand craft repositories after each release or create a deployment script doing so.

An example for such repositories is https://github.com/SublimeText/wbond-packages/blob/main/packages.json

[reagle]

This morning I'm seeing this problem, which might be related?

Package Control: Hit rate limit of 60 for api.github.com. Skipping all futher download requests for this domain.

[deathaxe]

This indicates you not being able to even see all packages from that repo.

There's nothing Package Control can do against hitting a repository hoster's API rate limits.

The only issue is basic auth not working for PC3.x Thus being restricted to lowest available rate limits. This issue is fixed in PC4.0