vimPlugins: Add description attribute from GitHub metadata.

[progval]

Motivation for this change

Adding the missing metadata attribute (for searches, etc.)

Things done

Changed the update.py to output it, and recreated generated.nix.

There are no changes to the derivations.

[jonringer]

@progval thanks for opening your first nixpkgs PR :)

[jonringer]

not entirely sure why, but the description line is missing a space:

meta.description ="Vim configuration for Zig";

should be

meta.description = "Vim configuration for Zig";

[progval]

oh yeah I committed an older version of the file. It's fixed now

[jonringer]

diff LGTM
1 package is getting updated, but I think that's acceptable.

just want to hear from some others

[timokau]

Great idea, this would be nice to have. I do have concerns about the rate limit though.

[timokau]

The script previously explicitly avoided the GitHub API because of the rate limiting. Requiring every contributor to generate & set an oauth token is a significant change. I'm not entirely sure its worth the benefit.

[timokau]

Also if we do decide to require an OAUTH token, it should be passed in through the environment instead of hard-coded into the script (os.environ.get("GH_TOKEN") with some appropriate error message when it is not specified).

[SuperSandro2000]

The script previously explicitly avoided the GitHub API because of the rate limiting. Requiring every contributor to generate & set an oauth token is a significant change. I'm not entirely sure its worth the benefit.

You needed to provide one already anyway or otherwise you would get most likely rate limited.

[progval]

Yeah, that's a concern.
But it shouldn't be too much of an issue unless generated.nix is recreated from scratch.

Btw, when generating it from scratch, I also got rate-limited on commits.atom, and needed to change the user-agent to get around it

[timokau]

But it shouldn't be too much of an issue unless generated.nix is recreated from scratch.

That happens a lot though. Every time we update the vim plugins, which often happens when someone adds a new plugin. Requiring an oauth token increases the friction, meaning it will happen less often.

I also got rate-limited on commits.atom

I added the backoff-retry mechanism at some point to work around that. Doesn't that work anymore?

The advantage is that the rate limit is much softer than the one for the API. A backoff of ~10 seconds may already help. The API has a strict hourly rate limit (60 unauthenticated requests / hour), so just waiting it out is not really an option.

I think the best way forward would be to teach nixpkgs-update about vim plugins (#83119 (comment)). Then individual contributors would only be concerned with new additions, which should fit into the 60 request limits.

As long as vim plugin updates are manual, I'd prefer to keep the friction as low as possible so that people will keep doing it.

[progval]

I added the backoff-retry mechanism at some point to work around that. Doesn't that work anymore?

not with the default value of --proc (and probably only when git repositories are already cached locally), I had to lower it and try a couple of times.
Or maybe it's because I already did a bunch of requests before doing a full run, or GitHub doesn't like my IP range, who knows

As long as vim plugin updates are manual, I'd prefer to keep the friction as low as possible so that people will keep doing it.

Understandable. Should I close this PR?

[timokau]

I added the backoff-retry mechanism at some point to work around that. Doesn't that work anymore?

not with the default value of --proc (and probably only when git repositories are already cached locally), I had to lower it and try a couple of times.
Or maybe it's because I already did a bunch of requests before doing a full run, or GitHub doesn't like my IP range, who knows

That's a shame. The way we currently do things definitely isn't idea. Once we have nixpkgs-update do the package set updates, we could also start using the proper API for everything else the script does.

As long as vim plugin updates are manual, I'd prefer to keep the friction as low as possible so that people will keep doing it.

Understandable. Should I close this PR?

I have my concerns, but I don't want to block this PR alone. I'm curious what @jonringer, @teto or @evanjs think.

Another approach would be to conditionally only fetch descriptions if the oauth token is set. That would mean that the descriptions would sometimes be available, sometimes not (depending on who last made the update). Might be confusing.

If you do want to invest some time into improving the vim plugin infrastructure on nix, consider working on the nixpkgs-update integration. @ryantm has previously said that he would be open to contributions there.

[progval]

I don't really have the time to work on that, sorry :/

[Mic92]

I added the backoff-retry mechanism at some point to work around that. Doesn't that work anymore?

not with the default value of --proc (and probably only when git repositories are already cached locally), I had to lower it and try a couple of times.
Or maybe it's because I already did a bunch of requests before doing a full run, or GitHub doesn't like my IP range, who knows

That's a shame. The way we currently do things definitely isn't idea. Once we have nixpkgs-update do the package set updates, we could also start using the proper API for everything else the script does.

As long as vim plugin updates are manual, I'd prefer to keep the friction as low as possible so that people will keep doing it.

Understandable. Should I close this PR?

I have my concerns, but I don't want to block this PR alone. I'm curious what @jonringer, @teto or @evanjs think.

Another approach would be to conditionally only fetch descriptions if the oauth token is set. That would mean that the descriptions would sometimes be available, sometimes not (depending on who last made the update). Might be confusing.

If you do want to invest some time into improving the vim plugin infrastructure on nix, consider working on the nixpkgs-update integration. @ryantm has previously said that he would be open to contributions there.

I think we actually need to switch to GITHUB_TOKEN because github also sets tight rate limits when cloning repositories or fetching rss feeds, which breaks parallel downloads. If we get descriptions that's another plus making having to generate a token.

[timokau]

We probably need to switch at some point, but I think currently at least without any parallelism the update still works right?

Being able to use the oauth token would be much nicer, but I do think it would decrease the amount of people willing to do an update.

[Mic92]

We probably need to switch at some point, but I think currently at least without any parallelism the update still works right?

Being able to use the oauth token would be much nicer, but I do think it would decrease the amount of people willing to do an update.

It might be bearable if it was easy to setup i.e. having to run a one liner. After all everyone has a github account anyway when contributing to nixpkgs.

[timokau]

Yeah my concern is less that people cannot do it, its just that it adds just enough friction that they won't. Now you either have to store your oauth token somewhere (and do that securely) or regenerate it every time.

Still I'm not completely opposed to this, since the status quo is far from perfect too. I don't know. If you want to move this forward, go ahead :)

[stale]

I marked this as stale due to inactivity. → More info

[teto]

an interesting thing to have. Maybe without token, it could retreive the existing nix description or leave the field empty.

[stale]

I marked this as stale due to inactivity. → More info