Build analytics into the template

[Keyrxng]

Investigating this issue I built a minimal Octokit Proxy to capture access and call counts.

We should built something like this with a toggle maybe into the template especially if we are going to restrict plugin devs to a % of our the hourly call count quota.

We should consider adding support for detecting a secondary rate limit being hit as this is usually the one we hit.

context.octokit is the sole octokit object throughout plugins so it makes it very easily to setup decent analytics through a Proxy.

function octoProxy(octokit: Octokit) {
  const callCount = new Map<string, number>();

  function createProxy(target: any, path: string): any {
    return new Proxy(target, {
      get(target, prop, receiver) {
        const value = Reflect.get(target, prop, receiver);
        const propPath = path ? `${path}.${String(prop)}` : String(prop);

        if (typeof value === 'function') {
          return function (...args: any[]) {
            const count = callCount.get(propPath) || 0;
            callCount.set(propPath, count + 1);
            return value.apply(this, args);
          };
        } else if (typeof value === 'object' && value !== null) {
          return createProxy(value, propPath);
        } else {
          return value;
        }
      },
    });
  }

  const proxy = createProxy(octokit, '');

  return { proxy, callCount };
}

[Keyrxng]

@0x4007 I think this should be given a higher priority and priced for at least a day.

Pros:

• prevent new feature requests in plugins merging a feature that would likely break things
• provides insight into how we can optimize our core plugin call limits
• all plugins could return this standardized output to the kernel and it can give warnings dynamically via telegram that a particular plugin is consuming too much etc. This would allow for per-hour calcs too

Limits have tiers so for V1 we can cover requests-per-minute etc but per-hour would involve calculating how often the plugin will likely run which would pretty difficult to implement.

• We should implement this in all of our plugins and create a standardized output for action runs and for worker logs. This ensures future feature requests into all plugins are protected and the dev is notified during development.
• Some endpoints, like the search endpoints, have more restrictive limits. For more information about these endpoints, see "REST API endpoints for rate limits.
• different rate limits to consider for both REST, GQL and per-endpoint potentially
• it involves a test suite to confirm it works

Make too many requests per minute. No more than 90 seconds of CPU time per 60 seconds of real time is allowed. No more than 60 seconds of this CPU time may be for the GraphQL API. You can roughly estimate the CPU time by measuring the total response time for your API requests.
Make too many requests that consume excessive compute resources in a short period of time.
Create too much content on GitHub in a short amount of time. In general, no more than 80 content-generating requests per minute and no more than 500 content-generating requests per hour are allowed. Some endpoints have lower content creation limits. Content creation limits include actions taken on the GitHub web interface as well as via the REST API and GraphQL API.