🚀GitHub Copilot Metrics API GA release now available

[Akash1134]

🚀 Now Available: GitHub Copilot Metrics API in General Availability! 🚀

We’re thrilled to introduce the GA release of the GitHub Copilot Metrics API, now open to all GitHub Copilot Business and GitHub Copilot Enterprise users. This release is packed with features to give you greater visibility into Copilot's impact across your development workflow. Ready to dive into the data?

---

📊 What’s the GitHub Copilot Metrics API?

Imagine having detailed insights into how Copilot is being used across your enterprise. The Metrics API brings this to life by providing you with data on Copilot's usage—across teams, organizations, and individual features. Perfect for tracking trends, understanding usage patterns, and integrating with your own dashboards, the Metrics API helps you see where Copilot is adding value and how you can optimize its use.

🎉 What's New in the GA Release?

Here’s a peek at what’s included:

• New metrics for Pull Request summaries and Copilot Chat on GitHub.com.
• Clarity on code completions and Copilot Chat usage in IDEs.
• User engagement summaries updated daily.
• Custom model slicing support, coming soon!
• Data is aggregated by enterprise, organization, and team.
• Up to 28 days of historical data available for trend analysis.

With data loaded daily (UTC) and summarized by day, you can view patterns in real-time and make more data-informed decisions.

🤔 Will My Current Reporting Be Affected?

To make the transition smooth, we’ve kept the Beta route online through the end of the year. This means any current reporting tied to the Beta API will continue without interruption, while you explore the newly revised schema and expanded metrics in the GA version.

📚 Resources to Get Started

• Docs: Ready to get started? Explore the API documentation for the full breakdown of metrics and schema.

---

💬 Your Turn!

How are you planning to use the Copilot Metrics API? What specific metrics would give you the best insight into Copilot's impact on your projects? Whether it’s tracking usage, surfacing adoption trends, or something else entirely, we’d love to hear your ideas. Drop your thoughts below and let’s shape the future of Copilot analytics together! 👇

[charliew33]

@Akash1134 This looks great, would love to see more detailed metrics about Copilot usage and engagement.

Question: it looks like this is disabled for my org, do I need an enterprise owner to enable? The docs are a little light but it sounds like if I get an enterprise owner to enable the Copilot Metrics API for the enterprise I can then get them for my org?

To access this endpoint, the Copilot Metrics API access policy must be enabled or set to "no policy" for the enterprise within GitHub settings. Only owners and billing managers for the enterprise that contains the enterprise team can view Copilot metrics for the enterprise team.

[samus-aran]

Hi @charliew33,

Yes, you would need to enable at the Enterprise level. 👍

[detty-down]

What specific detailed metrics about Copilot usage and engagement are you interested in?

[dpewaal-iba]

Do you know who the enterprise owner is for your organization?

[mno-ly]

Are there other team members in your org who also need access to these metrics?

[DevOps-zhuang]

Thanks for the GA, it looks more detailed, and indeeed more complex :-)
like for the previous usage route, it displays the aggregate code summary data for each day, then break down it by editor/language.

day": "2023-10-15",
    **"total_suggestions_count": 5000,
    "total_acceptances_count": 3000,
    "total_lines_suggested": 7000,
    "total_lines_accepted": 3500,**
    "total_active_users": 15,
    "total_chat_acceptances": 45,
    "total_chat_turns": 350,
    "total_active_chat_users": 8,

for the GA metric route, I was doubting whether we still need an aggregate high-leve total_code summary for each day, maybe per day per language? currently, for the copilot_ide_code_completions part, it only has one direct KV which is "total_engaged_users": 20. in order to get the total_code_* metircs as before, we have to check the breakdown data, which is by editor/models/language, the merge them togeterh. based on my observation, the total_code_* data per day per language is a strong requirement. so it is better to add it. thanks for your consideration.

like currently, for the lanuages part under "copilot_ide_code_completions", it only display the total_engaged_users part, is it possible to add total_code_* part for each language? thanks.

"copilot_ide_code_completions": {
"total_engaged_users": 20,
"languages": [
{
"name": "python",
"total_engaged_users": 10
},
{
"name": "ruby",
"total_engaged_users": 10
}
],

[DevOps-zhuang]

and the other consideration is about the 'engaged user' and 'active users', to be honest, I am not quite understand the difference, and I am still oberving and check the difference.

[rhlbhatnagar]

Hey,
While using this API I get:

"Your organization has disabled Copilot Metrics API access. Enable it in GitHub settings to access this endpoint."

In github settings I'm unable to enable access to the metrics API as admin. I see a link to a doc on how for github pre release license terms
https://docs.github.com/en/site-policy/github-terms/github-pre-release-license-terms

But don't actually see how to get my enterprise access to these pre release features.

[veverkap]

You need to go into your enterprise settings first (https://github.com/enterprises/{YOUR_ENTERPRISE_SLUG}/settings/copilot?tab=policies) and chose either "No Policy" or "Enabled"

If you chose "Enabled", then the policy is enabled for all organizations and they cannot change the setting at the organizaiton level.

If you chose "No Policy", then the policy is configurable at the organization level (https://github.com/organizations/{YOUR_ORGANIZATION}/settings/copilot/policies

[rajeshharne]

Accessing GitHub Copilot Usage API with a Business Account – Is it supported?

Question: @Akash1134 / Team

Hello everyone,

I’m looking to track GitHub Copilot usage metrics through the API for our organization, which currently has a GitHub Business license. Specifically, I’d like to use the following endpoint:

/enterprises/{enterprise}/copilot/usage

Before diving in, I wanted to confirm: does the GitHub Business license provide access to the Copilot usage API, or is this feature exclusively available for GitHub Enterprise accounts? If it’s enterprise-only, are there any alternative methods for tracking Copilot usage in a Business account?

Thanks for any insights or guidance!

[veverkap]

Yes, a Copilot Business subscription with users is all that is needed.

[karpikpl]

how to access new metrics?
the docs say

To make the transition smooth, we’ve kept the Beta route online through the end of the year.
what's the difference between beta route and new route?

I get old data for
x-github-api-version-selected: 2022-11-28
GET https://api.github.com/orgs//copilot/usage

also the link to the documentation doesn't work

[DevOps-zhuang]

copilot/usage is the old usage API. and copilot/metrics is the new one

[karpikpl]

lol, thanks. I don't know how I missed that :)

[DevOps-zhuang]

we found some data issues, the total_suggested is also 0. though there are accepated, can you double check the data quality?
the data is as below:
"copilot_ide_code_completions": {
"editors": [
{
"name": "vscode",
"models": [
{
"name": "default",
"languages": [
{
"name": "typescript",
"total_engaged_users": 1,
"total_code_acceptances": 39,
"total_code_suggestions": 0,
"total_code_lines_accepted": 22,
"total_code_lines_suggested": 0
}
],
"is_custom_model": false,
"total_engaged_users": 1
}
],
"total_engaged_users": 1
}
],
"languages": [
{
"name": "typescript",
"total_engaged_users": 1
}
],
"total_engaged_users": 1
}
},

[DevOps-zhuang]

and "copilot_ide_code_completions": {
"editors": [
{
"name": "JetBrains",
"models": [
{
"name": "default",
"languages": [
{
"name": "java",
"total_engaged_users": 1,
"total_code_acceptances": 8,
"total_code_suggestions": 0,
"total_code_lines_accepted": 19,
"total_code_lines_suggested": 0
}
],
"is_custom_model": false,
"total_engaged_users": 1
}
],
"total_engaged_users": 1
}
],

[karpikpl]

I think we've seen similar issues in sample data and API data before this was GA :(

[DevOps-zhuang]

I think we've seen similar issues in sample data and API data before this was GA :(

Yes, we see different issues. hope it can be solved ASAP. since it is GAed, good quality is hoped.

[levanvunam]

I tried to use the billing/seats endpoint and seems like the "assigning_team" of member is always missing, trying to export csv from the interface also show no team, anyone has similar problem?

[DevOps-zhuang]

I tried to use the billing/seats endpoint and seems like the "assigning_team" of member is always missing, trying to export csv from the interface also show no team, anyone has similar problem?

I met the same issue before, and then found out that it is because the 'assigning_team' is the team where the user got the copilot license, that means, we need to assign the license to the individual by teams, not to the indiviual directly. ony with this way, you can get the assigning_team, it is not the team the copilot seats belong to... and I do think this design is some strange, if it can be updated to the team the user currently in, it is better. and will be align with the usage/metrics API>

[Akash1134]

Important Announcement 🔈

We are pleased to inform you that the GitHub Copilot Metrics Power BI app is now live and available for use! This robust tool provides valuable insights into your GitHub Copilot usage, enabling you to monitor license assignments and track daily usage trends across your GitHub organizations. It supports various Copilot integrations, including code completion in the IDE, chat functionality within the IDE, interactions in the GitHub.com UI, and summarizing pull requests.

Where can I access the dashboard?

You can download the app from AppSource and install it in your preferred Power BI workspace. For configuration, please refer to the provided setup instructions.

Which GitHub Copilot versions are supported?

The app supports GitHub Copilot Business and GitHub Copilot Enterprise.

Thanks!

[dharinp]

We are trying to call both /usage (old API) and /metrics (new API) - but aren't getting data for the 25th Nov and 26th Nov, across multiple Orgs.

Is there an ongoing issue about this is being tracked internally?

[dharinp]

@bartvd2024 Have created a separate discussion to track this:
https://github.com/orgs/community/discussions/145716

Hoping for some responses there!

[LilianHoeltzener]

The same for me. To my knowledge, this is the only solution we have to obtain the analytics of GHC usage. I will soon have to make a presentation in my company on the use of copilot. We need this issue resolved quickly.

[Akash1134]

Hi there, We are investigating this issue and will update once we have more details to share.

[LilianHoeltzener]

Hello there. It seems to have been a fix. Since yesterday, I'm back to the values I expected up to the day before.

[DevOps-zhuang]

for my env, data of 11/26 is still missing.

Some metrics are invalid! nonContinuousDates: [ "2024-11-26" ] invalidCodeCompletions: [ { "date": "2024-11-18", "editor": "JetBrains", "language": "java" }, { "date": "2024-11-18", "editor": "JetBrains", "language": "yaml" }, { "date": "2024-11-18", "editor": "JetBrains", "language": "xml" }, { "date": "2024-11-18", "editor": "JetBrains", "language": "ini" }, { "date": "2024-11-18", "editor": "JetBrains", "language": "go" }, { "date": "2024-11-18", "editor": "JetBrains", "language": "unknown" }, { "date": "2024-11-19", "editor": "JetBrains", "language": "java" }, { "date": "2024-11-19", "editor": "JetBrains", "language": "yaml" }, { "date": "2024-11-19", "editor": "JetBrains", "language": "go" }, { "date": "2024-11-19", "editor": "JetBrains", "language": "ini" }, { "date": "2024-11-19", "editor": "JetBrains", "language": "unknown" } ]