feat(mgmt): update metrics endpoints to serve new dashboard design

[joremysh]

Because

• Pipeline and credit endpoints have different params and schemas.
• Some of the information served by the existing endpoints will be handled by the pipeline run feature.

This commit

• Removes unused endpoints and updates metrics contracts to serve the new dashboard.

[linear]

INS-5379 Credit dashboard endpoints

[github-actions]

The latest Buf updates on your PR. Results from workflow Buf CI / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
✅ passed	✅ passed	✅ passed	✅ passed	Nov 6, 2024, 10:25 AM

[jvallesm]

I added some suggestions to improve the comments. Thanks for taking into account the notes on #435. You could have pushed your changes to that branch to avoid duplication (I don't mind handing over a PR or having changes pushed on top of mine 😄). If you could please close that PR once you merge this one, it'll remove some noise from the repo 🧹

[joremysh]

We are planning to replace ListPipelineTriggerTableRecords with GetPipelineTriggerCount and modify response of ListPipelineTriggerChartRecords as frontend does not need to display trigger count for each pipeline now. After the change, frontend does not need to sum the numbers of pipeline triggers anymore. It will also help simplify the influxDB query in backend.
@EiffelFly Could you help update the dashboard? In which sprint could we merge the backend?

ListPipelineTriggerTableRecords URL: /v1beta/metrics/vdp/pipeline/tables

GetPipelineTriggerCount response will be like:

{
    "pipelineTriggerCounts":
    [
        {
            "triggerCount": 3,
            "status": "STATUS_COMPLETED"
        }
    ]
}

ListPipelineTriggerChartRecords response will be like:

{
    "pipelineTriggerChartRecords":
    [
        {
            "timeBuckets":
            [
                "2024-11-07T01:00:00Z",
                "2024-11-07T02:00:00Z",
                "2024-11-07T03:00:00Z",
                "2024-11-07T04:00:00Z",
                "2024-11-07T05:00:00Z",
                "2024-11-07T06:00:00Z",
                "2024-11-07T06:41:33.706786732Z"
            ],
            "triggerCounts":
            [
                0,
                0,
                0,
                0,
                0,
                0,
                3
            ],
            "requesterId": "requesterId-A"
        }
    ]
}

URL of ListPipelineTriggerChartRecords will be changed from /v1beta/metrics/vdp/pipeline/charts to /v1beta/pipeline-runs/query-charts following discussions in previous API design meeting.

[jvallesm]

These changes will be tackled in #435, where the changes will be made in a backwards-compatible way so the frontend can implement the changes according to the devs' availability. Another PR will clean up the deprecated endpoints.