Add metric to track how many times is each key requested

[lbaquerofierro]

No description provided.

[fisherdarling]

I have a concern with cardinality here. We usually try not to add prometheus labels with unbounded cardinality, due to each unique label creating a unique prometheus timeseries. I guess the kids will be short-lived ~1 week, so this might be fine.

[thibmeu]

@fisherdarling cardinality of key_id is 256 (from 0 to 255). I agree this is large.
The number of values then depends on the prometheus server retention time, speed of the rotation, and numbers of keys to retain in parallel.
If rotation is once a day and retention of two weeks, the upperbound is 14 (for 14 days).

[thibmeu]

@lbaquerofierro I've posted a change proposal to your repo lbaquerofierro#1

It does not create an additional metric, and allows to set default labels.

[thibmeu]

Suggested change

	ctx.metrics.erroredRequestsTotal.inc({ env: ctx.env.ENVIRONMENT, servcice: ctx.env.SERVICE });
	ctx.metrics.erroredRequestsTotal.inc({ env: ctx.env.ENVIRONMENT, service: ctx.env.SERVICE });

[thibmeu]

only use this metric

Suggested change

	ctx.metrics.signedTokenTotal.inc({ env: ctx.env.ENVIRONMENT, service: ctx.env.SERVICE });
	const keyId = tokenRequest.truncatedTokenKeyId.toString();
	ctx.metrics.keyRequestTotal.inc({
	env: ctx.env.ENVIRONMENT,
	service: ctx.env.SERVICE,
	key_id: keyId,
	});
	const keyId = keyId = tokenRequest.truncatedTokenKeyId.toString();
	ctx.metrics.signedTokenTotal.inc({ env: ctx.env.ENVIRONMENT, service: ctx.env.SERVICE, key_id: keyId });

[fisherdarling]

@fisherdarling cardinality of key_id is 256 (from 0 to 255). I agree this is large. The number of values then depends on the prometheus server retention time, speed of the rotation, and numbers of keys to retain in parallel. If rotation is once a day and retention of two weeks, the upperbound is 14 (for 14 days).

Great! Thanks for walking through the true cardinality. 14 is perfectly fine.

[thibmeu]

change looks ok.
once you've addressed the nit, can you squash your changes, and rebase your branch on origin/main

[thibmeu]

nit: fix the version

[thibmeu]

I proposed one additional commit lbaquerofierro#2 that you should be able to merge.
Once done, we can rebase and merge this branch