[improve][fn] Add API endpoint for function-worker for liveness check with configurable flag #358
Conversation
mukesh-ctds
changed the title
mukesh-ctds
changed the title
| @@ -429,4 +430,18 @@ public void updateFunctionOnWorkerLeader(final @PathParam("tenant") String tenan | |||
| functions().updateFunctionOnWorkerLeader(tenant, namespace, functionName, uploadedInputStream, | |||
| delete, uri.getRequestUri(), authParams()); | |||
| } | |||
|
|
|||
| @GET | |||
| @Path("/live") | |||
There was a problem hiding this comment.
I think /healthz is the common convention for this endpoint name.
Also, I think it would make more sense to put it directly under the root path instead of under /admin/v3/functions.
There was a problem hiding this comment.
The API has been updated to include a /healthz endpoint.
We cannot place this directly under the root path, as the root path does not have access to the necessary flag. The flag is available at the function level instead. Additionally, v2 does not contain any logic that could trigger a ProducerFencedException; this issue is specific to v3.
| @@ -693,6 +695,8 @@ public void updateFunctionOnWorkerLeader(final String tenant, | |||
| try { | |||
| functionMetaDataManager.updateFunctionOnLeader(functionMetaData, delete); | |||
| } catch (IllegalStateException e) { | |||
| log.error("Function worker status has been set to false due to ProducerFencedException."); | |||
There was a problem hiding this comment.
What happens here if we get an IllegalStateException which is not a ProducerFencedException? Maybe it would be better to catch the specific exception type.
There was a problem hiding this comment.
The block of code has been moved to a location where we can check if the cause is a ProducerFencedException.
Motivation
This pull request implements new health check functionality for Kubernetes deployment, focusing on the liveness probe for function worker. This liveliness API is required for Kubernetes-based applications, enabling automated pod restarts in case of failure (via liveness checks) whenever ProducerFencedException occurs.
Modifications
This update introduces an API endpoint to perform a liveliness check on the function worker pod. The API returns an HTTP status of
200 (OK)when theisLiveflag withinFunctionImplis true. If the flag is false, typically after aProducerFencedExceptionoccurs, the API will return a status of503 (Service Unavailable).The Kubernetes deployment configuration has been updated to use this new API endpoint in the liveness probe along with existing
metricsendpoint, allowing the system to monitor the health and availability of the function worker.Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:
If
yeswas chosen, please highlight the changesDocumentation
Check the box below or label this PR directly (if you have committer privilege).
Need to update docs?
doc-requiredno-need-docdoc