Running server-streaming calls on HealthCheck API delay server shutdown

[freibenjamin]

What version of gRPC and what language are you using?

Grpc 2.46.6
Grpc.AspNetCore.HealthChecks 2.66.0

What operating system (Linux, Windows,...) and version?

Windows 11

What runtime / compiler are you using (e.g. .NET Core SDK version dotnet --info)

.NET SDK 8.0.404

What did you do?

We integrated the Grpc.AspNetCore.HealthChecks NuGet package and tested its gRPC API with clients using the Watch server-streaming RPC. To monitor the server's status, we implemented a simple health check class that verifies the server's running state. However, during server shutdown, we noticed a delay if clients were connected to the Watch RPC at the time.

What did you expect to see?

A graceful shutdown that completes within a few seconds at most.

What did you see instead?

When the server stops, the shutdown is significantly delayed if clients are connected to the Watch RPC at the time of shutdown. The still opened calls cause the server to reach its shutdown timeout. We retained the host shutdown timeout at 30 seconds.
The screenshot below shows a client connected to the Watch RPC. The shutdown was initiated at 12:07:26 (via the terminal) and completed at 12:07:56.

Anything else we should know?

This issue is related to: #2357

The problem is that running server-streaming calls are not informed about the shutdown. You will find my proposed solution later in the linked PR.