Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

guides/features: add document for how to gracefully shutdown server #1388

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions content/en/docs/guides/server-graceful-stop.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
---
title: Gracefully stopping a server
description: >-
Explains how to gracefully stop/shutdown a gRPC server.
dfawley marked this conversation as resolved.
Show resolved Hide resolved
---

### Overview

gRPC servers often need to shut down gracefully, ensuring that in-flight RPCs
are completed within a reasonable timeframe and new RPCs are no longer
accepted. The "Graceful shutdown function" facilitates this process, allowing
the server to transition smoothly without abruptly terminating active
connections.

When the "Graceful shutdown function" is called, the server immediately stops
accepting new RPCs. In-flight RPCs are allowed to continue until they complete
dfawley marked this conversation as resolved.
Show resolved Hide resolved
or a specified deadline is reached. Once all active RPCs finish or the deadline
expires, the server shuts down completely.

### How to do Graceful Server Shutdown

The exact implementation of the "Graceful shutdown function" varies depending on
the programming language you are using. However, the general pattern
involves:

- Initiating the graceful shutdown process by calling "Graceful shutdown
dfawley marked this conversation as resolved.
Show resolved Hide resolved
Function" on your gRPC server object. This function blocks until all
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's be consistent with our capitalization please. Either "Function" everywhere when used in quotes like this, or "function" instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed everywhere

currently running RPCs complete. This ensures that in-flight requests are
allowed to finish processing.
- Specify a timeout period to limit the time allowed for in-progress RPCs to
finish. It's crucial to call the "Forceful shutdown function" on the server
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about "It's crucial in Go to call", since this is really go specific?

object with a timeout before calling the "Graceful shutdown function". This
acts as a safety net, ensuring that the server eventually shuts down even if
some in-flight RPCs don't complete within a reasonable timeframe. This
prevents indefinite blocking.

The following shows the sequence of events that occur, when a server's graceful
stop is invoked, in-flight RPCs continue to prorcess, but new RPCs are
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*process -- please run a spell check.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed all the spellings

rejected. If some in-flight RPCs are not finished in time, server is forcefully
shutdown.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verb form, so should be "shut down"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

```mermaid
sequenceDiagram
Client->>Server: New RPC Request 1
Client->>Server: New RPC Request 2
Server->>Client: Processing RPC 1
Server->>Client: Processing RPC 2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found these lines to be confusing when initially reading. I thought they indicated the RPCs had completed. Can the same meaning be conveyed if these lines are just deleted?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah i removed. The fact that New RPC Request 1 and New RPC Request 2 is Client->Server, it means request is received by server for processing.

Server-->>Server: Graceful Stop Invoked
Server->>Client: Continues Processing In-Flight RPCs
Client->>Server: New RPC Request 3 (Rejected)
alt RPCs complete within timeout
Server->>Client: Completes RPC 1
Server->>Client: Completes RPC 2
Server-->>Server: Shutdown Complete
Client->>Server: New RPC Request 4 (Rejected)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once a connection has been closed, RPCs aren't technically "rejected" anymore. They literally can't even be attempted.

Also should we discuss that the client will detect the server shutting down and go find other servers to send its traffic to, if at all possible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the rejected part and added the client will find another server if applicable

else Timeout reached
Server->>Client: RPC 2 and other pending RPCs forcefully terminated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should clearly show the forceful shutdown function being called by the application.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

Server-->>Server: Shutdown Complete
Client->>Server: New RPC Request 4 (Rejected)
end
```
The following is a state based view
```mermaid
stateDiagram-v2
[*] --> RUNNING : Server Started
RUNNING --> GRACEFUL_STOP_INITIATED : GracefulStop() Called (with Timeout)
GRACEFUL_STOP_INITIATED --> GRACEFUL_STOPPING : Reject New RPCs
dfawley marked this conversation as resolved.
Show resolved Hide resolved
GRACEFUL_STOPPING --> TERMINATED : Complete In-Flight RPCs (Before Timeout)
GRACEFUL_STOPPING --> FORCE_STOP : Timeout Reached
FORCE_STOP --> TERMINATED : Force Terminate Pending RPCs and Shutdown
dfawley marked this conversation as resolved.
Show resolved Hide resolved
```

### Alternatives

- Forceful Shutdown: Immediately terminates the server using "Forceful Shutdown
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And here "Shutdown" is capitalized.

Copy link
Contributor Author

@purnesh42H purnesh42H Dec 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Function" on server object, potentially interrupting active RPCs and leading
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/"Forceful Shutdown Function" on server/the "Forceful Shutdown Function" on the server/
s/potentially interrupting active RPCs and leading to/any active RPCs are interrupted, which will send/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done everywhere including graceful shutdown function

to errors on the client-side. Use only as a last resort.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This "alternative" section is awkward. And this "last resort" comment as well. Maybe the overview can just mention at the end that gracefully shutting down is not mandatory -- you can forcefully shut down without a graceful shutdown first, but that would not be recommended as it would result in waste. Or don't even mention it since it's not recommended?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed alternative section and added a sentence in Overview


### Language Support

| Language | Example |
|----------|-------------------|
| Java | |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

| Go | [Go example] |
| Python | |
dfawley marked this conversation as resolved.
Show resolved Hide resolved

[Go example]: https://github.com/grpc/grpc-go/tree/master/examples/features/gracefulstop