Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

guides/features: add document for how to gracefully shutdown server #1388

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

guides/features: add document for how to gracefully shutdown server #1388

Changes from 2 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
d2aa9ab
guides/features: add server graceful shutdown document
purnesh42H Dec 10, 2024
4f1bf98
address language comments
purnesh42H Dec 10, 2024
37f05fc
punctuation and typos
purnesh42H Dec 10, 2024
2ed94db
generic forceful shutdown
purnesh42H Dec 11, 2024
07ec527
address doug's comments
purnesh42H Dec 12, 2024
53a97a5
Address more documentation comments from doug
purnesh42H Dec 20, 2024
a144c96
few more comments from dough
purnesh42H Dec 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions content/en/docs/guides/server-graceful-stop.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
title: Gracefully stopping a server
description: >-
Explains how to gracefully stop/shutdown a gRPC server.
dfawley marked this conversation as resolved.
Show resolved Hide resolved
---

### Overview

gRPC servers often need to shut down gracefully, ensuring that in-flight RPCs
are completed within a reasonable timeframe and new RPCs are no longer
accepted. "Graceful shutdown function" facilitates this process, allowing the server to
transition smoothly without abruptly terminating active connections.

When "Graceful shutdown function" is called, the server immediately stops accepting new
RPCs. In-flight RPCs are allowed to continue until they complete or a specified
deadline is reached. Once all active RPCs finish or the deadline expires, the
server shuts down completely.

### How to use Wait-for-Ready
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to replace "Wait-for-Ready"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


The exact implementation of "Graceful shutdown function" varies depending on the
programming language you are using. However, the general pattern
involves:

- Initiating the graceful shutdown process by calling "Graceful shutdown
dfawley marked this conversation as resolved.
Show resolved Hide resolved
Function" on your gRPC server object. This function blocks until all
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's be consistent with our capitalization please. Either "Function" everywhere when used in quotes like this, or "function" instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed everywhere

currently running RPCs completed. This ensures that in-flight requests are
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/completed/complete/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

allowed to finish processing.
- Specify a timeout period to limit the time allowed for in-progress RPCs to
finish. It's crucial to call "Forceful Shutdown Function" on server object
with a timeout before calling "Graceful shutdown function". This acts as a
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These need to be reversed, first graceful, then forceful.

Copy link
Contributor Author

@purnesh42H purnesh42H Dec 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh this is intentional. So, forceful shutdown function needs to be scheduled to execute in a separate thread after some time because graceful shutdown is a blocking call. In go we use a timeAfter function which executes the forceful shutdown in a new goroutine after the specified time passes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very golang specific. Other languages would initiate a graceful shutdown with a timeout and then if that indicated that it timed out would call a forceful shutdown. For example, in Java it looks like:

        server.shutdown();
        try {
          // Wait for RPCs to complete processing
          if (!server.awaitTermination(30, TimeUnit.SECONDS)) {
            // That was plenty of time. Let's cancel the remaining RPCs
            server.shutdownNow();
            // shutdownNow isn't instantaneous, so give a bit of time to clean resources up
            // gracefully. Normally this will be well under a second.
            server.awaitTermination(5, TimeUnit.SECONDS);
          }
        } catch (InterruptedException ex) {
          server.shutdownNow();
        }

I'm not sure of the best approach, because for Go it is very important to setup a timer to run the forceful stop on another thread, while for Java the stop and wait are separate commands so can be done in the same thread.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made it more generic. Let me know if this looks good to you.

safety net, ensuring that the server eventually shuts down even if some
in-flight RPCs don't complete within a reasonable timeframe. This prevents
indefinite blocking.

The following shows the sequence of events that occur, when a server graceful
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/server/server's/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

stop is invoked, in-flight RPCs continue to prorcess but new RPCs are rejected.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace comma with a period.
s/prorcess but/process, but/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

If all in-flight RPCs are not finished in time, server is forcefully shutdown.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/If all in-flight RPCs are not finished/If some in-flight RPCS do not finish/
s/, server/, the server/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

```mermaid
sequenceDiagram
Client->>Server: New RPC Request 1
Client->>Server: New RPC Request 2
Server->>Client: Processing RPC 1
Server->>Client: Processing RPC 2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found these lines to be confusing when initially reading. I thought they indicated the RPCs had completed. Can the same meaning be conveyed if these lines are just deleted?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah i removed. The fact that New RPC Request 1 and New RPC Request 2 is Client->Server, it means request is received by server for processing.

Server-->>Server: Graceful Stop Invoked
Server->>Client: Continues Processing In-Flight RPCs
Client->>Server: New RPC Request 3 (Rejected)
alt RPCs complete within timeout
Server->>Client: Completes RPC 1
Server->>Client: Completes RPC 2
Server-->>Server: Shutdown Complete
Client->>Server: New RPC Request 4 (Rejected)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once a connection has been closed, RPCs aren't technically "rejected" anymore. They literally can't even be attempted.

Also should we discuss that the client will detect the server shutting down and go find other servers to send its traffic to, if at all possible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the rejected part and added the client will find another server if applicable

else Timeout reached
Server->>Client: RPC 2 and other pending RPCs forcefully terminated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should clearly show the forceful shutdown function being called by the application.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

Server-->>Server: Shutdown Complete
Client->>Server: New RPC Request 4 (Rejected)
end
```
The following is a state based view
```mermaid
stateDiagram-v2
[*] --> RUNNING : Server Started
RUNNING --> GRACEFUL_STOP_INITIATED : GracefulStop() Called (with Timeout)
GRACEFUL_STOP_INITIATED --> GRACEFUL_STOPPING : Reject New RPCs
dfawley marked this conversation as resolved.
Show resolved Hide resolved
GRACEFUL_STOPPING --> TERMINATED : Complete In-Flight RPCs (Before Timeout)
GRACEFUL_STOPPING --> FORCE_STOP : Timeout Reached
FORCE_STOP --> TERMINATED : Force Terminate Pending RPCs and Shutdown
dfawley marked this conversation as resolved.
Show resolved Hide resolved
```

### Alternatives

- Forceful Shutdown: Immediately terminates the server using "Forceful Shutdown
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And here "Shutdown" is capitalized.

Copy link
Contributor Author

@purnesh42H purnesh42H Dec 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Function" on server object, potentially interrupting active RPCs and leading
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/"Forceful Shutdown Function" on server/the "Forceful Shutdown Function" on the server/
s/potentially interrupting active RPCs and leading to/any active RPCs are interrupted, which will send/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done everywhere including graceful shutdown function

to errors on the client-side. Use only as a last resort.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This "alternative" section is awkward. And this "last resort" comment as well. Maybe the overview can just mention at the end that gracefully shutting down is not mandatory -- you can forcefully shut down without a graceful shutdown first, but that would not be recommended as it would result in waste. Or don't even mention it since it's not recommended?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed alternative section and added a sentence in Overview


### Language Support

| Language | Example |
|----------|-------------------|
| Java | |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can point to an existing example for Java: https://github.com/grpc/grpc-java/tree/master/examples/example-hostname/src/main/java/io/grpc/examples/hostname

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

| Go | [Go example] |
| Python | |
dfawley marked this conversation as resolved.
Show resolved Hide resolved

[Go example]: https://github.com/grpc/grpc-go/tree/master/examples/features/gracefulstop