Skip to content

Commit

Permalink
apacheGH-45079: [FlightRPC][C++] Deprecate InitializeFlightUcx before…
Browse files Browse the repository at this point in the history 
… removing UCX (apache#45080)

### Rationale for this change

We want to remove the UCX experiment as discussed on the issue:
- apache#43296

Based on the discussions of the PR removal we have decided to deprecate it before.

### What changes are included in this PR?

Deprecation of InitializeFlightUcx

### Are these changes tested?

No, I don't think we test `ARROW_WITH_UCX` on CI.

### Are there any user-facing changes?

Yes, the PR deprecated Flight UCX
* GitHub Issue: apache#45079

Authored-by: Raúl Cumplido <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
  • Loading branch information
@raulcd
raulcd authored Dec 20, 2024
1 parent 5293379 commit 164e18a
Show file tree
Hide file tree
Showing 4 changed files with 18 additions and 44 deletions.
2 changes: 2 additions & 0 deletions cpp/src/arrow/flight/transport/ucx/ucx.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ namespace flight {
namespace transport {
namespace ucx {

/// \deprecated Deprecated in 19.0.0. Flight UCX is deprecated.
ARROW_DEPRECATED(" Deprecated in 19.0.0. Flight UCX is deprecated.")
ARROW_FLIGHT_EXPORT
void InitializeFlightUcx();

Expand Down
34 changes: 0 additions & 34 deletions docs/source/cpp/flight.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -362,38 +362,4 @@ Closing unresponsive connections
.. _ARROW-16697: https://issues.apache.org/jira/browse/ARROW-16697
.. _ARROW-6062: https://issues.apache.org/jira/browse/ARROW-6062


Alternative Transports
======================

The standard transport for Arrow Flight is gRPC_. The C++
implementation also experimentally supports a transport based on
UCX_. To use it, use the protocol scheme ``ucx:`` when starting a
server or creating a client.

UCX Transport
-------------

Not all features of the gRPC transport are supported. See
:ref:`status-flight-rpc` for details. Also note these specific
caveats:

- The server creates an independent UCP worker for each client. This
consumes more resources but provides better throughput.
- The client creates an independent UCP worker for each RPC
call. Again, this trades off resource consumption for
performance. This also means that unlike with gRPC, it is
essentially equivalent to make all calls with a single client or
with multiple clients.
- The UCX transport attempts to avoid copies where possible. In some
cases, it can directly reuse UCX-allocated buffers to back
:class:`arrow::Buffer` objects, however, this will also extend the
lifetime of associated UCX resources beyond the lifetime of the
Flight client or server object.
- Depending on the transport that UCX itself selects, you may find
that increasing ``UCX_MM_SEG_SIZE`` from the default (around 8KB) to
around 60KB improves performance (UCX will copy more data in a
single call).

.. _gRPC: https://grpc.io/
.. _UCX: https://openucx.org/
7 changes: 6 additions & 1 deletion docs/source/format/Flight.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -333,9 +333,14 @@ schemes for the given transports:
+----------------------------+--------------------------------+
| (reuse connection) | arrow-flight-reuse-connection: |
+----------------------------+--------------------------------+
| UCX_ (plaintext) | ucx: |
| UCX_ (plaintext) (1) | ucx: |
+----------------------------+--------------------------------+

Notes:

* \(1) Flight UCX transport has been deprecated on the 19.0.0 release.
The :ref:`dissociated-ipc` section proposes an alternative solution.

.. _UCX: https://openucx.org/

Connection Reuse
Expand Down
19 changes: 10 additions & 9 deletions docs/source/status.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ Flight RPC
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+
| gRPC + TLS transport (grpc+tls:) | ✓ | ✓ | ✓ | | ✓ | ✓ | | |
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+
| UCX_ transport (ucx:) | ✓ | | | | | | | |
| UCX_ transport (ucx:) (1) | ✓ | | | | | | | |
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+

Supported features in the gRPC transport:
Expand All @@ -212,13 +212,13 @@ Supported features in the gRPC transport:
+============================================+=======+=======+=======+====+=======+=======+=======+=======+
| All RPC methods | ✓ | ✓ | ✓ | | ✓ | ✓ | | |
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+
| Authentication handlers | ✓ | ✓ | ✓ | | ✓ (1) | ✓ | | |
| Authentication handlers | ✓ | ✓ | ✓ | | ✓ (2) | ✓ | | |
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+
| Call timeouts | ✓ | ✓ | ✓ | | | ✓ | | |
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+
| Call cancellation | ✓ | ✓ | ✓ | | | ✓ | | |
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+
| Concurrent client calls (2) | ✓ | ✓ | ✓ | | ✓ | ✓ | | |
| Concurrent client calls (3) | ✓ | ✓ | ✓ | | ✓ | ✓ | | |
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+
| Custom middleware | ✓ | ✓ | ✓ | | | ✓ | | |
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+
Expand All @@ -230,15 +230,15 @@ Supported features in the UCX transport:
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+
| Flight RPC Feature | C++ | Java | Go | JS | C# | Rust | Julia | Swift |
+============================================+=======+=======+=======+====+=======+=======+=======+=======+
| All RPC methods | ✓ (3) | | | | | | | |
| All RPC methods | ✓ (4) | | | | | | | |
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+
| Authentication handlers | | | | | | | | |
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+
| Call timeouts | | | | | | | | |
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+
| Call cancellation | | | | | | | | |
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+
| Concurrent client calls | ✓ (4) | | | | | | | |
| Concurrent client calls | ✓ (5) | | | | | | | |
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+
| Custom middleware | | | | | | | | |
+--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+
Expand All @@ -247,10 +247,11 @@ Supported features in the UCX transport:

Notes:

* \(1) Support using AspNetCore authentication handlers.
* \(2) Whether a single client can support multiple concurrent calls.
* \(3) Only support for DoExchange, DoGet, DoPut, and GetFlightInfo.
* \(4) Each concurrent call is a separate connection to the server
* \(1) Flight UCX transport has been deprecated on the 19.0.0 release.
* \(2) Support using AspNetCore authentication handlers.
* \(3) Whether a single client can support multiple concurrent calls.
* \(4) Only support for DoExchange, DoGet, DoPut, and GetFlightInfo.
* \(5) Each concurrent call is a separate connection to the server
(unlike gRPC where concurrent calls are multiplexed over a single
connection). This will generally provide better throughput but
consumes more resources both on the server and the client.
Expand Down

0 comments on commit 164e18a

Please sign in to comment.