From 164e18a45e66792221b05c4dd14bbd7ba879c2f4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ra=C3=BAl=20Cumplido?= Date: Fri, 20 Dec 2024 23:48:14 +0100 Subject: [PATCH] GH-45079: [FlightRPC][C++] Deprecate InitializeFlightUcx before removing UCX (#45080) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ### Rationale for this change We want to remove the UCX experiment as discussed on the issue: - https://github.com/apache/arrow/issues/43296 Based on the discussions of the PR removal we have decided to deprecate it before. ### What changes are included in this PR? Deprecation of InitializeFlightUcx ### Are these changes tested? No, I don't think we test `ARROW_WITH_UCX` on CI. ### Are there any user-facing changes? Yes, the PR deprecated Flight UCX * GitHub Issue: #45079 Authored-by: Raúl Cumplido Signed-off-by: Sutou Kouhei --- cpp/src/arrow/flight/transport/ucx/ucx.h | 2 ++ docs/source/cpp/flight.rst | 34 ------------------------ docs/source/format/Flight.rst | 7 ++++- docs/source/status.rst | 19 ++++++------- 4 files changed, 18 insertions(+), 44 deletions(-) diff --git a/cpp/src/arrow/flight/transport/ucx/ucx.h b/cpp/src/arrow/flight/transport/ucx/ucx.h index dda2c83035c6d..7a12987c31e00 100644 --- a/cpp/src/arrow/flight/transport/ucx/ucx.h +++ b/cpp/src/arrow/flight/transport/ucx/ucx.h @@ -26,6 +26,8 @@ namespace flight { namespace transport { namespace ucx { +/// \deprecated Deprecated in 19.0.0. Flight UCX is deprecated. +ARROW_DEPRECATED(" Deprecated in 19.0.0. Flight UCX is deprecated.") ARROW_FLIGHT_EXPORT void InitializeFlightUcx(); diff --git a/docs/source/cpp/flight.rst b/docs/source/cpp/flight.rst index a1e9420bfd34e..c076e0b8c1a67 100644 --- a/docs/source/cpp/flight.rst +++ b/docs/source/cpp/flight.rst @@ -362,38 +362,4 @@ Closing unresponsive connections .. _ARROW-16697: https://issues.apache.org/jira/browse/ARROW-16697 .. _ARROW-6062: https://issues.apache.org/jira/browse/ARROW-6062 - -Alternative Transports -====================== - -The standard transport for Arrow Flight is gRPC_. The C++ -implementation also experimentally supports a transport based on -UCX_. To use it, use the protocol scheme ``ucx:`` when starting a -server or creating a client. - -UCX Transport -------------- - -Not all features of the gRPC transport are supported. See -:ref:`status-flight-rpc` for details. Also note these specific -caveats: - -- The server creates an independent UCP worker for each client. This - consumes more resources but provides better throughput. -- The client creates an independent UCP worker for each RPC - call. Again, this trades off resource consumption for - performance. This also means that unlike with gRPC, it is - essentially equivalent to make all calls with a single client or - with multiple clients. -- The UCX transport attempts to avoid copies where possible. In some - cases, it can directly reuse UCX-allocated buffers to back - :class:`arrow::Buffer` objects, however, this will also extend the - lifetime of associated UCX resources beyond the lifetime of the - Flight client or server object. -- Depending on the transport that UCX itself selects, you may find - that increasing ``UCX_MM_SEG_SIZE`` from the default (around 8KB) to - around 60KB improves performance (UCX will copy more data in a - single call). - .. _gRPC: https://grpc.io/ -.. _UCX: https://openucx.org/ diff --git a/docs/source/format/Flight.rst b/docs/source/format/Flight.rst index 2c5487d857ea4..2a34db0f1ba50 100644 --- a/docs/source/format/Flight.rst +++ b/docs/source/format/Flight.rst @@ -333,9 +333,14 @@ schemes for the given transports: +----------------------------+--------------------------------+ | (reuse connection) | arrow-flight-reuse-connection: | +----------------------------+--------------------------------+ -| UCX_ (plaintext) | ucx: | +| UCX_ (plaintext) (1) | ucx: | +----------------------------+--------------------------------+ +Notes: + +* \(1) Flight UCX transport has been deprecated on the 19.0.0 release. + The :ref:`dissociated-ipc` section proposes an alternative solution. + .. _UCX: https://openucx.org/ Connection Reuse diff --git a/docs/source/status.rst b/docs/source/status.rst index 83bee8975bbf7..c5883afa8f345 100644 --- a/docs/source/status.rst +++ b/docs/source/status.rst @@ -202,7 +202,7 @@ Flight RPC +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ | gRPC + TLS transport (grpc+tls:) | ✓ | ✓ | ✓ | | ✓ | ✓ | | | +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ -| UCX_ transport (ucx:) | ✓ | | | | | | | | +| UCX_ transport (ucx:) (1) | ✓ | | | | | | | | +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ Supported features in the gRPC transport: @@ -212,13 +212,13 @@ Supported features in the gRPC transport: +============================================+=======+=======+=======+====+=======+=======+=======+=======+ | All RPC methods | ✓ | ✓ | ✓ | | ✓ | ✓ | | | +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ -| Authentication handlers | ✓ | ✓ | ✓ | | ✓ (1) | ✓ | | | +| Authentication handlers | ✓ | ✓ | ✓ | | ✓ (2) | ✓ | | | +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ | Call timeouts | ✓ | ✓ | ✓ | | | ✓ | | | +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ | Call cancellation | ✓ | ✓ | ✓ | | | ✓ | | | +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ -| Concurrent client calls (2) | ✓ | ✓ | ✓ | | ✓ | ✓ | | | +| Concurrent client calls (3) | ✓ | ✓ | ✓ | | ✓ | ✓ | | | +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ | Custom middleware | ✓ | ✓ | ✓ | | | ✓ | | | +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ @@ -230,7 +230,7 @@ Supported features in the UCX transport: +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ | Flight RPC Feature | C++ | Java | Go | JS | C# | Rust | Julia | Swift | +============================================+=======+=======+=======+====+=======+=======+=======+=======+ -| All RPC methods | ✓ (3) | | | | | | | | +| All RPC methods | ✓ (4) | | | | | | | | +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ | Authentication handlers | | | | | | | | | +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ @@ -238,7 +238,7 @@ Supported features in the UCX transport: +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ | Call cancellation | | | | | | | | | +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ -| Concurrent client calls | ✓ (4) | | | | | | | | +| Concurrent client calls | ✓ (5) | | | | | | | | +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ | Custom middleware | | | | | | | | | +--------------------------------------------+-------+-------+-------+----+-------+-------+-------+-------+ @@ -247,10 +247,11 @@ Supported features in the UCX transport: Notes: -* \(1) Support using AspNetCore authentication handlers. -* \(2) Whether a single client can support multiple concurrent calls. -* \(3) Only support for DoExchange, DoGet, DoPut, and GetFlightInfo. -* \(4) Each concurrent call is a separate connection to the server +* \(1) Flight UCX transport has been deprecated on the 19.0.0 release. +* \(2) Support using AspNetCore authentication handlers. +* \(3) Whether a single client can support multiple concurrent calls. +* \(4) Only support for DoExchange, DoGet, DoPut, and GetFlightInfo. +* \(5) Each concurrent call is a separate connection to the server (unlike gRPC where concurrent calls are multiplexed over a single connection). This will generally provide better throughput but consumes more resources both on the server and the client.