diff --git a/modules/concept-docs/pages/compression.adoc b/modules/concept-docs/pages/compression.adoc index 54989514..a4e161c2 100644 --- a/modules/concept-docs/pages/compression.adoc +++ b/modules/concept-docs/pages/compression.adoc @@ -1,5 +1,5 @@ = Compression -:description: Data compression to reduce traffic costs from app to Server. +:description: In response to increasing volumes of data being sent over the wire, Couchbase Data Platform provides data compression between the SDK and Couchbase Server. :page-topic-type: concept :page-edition: Enterprise Edition :page-aliases:ROOT:compression-intro,compression-intro @@ -7,21 +7,120 @@ include::project-docs:partial$attributes.adoc[] [abstract] -In response to increasing volumes of data being sent over the wire, Couchbase Data Platform now provides data compression between the SDK and Couchbase Server. -However, stable Snappy support is not yet available in Microsoft's https://dotnet.microsoft.com/en-us/[.NET platform]. +{description} +[Note] +==== +From version 3.5.2, the .NET SDK enables compression by default using https://www.nuget.org/packages/Snappier[Snappier], a performance-focused C# port of https://github.com/google/snappy[Snappy] provided by https://btburnett.com/[Brant Burnett]. This was previously opt-in only via installing the corresponding https://www.nuget.org/packages/Couchbase.Extensions.Compression.Snappier/1.0.0-beta.1[Couchbase.Extensions] package. +==== +== Overview -Couchbase Server (in the Enterprise Edition) stores documents in compressed form, xref:7.1@server:learn:buckets-memory-and-storage/compression.adoc#compression-modes[when it is enabled in the server for a particular bucket], using Snappy Compression. -As the Snappy compression library is not available for .NET, the server will automatically uncompress any compressed documents before sending them to the .NET client. +Documents may already be stored compressed with Snappy on the server. +New documents may be passed from client applications to the server in compressed form, saving around 40% in bandwidth (depending on the document content and size), and also transmission time. +These operations take place automatically, after the SDK negotiates the capability with the server, and do not require any changes on the client side. -If compression is set to _active_ on the server, documents will be stored there in compressed form, even though the .NET client has sent them uncompressed, thus saving storage space (but not network bandwidth). +For SDKs with Snappy compression enabled, documents will be automatically compressed if compression is not turned `Off` in xref:7.1@server:learn:buckets-memory-and-storage/compression.adoc#compression-modes[Couchbase Server] (see xref:#minimum-size[below]). +Instructions to disable compression can be found at xref:#threshold[the bottom of the page]. +== Usage and customization +Starting with .NET SDK v3.5.2, Snappier compression is enabled by default in the ClusterOptions. +You can disable compression or customize certain settings, both via ClusterOptions or Connection String parameters: -== Community-Supported Snappy Library +[source,dotnet] +---- +// ConnectionString parameters +var connectionString = "couchbase://localhost:8091?compression=true&compression_min_size=512&compression_min_ratio=0.82"; -Whilst there is no official Microsoft-supported Snappy compression in the .NET platform, the community has produced a promising option in https://github.com/brantburnett/Snappier[Snappier] -- -integrated with the the https://github.com/couchbaselabs/Couchbase.Extensions[.NET Couchbase.Extensions Library], +// ClusterOptions parameters +var clusterOptions = new ClusterOptions +{ + Compression = true, + CompressionMinRatio = 0.82f, // Float value between 1 and 0. + CompressionMinSize = 512 // Integer value represented in bytes. +}; + +// Connect +clusterOptions.WithConnectionString(connectionString); +clusterOptions.WithCredentials("Administrator", "password"); + +var cluster = await Cluster.ConnectAsync(clusterOptions).ConfigureAwait(false); +---- + +Please refer to the https://docs.couchbase.com/sdk-api/couchbase-net-client/api/Couchbase.ClusterOptions.html#Couchbase_ClusterOptions_Compression[API Documentation] for details on `CompressionMinRatio` and `CompressionMinSize`. + +== **Advanced**: Bring Your Own Compression +You can provide the SDK with your own implementation of Snappy compression/decompression by implementing the `ICompressionAlgorithm` interface, and passing it through your ClusterOptions with: +[source,dotnet] +---- +clusterOptions.WithCompressionAlgorithm(myCompressionImplementation); + +// See Couchbase.Compression.Snappier.Internal.SnappierCompression for details on how Snappier implements the interface. +---- + +[Note] +==== +Even when compression is turned `Off` server-side, Couchbase Server will decompress documents in memory, and store them recompressed on disk (using Snappy) (xref:7.1@server:learn:buckets-memory-and-storage/compression.adoc#compression-modes[see server compression docs]). + +It is therefore not recommended to implement a different compression algorithm than Snappy as this may very well cause data corruption. +==== + +== Limits + +The document must be below 20MiB in size in both compressed and uncompressed form. +Compression is only available in the Enterprise Edition of Couchbase Data Platform. + +[TIP] +==== +This size limit is enforced by Couchbase Server; in practice it will affect very few users, as most JSON documents are considerably smaller. +A compressed doument of just under 20MB, which is greater than 20,971,520 bytes (20 MiB) when uncompressed, will be rejected by the server as follows: + +* Couchbase Server decompresses the document to check that it is valid JSON, and is correctly compressed with _Snappy_, and at this point measures it against `max data size` (20 MiB). +* If the decompressed value's size exceeds this limit, the mutation is failed with a "too big" error code (E2BIG code 3). + +Therefore, where necessary, enforce document size limits in your application on _uncompressed_ documents. +==== + +== Operating Modes (Server-side) + +The three modes in which compression can be used, "off", "passive" and "active", are configured per bucket in the configuration settings on the cluster. + +.Compression Operating Modes +[#compression-operating-modes] +|=== +| | *OFF* | *PASSIVE* | *ACTIVE* + +| Negotiating SNAPPY +| ignore +| acknowledge +| acknowledge + +| Sending compressed data when SNAPPY feature *enabled* +| - +| accept +| accept + +| Sending compressed data when SNAPPY feature *disabled* +| reject as invalid +| reject as invalid +| reject as invalid + +| Receiving data which were previously compressed on the server +| server inflates and sends plain data +| server sends compressed data +| server sends compressed data + +| Receiving data which were *not* previously compressed on the server +| server sends plain data +| server sends plain data +| server might send compressed data (the compression is done in the background on the server) +|=== + + +== Compression for .NET SDK 3.5.1 and below + +https://github.com/brantburnett/Snappier[Snappier] is +integrated with the https://github.com/couchbaselabs/Couchbase.Extensions[.NET Couchbase.Extensions Library]. Since xref:project-docs:sdk-release-notes.adoc#version-3-4-10[.NET SDK 3.4.10], the .NET SDK has been able to work with external Snappy libraries as the setting for the Server flag `SnappyEverywhere` is now supported. Install Snappier with the https://github.com/couchbaselabs/Couchbase.Extensions[.NET Couchbase.Extensions Library],