diff --git a/docs/modules/cluster-performance/pages/best-practices.adoc b/docs/modules/cluster-performance/pages/best-practices.adoc index 2e0fb2da0..8cf448991 100644 --- a/docs/modules/cluster-performance/pages/best-practices.adoc +++ b/docs/modules/cluster-performance/pages/best-practices.adoc @@ -867,299 +867,6 @@ you have to configure a unique storage filename for each client or run them from If two clients would write into the same file, only the first client succeeds. The following clients throw an exception as soon as the Near Cache preloader is triggered. -== Data Affinity - -Data affinity ensures that related entries exist on the same member. If related data is on the same member, operations can -be executed without the cost of extra network calls and extra wire data. This feature is provided by using the same partition keys for related data. - -=== PartitionAware - -**Co-location of related data and computation** - -Hazelcast has a standard way of finding out which member owns/manages each key object. -The following operations are routed to the same member, since all them are operating based on the same key `"key1"`. - -[source,java] ----- -HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(); -Map mapA = hazelcastInstance.getMap( "mapA" ); -Map mapB = hazelcastInstance.getMap( "mapB" ); -Map mapC = hazelcastInstance.getMap( "mapC" ); - -// since map names are different, operation will be manipulating -// different entries, but the operation will take place on the -// same member since the keys ("key1") are the same -mapA.put( "key1", value ); -mapB.get( "key1" ); -mapC.remove( "key1" ); - -// lock operation will still execute on the same member -// of the cluster since the key ("key1") is same -hazelcastInstance.getLock( "key1" ).lock(); - -// distributed execution will execute the 'runnable' on the -// same member since "key1" is passed as the key. -hazelcastInstance.getExecutorService().executeOnKeyOwner( runnable, "key1" ); ----- - -When the keys are the same, entries are stored on the same member. -But we sometimes want to have related entries stored on the same member, such as a customer and his/her order entries. -We would have a customers map with customerId as the key and an orders map with orderId as the key. -Since customerId and orderId are different keys, a customer and -his/her orders may fall into different members in your cluster. So how can we have them stored on the same member? -We create an affinity between customer and orders. If we make them part of the same partition then -these entries will be co-located. We achieve this by making `orderKey` s `PartitionAware`. - -[source,java] ----- -include::ROOT:example$/performance/OrderKey.java[tag=orderkey] ----- - -Notice that OrderKey implements `PartitionAware` and that `getPartitionKey()` returns the `customerId`. -These make sure that the `Customer` entry and its ``Order``s are stored on the same member. - -[source,java] ----- -HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(); -Map mapCustomers = hazelcastInstance.getMap( "customers" ); -Map mapOrders = hazelcastInstance.getMap( "orders" ); - -// create the customer entry with customer id = 1 -mapCustomers.put( 1, customer ); - -// now create the orders for this customer -mapOrders.put( new OrderKey( 21, 1 ), order ); -mapOrders.put( new OrderKey( 22, 1 ), order ); -mapOrders.put( new OrderKey( 23, 1 ), order ); ----- - -Assume that you have a customers map where `customerId` is the key and the customer object is the value. -You want to remove one of the customer orders and return the number of remaining orders. -Here is how you would normally do it. - -[source,java] ----- -public static int removeOrder( long customerId, long orderId ) throws Exception { - IMap mapCustomers = instance.getMap( "customers" ); - IMap mapOrders = hazelcastInstance.getMap( "orders" ); - - mapCustomers.lock( customerId ); - mapOrders.remove( new OrderKey(orderId, customerId) ); - Set orders = orderMap.keySet(Predicates.equal( "customerId", customerId )); - mapCustomers.unlock( customerId ); - - return orders.size(); -} ----- - -There are a couple of things you should consider. - -* There are four distributed operations there: lock, remove, keySet, unlock. Can you reduce -the number of distributed operations? -* The customer object may not be that big, but can you not have to pass that object through the -wire? Think about a scenario where you set order count to the customer object for fast access, so you -should do a get and a put, and as a result, the customer object is passed through the wire twice. - -Instead, why not move the computation over to the member (JVM) where your customer data resides. -Here is how you can do this with distributed executor service. - -. Send a `PartitionAware` `Callable` task. -. `Callable` does the deletion of the order right there and returns with the remaining -order count. -. Upon completion of the `Callable` task, return the result (remaining order count). You -do not have to wait until the task is completed; since distributed executions are asynchronous, -you can do other things in the meantime. - -Here is an example code. - -[source,java] ----- -HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(); - - public int removeOrder(long customerId, long orderId) throws Exception { - IExecutorService executorService = hazelcastInstance.getExecutorService("ExecutorService"); - - OrderDeletionTask task = new OrderDeletionTask(customerId, orderId); - Future future = executorService.submit(task); - int remainingOrders = future.get(); - - return remainingOrders; - } - - public static class OrderDeletionTask - implements Callable, PartitionAware, Serializable, HazelcastInstanceAware { - - private long orderId; - private long customerId; - private HazelcastInstance hazelcastInstance; - - public OrderDeletionTask() { - } - - public OrderDeletionTask(long customerId, long orderId) { - this.customerId = customerId; - this.orderId = orderId; - } - - @Override - public Integer call() { - IMap customerMap = hazelcastInstance.getMap("customers"); - IMap orderMap = hazelcastInstance.getMap("orders"); - - customerMap.lock(customerId); - - Predicate predicate = Predicates.equal("customerId", customerId); - Set orderKeys = orderMap.localKeySet(predicate); - int orderCount = orderKeys.size(); - for (OrderKey key : orderKeys) { - if (key.orderId == orderId) { - orderCount--; - orderMap.delete(key); - } - } - - customerMap.unlock(customerId); - - return orderCount; - } - - @Override - public Object getPartitionKey() { - return customerId; - } - - @Override - public void setHazelcastInstance(HazelcastInstance hazelcastInstance) { - this.hazelcastInstance = hazelcastInstance; - } - } ----- - -The following are the benefits of doing the same operation with distributed `ExecutorService` based on the key: - -* only one distributed execution (`executorService.submit(task)`), instead of four -* less data is sent over the wire -* less lock duration, i.e., higher concurrency, for the `Customer` entry since -lock/update/unlock cycle is done locally (local to the customer data) - -=== PartitioningStrategy - -Another way of storing the related data on the same location is using/implementing -the class `PartitioningStrategy`. Normally (if no partitioning strategy is defined), -Hazelcast finds the partition of a key first by converting the object to binary and then by hashing this binary. -If a partitioning strategy is defined, Hazelcast injects the key to the strategy and -the strategy returns an object out of which the partition is calculated by hashing it. - -Hazelcast offers the following out-of-the-box partitioning strategies: - -* `DefaultPartitioningStrategy`: Default strategy. It checks whether the key implements `PartitionAware`. -If it implements, the object is converted to binary and then hashed, to find the partition of the key. -* `StringPartitioningStrategy`: Works only for string keys. It uses the string after `@` character as the partition ID. -For example, if you have two keys `ordergroup1@region1` and `customergroup1@region1`, -both `ordergroup1` and `customergroup1` fall into the partition where `region1` is located. -* `StringAndPartitionAwarePartitioningStrategy`: Works as the combination of the above two strategies. -If the key implements `PartitionAware`, it works like the `DefaultPartitioningStrategy`. -If it is a string key, it works like the `StringPartitioningStrategy`. - -Following are the example configuration snippets. Note that these strategy configurations are **per map**. - -**Declarative Configuration:** - -[tabs] -==== -XML:: -+ --- -[source,xml] ----- - - ... - - - com.hazelcast.partition.strategy.StringAndPartitionAwarePartitioningStrategy - - - ... - ----- --- - -YAML:: -+ -[source,yaml] ----- -hazelcast: - map: - name-of-the-map: - partition-strategy: com.hazelcast.partition.strategy.StringAndPartitionAwarePartitioningStrategy ----- -==== - -**Programmatic Configuration:** - -[source,java] ----- -Config config = new Config(); -MapConfig mapConfig = config.getMapConfig("name-of-the-map"); -PartitioningStrategyConfig psConfig = mapConfig.getPartitioningStrategyConfig(); -psConfig.setPartitioningStrategyClass( "com.hazelcast.partition.strategy.StringAndPartitionAwarePartitioningStrategy" ); - -// OR -psConfig.setPartitioningStrategy(YourCustomPartitioningStrategy); -... ----- - -You can also define your own partition strategy by implementing the class `PartitioningStrategy`. -To enable your implementation, add the full class name to your Hazelcast configuration using either -the declarative or programmatic approach, as exemplified above. - -The examples above show how to define a partitioning strategy per map. -Note that all the members of your cluster must have the same -partitioning strategy configurations. - -You can also change a global strategy which is applied to all the data structures in your cluster. -This can be done by defining the `hazelcast.partitioning.strategy.class` system property. -An example declarative way of configuring this property is shown below: - -[tabs] -==== -XML:: -+ --- -[source,xml] ----- - - ... - - - com.hazelcast.partition.strategy.StringAndPartitionAwarePartitioningStrategy - - - ... - ----- --- - -YAML:: -+ -[source,yaml] ----- -hazelcast: - properties: - hazelcast.partitioning.strategy.class: com.hazelcast.partition.strategy.StringAndPartitionAwarePartitioningStrategy ----- -==== - -You can specify the aforementioned out-of-the-box strategies or your custom -partitioning strategy. - -You can also use other system property configuring options as explained in the -xref:configuration:configuring-with-system-properties.adoc[Configuring with System Properties section]. - -The per map and global (cluster) partitioning strategies are supported on the member side. -Hazelcast Java clients only support the global strategy and it is configured -via the same system property used in the members (`hazelcast.partitioning.strategy.class `). - == CPU Thread Affinity Hazelcast offers configuring CPU threads so that you have a lot better control diff --git a/docs/modules/cluster-performance/pages/data-affinity.adoc b/docs/modules/cluster-performance/pages/data-affinity.adoc new file mode 100644 index 000000000..0229c0ea4 --- /dev/null +++ b/docs/modules/cluster-performance/pages/data-affinity.adoc @@ -0,0 +1,369 @@ += Data Affinity +:description: Data affinity ensures that related entries exist on the same member. If related data is on the same member, operations can +be executed without the cost of extra network calls and extra wire data. This feature is provided by using the same partition keys for related data. + +{description} + +You can achieve data affinity by implementing the <> interface or applying the <> as described in the following topics. + +== PartitionAware + +Hazelcast has a standard way of finding out which member owns or manages each key object. +The operations in the following example code are routed to the same member, since all of them are operating based on the same `key1` key. + +[source,java] +---- +HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(); +Map mapA = hazelcastInstance.getMap( "mapA" ); +Map mapB = hazelcastInstance.getMap( "mapB" ); +Map mapC = hazelcastInstance.getMap( "mapC" ); + +// since map names are different, operation will be manipulating +// different entries, but the operation will take place on the +// same member since the keys ("key1") are the same +mapA.put( "key1", value ); +mapB.get( "key1" ); +mapC.remove( "key1" ); + +// lock operation will still execute on the same member +// of the cluster since the key ("key1") is same +hazelcastInstance.getLock( "key1" ).lock(); + +// distributed execution will execute the 'runnable' on the +// same member since "key1" is passed as the key. +hazelcastInstance.getExecutorService().executeOnKeyOwner( runnable, "key1" ); +---- + +When the keys are the same, entries are stored on the same member. +But we sometimes want to have related entries stored on the same member, such as a customer and their order entries. +We would have a customers map with customerId as the key and an orders map with orderId as the key. +Since customerId and orderId are different keys, a customer and +their orders may fall into different members in your cluster. So how can we have them stored on the same member? +We create an affinity between customer and orders. If we make them part of the same partition then +these entries will be co-located. We achieve this by making `orderKey` s `PartitionAware`. + +[source,java] +---- +include::ROOT:example$/performance/OrderKey.java[tag=orderkey] +---- + +Notice that OrderKey implements `PartitionAware` and that `getPartitionKey()` returns the `customerId`. +These make sure that the `Customer` entry and its ``Order``s are stored on the same member. + +[source,java] +---- +HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(); +Map mapCustomers = hazelcastInstance.getMap( "customers" ); +Map mapOrders = hazelcastInstance.getMap( "orders" ); + +// create the customer entry with customer id = 1 +mapCustomers.put( 1, customer ); + +// now create the orders for this customer +mapOrders.put( new OrderKey( 21, 1 ), order ); +mapOrders.put( new OrderKey( 22, 1 ), order ); +mapOrders.put( new OrderKey( 23, 1 ), order ); +---- + +Assume that you have a customers map where `customerId` is the key and the customer object is the value. +You want to remove one of the customer orders and return the number of remaining orders. +Here is how you would normally do it. + +[source,java] +---- +public static int removeOrder( long customerId, long orderId ) throws Exception { + IMap mapCustomers = instance.getMap( "customers" ); + IMap mapOrders = hazelcastInstance.getMap( "orders" ); + + mapCustomers.lock( customerId ); + mapOrders.remove( new OrderKey(orderId, customerId) ); + Set orders = orderMap.keySet(Predicates.equal( "customerId", customerId )); + mapCustomers.unlock( customerId ); + + return orders.size(); +} +---- + +There are couple of things you should consider. + +* There are four distributed operations there: lock, remove, keySet, unlock. Can you reduce +the number of distributed operations? +* The customer object may not be that big, but can you not have to pass that object through the +wire? Think about a scenario where you set order count to the customer object for fast access, so you +should do a get and a put, and as a result, the customer object is passed through the wire twice. + +Instead, why not move the computation over to the member (JVM) where your customer data resides. +Here is how you can do this with distributed executor service. + +. Send a `PartitionAware` `Callable` task. +. `Callable` does the deletion of the order right there and returns with the remaining +order count. +. Upon completion of the `Callable` task, return the result (remaining order count). You +do not have to wait until the task is completed; since distributed executions are asynchronous, +you can do other things in the meantime. + +Here is an example code. + +[source,java] +---- +HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(); + + public int removeOrder(long customerId, long orderId) throws Exception { + IExecutorService executorService = hazelcastInstance.getExecutorService("ExecutorService"); + + OrderDeletionTask task = new OrderDeletionTask(customerId, orderId); + Future future = executorService.submit(task); + int remainingOrders = future.get(); + + return remainingOrders; + } + + public static class OrderDeletionTask + implements Callable, PartitionAware, Serializable, HazelcastInstanceAware { + + private long orderId; + private long customerId; + private HazelcastInstance hazelcastInstance; + + public OrderDeletionTask() { + } + + public OrderDeletionTask(long customerId, long orderId) { + this.customerId = customerId; + this.orderId = orderId; + } + + @Override + public Integer call() { + IMap customerMap = hazelcastInstance.getMap("customers"); + IMap orderMap = hazelcastInstance.getMap("orders"); + + customerMap.lock(customerId); + + Predicate predicate = Predicates.equal("customerId", customerId); + Set orderKeys = orderMap.localKeySet(predicate); + int orderCount = orderKeys.size(); + for (OrderKey key : orderKeys) { + if (key.orderId == orderId) { + orderCount--; + orderMap.delete(key); + } + } + + customerMap.unlock(customerId); + + return orderCount; + } + + @Override + public Object getPartitionKey() { + return customerId; + } + + @Override + public void setHazelcastInstance(HazelcastInstance hazelcastInstance) { + this.hazelcastInstance = hazelcastInstance; + } + } +---- + +The following are the benefits of doing the same operation with distributed `ExecutorService` based on the key: + +* only one distributed execution (`executorService.submit(task)`), instead of four +* less data is sent over the wire +* less lock duration, i.e., higher concurrency, for the `Customer` entry since +lock/update/unlock cycle is done locally (local to the customer data) + +[[partitioningstrategy]] +== Partitioning Strategies + +Another way of storing the related data on the same location is using or implementing +the partitioning strategies. Normally (if no partitioning strategy is defined), +Hazelcast finds the partition of a key first by converting the object to binary and then by hashing this binary. +If a partitioning strategy is defined, Hazelcast injects the key to the strategy and +the strategy returns an object out of which the partition is calculated by hashing it. + +Hazelcast offers the following out-of-the-box partitioning strategies: + +* `DefaultPartitioningStrategy`: Default strategy. It checks whether the key implements `PartitionAware`. +If it does, the object is converted to binary and then hashed, to find the partition of the key. +* `StringPartitioningStrategy`: Works only for string keys. It uses the string after `@` character as the partition ID. +For example, if you have two keys `ordergroup1@region1` and `customergroup1@region1`, +both `ordergroup1` and `customergroup1` fall into the partition where `region1` is located. +* `StringAndPartitionAwarePartitioningStrategy`: Works as the combination of the above two strategies. +If the key implements `PartitionAware`, it works like the `DefaultPartitioningStrategy`. +If it is a string key, it works like the `StringPartitioningStrategy`. +* Attribute Based Partitioning Strategy: Distributes data based on partition keys. See <>. + +You can configure the partitioning strategies for each map or for all data structures in your cluster (global strategy configuration). + +NOTE: The per map and global partitioning strategies are supported on the member side. +Hazelcast Java clients only support the global strategy. + +=== Per Map Partitioning Strategy Configuration + +For the declarative configurations (XML, YAML), you use the `partition-strategy` element. +For the programmatic approach, you use the `setPartitioningStrategyClass()` method. + +[tabs] +==== +XML:: ++ +-- +[source,xml] +---- + + ... + + + com.hazelcast.partition.strategy.StringAndPartitionAwarePartitioningStrategy + ///OR + ///YourCustomPartitioningStrategyClass <1> + + + ... + +---- +-- + +YAML:: ++ +[source,yaml] +---- +hazelcast: + map: + myMap: + partition-strategy: com.hazelcast.partition.strategy.StringAndPartitionAwarePartitioningStrategy + # OR + # partition-strategy: YourCustomPartitioningStrategyClass <1> +---- + +Java Member API:: ++ +[source,java] +---- +Config config = new Config(); +MapConfig mapConfig = config.getMapConfig("myMap"); +PartitioningStrategyConfig psConfig = mapConfig.getPartitioningStrategyConfig(); +psConfig.setPartitioningStrategyClass( "StringAndPartitionAwarePartitioningStrategy" ); + +// OR +psConfig.setPartitioningStrategy(YourCustomPartitioningStrategy); <1> +... +---- +==== +<1> You can define your own partition strategy by implementing the class https://docs.hazelcast.org/docs/{full-version}/javadoc/com/hazelcast/partition/PartitioningStrategy.html[`PartitioningStrategy`]. To enable your implementation, add the full class name to your Hazelcast configuration using either +the declarative or programmatic approach, as shown above. + +NOTE: All the cluster members must have the same partitioning strategy configurations. + +=== Global Partitioning Strategy Configuration + +You can also set a global strategy, which is applied to all the data structures in your cluster, using the `hazelcast.partitioning.strategy.class` property. + +The following shows example configurations. + +[tabs] +==== +XML:: ++ +-- +[source,xml] +---- + + ... + + + com.hazelcast.partition.strategy.StringAndPartitionAwarePartitioningStrategy + + + ... + +---- +-- + +YAML:: ++ +[source,yaml] +---- +hazelcast: + properties: + hazelcast.partitioning.strategy.class: com.hazelcast.partition.strategy.StringAndPartitionAwarePartitioningStrategy +---- + +Java Member/Client API:: ++ +[source,java] +---- +Config config = new Config(); +config.setProperty( "hazelcast.partitioning.strategy.class", "com.hazelcast.partition.strategy.StringAndPartitionAwarePartitioningStrategy" ); +---- +==== + +You can specify these out-of-the-box strategies or your custom +partitioning strategy using this property. + +You can also use the other property configuring options as explained in the +xref:configuration:configuring-with-system-properties.adoc[Configuring with System Properties] section. + +=== Attribute Based Partitioning Strategy + +Attribute based partitioning allows the map entries having the same partition keys live in the same cluster member. +It can be configured per map only, and cannot be used globally. + +To configure this strategy for a map, you can provide the `partition-attributes` configuration property under the `map` element. +If this property is provided, the partitioning strategy becomes attribute based, automatically superseding any other strategies if any one of them is configured as explained in the previous sections. +Note that, when provided, `partition-attributes` should contain at least one attribute name. + +Dynamic configuration is supported, which means that you can add or change the configuration of this strategy without restarting the cluster. + +IMPORTANT: Attribute based partitioning is only supported on the member-side. + +The following is an example flow which shows how attribute based partitioning strategy works. + +This example assumes that you have a map named `myMap` of type `IMap` where `Person` has `{ public Long id; public String name; public Long orgId; }`. + +. Start a cluster by providing a map configuration that includes the attributes `id` and `name` (or you can add this configuration while a cluster is running). ++ +[tabs] +==== +XML:: ++ +-- +[source,xml] +---- + + + + id + name + + ... + + +---- +-- + +YAML:: ++ +[source,yaml] +---- +hazelcast: + map: + myMap: + partition-attributes: + - name: "id" + - name: "name" + ... +---- +==== +. When data is added to the map using the `put` method, the strategy configured for this map is used to determine the partition ID of the new entry. Specifically, the `AttributePartitioningStrategy` extracts the `id` and `name` attributes from the given object, and creates an `Object[]` array out of them. This partition key is then used by the member logic to calculate the partition ID of the new entry. ++ +To give an example, assume that you have a key/value pair `Person(1, "John", 3), 1`, where `id`=`1`, `name`=`John`, and `orgId`=`3`. ++ +Add this map entry using `myMap.put(new Person(1, "John", 3), 1);`. The key of this entry (`new Person(1, "John", 3)`) will be passed to the configured strategy. +. The `AttributePartitioningStrategy` that is configured for this map takes the key and only extracts the `id` and `name` attributes from it (`1` and `John`), and creates the {1, "John"} partition key. +. The member logic then calculates the partition ID out of the `{1, "John"}` partition key, and puts the entry to the cluster member which has the calculated partition ID. +. A new entry with the same partition key will be put to the same cluster member. + +When you want to query for an entry with the key `new Person(1, "John", 3)`, the exact same turn of events described above occurs, except the operation will be `get` instead of `put`. diff --git a/docs/modules/cluster-performance/partials/nav.adoc b/docs/modules/cluster-performance/partials/nav.adoc index e590db513..11307bb84 100644 --- a/docs/modules/cluster-performance/partials/nav.adoc +++ b/docs/modules/cluster-performance/partials/nav.adoc @@ -1,4 +1,5 @@ * Improving Cluster Performance ** xref:cluster-performance:performance-tuning.adoc[] ** xref:cluster-performance:best-practices.adoc[] +** xref:cluster-performance:data-affinity.adoc[] ** xref:storage:high-density-memory.adoc[] diff --git a/docs/modules/sql/pages/partition-pruning.adoc b/docs/modules/sql/pages/partition-pruning.adoc index 4f8e8e5b9..a5224a71f 100644 --- a/docs/modules/sql/pages/partition-pruning.adoc +++ b/docs/modules/sql/pages/partition-pruning.adoc @@ -8,7 +8,8 @@ the required data on their partitions, and optimizes the count of Jet scan proce [NOTE] ==== * By default, partition pruning works for primitive `__key` types (default attribute strategy). -* It also works for all key types if you configure the `AttributePartitioningStrategy` in `MapConfig`. +* It also works for all key types if you configure the attribute based partitioning strategy. +See the xref:cluster-performance:data-affinity.adoc#attribute-based-partitioning-strategy[Attribute Based Partitioning Strategy] topic. ==== NOTE: The partition pruning may be used *only* when querying an IMap mapping if the partitioning strategy attribute is enabled.