Merge remote-tracking branch 'upstream/main'

.github/workflows/backport.yml

@@ -3,32 +3,28 @@ on:
  push:
   branches:
    - main
+
+env:
+  MAINTENANCE_BRANCHES: "['5.3', '5.4', '5.5']"
+
 jobs:
-  get-maintenance-branches:
-    runs-on: ubuntu-latest
-    outputs:
-      branches: "['5.3', '5.4', '5.5']"
-    steps:
-      - run: exit 0
 
   backport-to-all-branch:
-    needs: get-maintenance-branches
     strategy:
       fail-fast: false
       matrix:
-        branch: ${{ fromJSON(needs.get-maintenance-branches.outputs.branches) }}
+        branch: ${{ fromJSON(env.MAINTENANCE_BRANCHES) }}
     uses: ./.github/workflows/backport-workflow.yml
     with:
       label-to-check-for: '["backport to all versions"]'
       target-branch: v/${{ matrix.branch }}
     secrets: inherit
 
   backport-to-specified-branch:
-    needs: get-maintenance-branches
     strategy:
       fail-fast: false
       matrix:
-        branch: ${{ fromJSON(needs.get-maintenance-branches.outputs.branches) }}
+        branch: ${{ fromJSON(env.MAINTENANCE_BRANCHES) }}
     uses: ./.github/workflows/backport-workflow.yml
     with:
       label-to-check-for: '["backport to ${{ matrix.branch }}"]'

.github/workflows/build-site.yml

@@ -8,7 +8,4 @@ on:
 
 jobs:
   dispatch:
-    runs-on: ubuntu-latest
-    steps:
-    - name: Trigger build
-      run: curl -X POST -d {} https://api.netlify.com/build_hooks/60e71936b15bb431b42b698a
+    uses: ./.github/workflows/deploy-docs-workflow.yml

.github/workflows/deploy-docs-workflow.yml

@@ -0,0 +1,13 @@
+name: Deploy docs
+description: Dispatch Netlify production deployment
+
+on:
+  workflow_call:
+
+jobs:
+  deploy:
+    runs-on: ubicloud-standard-2
+    steps:
+      - name: Dispatch docs deployment
+        #   https://docs.netlify.com/configure-builds/build-hooks/
+        run: curl -X POST -d {} https://api.netlify.com/build_hooks/60e71936b15bb431b42b698a

docs/modules/configuration/pages/understanding-configuration.adoc

@@ -23,7 +23,7 @@ The following topics are also relevant to static configuration:
 - xref:pattern-matcher.adoc[Configuration Pattern Matcher]
 - xref:using-wildcards.adoc[Using Wildcards]
 
-NOTE: Hazelcast performs schema validation through the `hazelcast-config-{full-version}.xsd` file,
+NOTE: Hazelcast performs schema validation through the `hazelcast-config-{minor-version}.xsd` file,
 which comes with Hazelcast libraries. If an error occurs in declarative or programmatic configuration, Hazelcast throws a meaningful exception.
 
 Static configuration cannot be changed at runtime. However, you can add <<dynamic-configuration, dynamic configuration>> for some features.

docs/modules/data-structures/pages/distributed-data-structures.adoc

@@ -143,6 +143,7 @@ when performing concurrent activities.
 |<<ap-data,Availability and partition tolerance>>
 |===
 
+[#aiml-data-structures]
 == AI/ML Data Structures
 [cols="20%a,40%a,20%a,20%a"]
 |===

docs/modules/data-structures/pages/fencedlock.adoc

@@ -51,7 +51,7 @@ image:ROOT:FencedLock.png[Fenced Lock]
 
 You can read more about the fencing token idea in Martin Kleppmann's
 https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html[How to do distributed locking]
-blog post and Google's https://ai.google/research/pubs/pub27897[Chubby paper].
+blog post and Google's https://static.googleusercontent.com/media/research.google.com/en//archive/chubby-osdi06.pdf[Chubby paper].
 `FencedLock` integrates this idea with the `j.u.c.locks.Lock` abstraction,
 excluding `j.u.c.locks.Condition`. `newCondition()` is not implemented and
 throws `UnsupportedOperationException`.
@@ -112,4 +112,4 @@ dead member will be removed so that those
 locks are immediately available for live members.
 * Locks are re-entrant. The same thread can lock multiple times on the same lock.
 Note that for other threads to be able to require this lock, the owner of the lock
-must call `unlock` as many times as the owner called `lock`.
+must call `unlock` as many times as the owner called `lock`.

docs/modules/data-structures/pages/vector-collections.adoc

@@ -50,6 +50,17 @@ Can include letters, numbers, and the symbols `-`, `_`, `*`.
 |Information about indexes configuration
 |Required
 |`NULL`
+
+|backup-count
+|Number of synchronous backups. See xref:data-structures:backing-up-maps.adoc#in-memory-backup-types[Backup Types]
+|Optional
+|`1`
+
+|async-backup-count
+|Number of asynchronous backups. See xref:data-structures:backing-up-maps.adoc#in-memory-backup-types[Backup Types]
+|Optional
+|`0`
+
 |===
 
 .Index configuration options
@@ -75,19 +86,19 @@ For further information on distance metrics, see the <<available-metrics, Availa
 |`N/A`
 
 |max-degree
-|Used to calculate the maximum number of neighbors per node. The calculation used is max-degree * 2
-|Required
-|`16`
+|Maximum number of neighbors per node. Note that the meaning of this parameter differs from that used in version 5.5.
+|Optional
+|`32`
 
 |ef-construction
 |The size of the search queue to use when finding nearest neighbors.
-|Required
+|Optional
 |`100`
 
 |use-deduplication
-|Whether or not to use vector deduplication.
+|Whether to use vector deduplication.
 When disabled, each added vector is treated as a distinct vector in the index, even if it is identical to an existing one. When enabled, the index consumes less space as duplicates share a vector, but the time required to add a vector increases.
-|Required
+|Optional
 |`TRUE`
 
 |===
@@ -125,6 +136,8 @@ XML::
 ----
 <hazelcast>
     <vector-collection name="books">
+        <backup-count>1</backup-count>
+        <async-backup-count>0</async-backup-count>
         <indexes>
             <index name="word2vec-index">
                 <dimension>6</dimension>
@@ -150,6 +163,8 @@ YAML::
 hazelcast:
   vector-collection:
     books:
+      backup-count: 1
+      async-backup-count: 0
       indexes:
         - name: word2vec-index
           dimension: 6
@@ -169,6 +184,8 @@ Java::
 ----
 Config config = new Config();
 VectorCollectionConfig collectionConfig = new VectorCollectionConfig("books")
+    .setBackupCount(1)
+    .setAsyncBackupCount(0)
     .addVectorIndexConfig(
             new VectorIndexConfig()
                 .setName("word2vec-index")
@@ -191,7 +208,7 @@ Python::
 --
 [source,python]
 ----
-client.create_vector_collection_config("books", indexes=[
+client.create_vector_collection_config("books", backup_count=1, async_backup_count=0, indexes=[
     IndexConfig(name="word2vec-index", metric=Metric.DOT, dimension=6),
     IndexConfig(name="glove-index", metric=Metric.DOT, dimension=10,
                 max_degree=32, ef_construction=256, use_deduplication=False),
@@ -692,17 +709,5 @@ As this is a beta version, Vector Collection has some limitations; the most sign
 
 1. The API could change in future versions
 2. The rolling-upgrade compatibility guarantees do not apply for vector collections. You might need to delete existing vector collections before migrating to a future version of Hazelcast
-3. The lack of fault tolerance, as backups cannot yet be configured. However, data in collections is migrated to other cluster members on graceful shutdown and a new member joining the cluster, which means that normal cluster maintenance (such as a rolling restart) is possible without data loss.
-4. Only on-heap storage of vector collections is available
-
-
-== Known issue
-
-There is currently a known issue that has potential for causing a memory leak in Vector collections in some scenarios:
-
-1. Using `destroy` or `clear` on a non-empty vector collection.
-2. Making a vector collection partition empty by removing the last entry using `deleteAsync` or `removeAsync`.
-3. Updating the only entry in a vector collection partition using one of the `put`/`set` methods.
-4. Repeating migrations back and forth when a member is not restarted. This should not significantly affect rolling restart, provided sufficient heap margin. The leak may manifest itself when a subset of members is not restarted and the rest of them are repeatedly shut down or restarted gracefully.
+3. Only on-heap storage of vector collections is available
 
-The workaround for scenarios 1 - 3 is to avoid those situations or restart the affected cluster. For scenario 4 the workaround is to restart the affected member or cluster. The restart can be graceful, which should not cause loss of data. 

docs/modules/data-structures/pages/vector-search-overview.adoc

@@ -43,17 +43,25 @@ The index is based on the link:https://github.com/jbellis/jvector[JVector] libra
 
 Each collection is partitioned and replicated based on the system's general partitioning rules. Data partitioning is carried out using the collection key.
 
-For further information on Hazelcast partitioning, see xref:architecture:data-partitioning.adoc[Data Partitioning and Replication].
+Vector collection is an xref:distributed-data-structures.adoc#aiml-data-structures[AP data structure] and implements standard xref:data-structures:backing-up-maps.adoc#in-memory-backup-types[in-memory backup types].
+Vector collection does not currently support xref:data-structures:backing-up-maps.adoc#enabling-in-memory-backup-reads-embedded-mode [reading from backup] and xref:data-structures:backing-up-maps.adoc#file-based-backups[file based backups].
 
-NOTE: Version 5.5/beta supports partitioning and migration but does not include support for the backup process.
+For further information on Hazelcast partitioning, see xref:architecture:data-partitioning.adoc[Data Partitioning and Replication].
 
 === Data store
+
 Hazelcast stores data in-memory (RAM) for faster access. Presently, the only available data storage option is the JVM heap store.
 
 === Fault Tolerance
+
 Hazelcast distributes storage data across all cluster members.
 In the event of a graceful shutdown, the data is migrated to remaining active members.
-In version 5.5, there is no automatic data restoration in the event of an unexpected member loss.
+In the event of member crash, if backups were configured, they are used to restore the data.
+If no backups were configured, data can be lost.
+
+NOTE: You can disable backups to speed up data ingestion into Vector Collection and reduce memory usage
+in development and test environments which do not require fault tolerance.
+With a single-member cluster (for example, an embedded dev cluster), you do not need to disable backups because these are not used in this case.
 
 == Partitioned similarity search
 

docs/modules/deploy/pages/deploying-on-azure.adoc

@@ -307,4 +307,4 @@ config.getPartitionGroupConfig()
 
 == Automated Deployment
 
-You can also use the https://github.com/Azure/azure-quickstart-templates/tree/master/hazelcast-vm-cluster[Azure Hazelcast Template] to automatically deploy a Hazelcast cluster.
+You can also use the https://github.com/Azure/azure-quickstart-templates/blob/master/application-workloads/hazelcase/hazelcast-vm-cluster/README.md[Azure Hazelcast Template] to automatically deploy a Hazelcast cluster.

docs/modules/fault-tolerance/pages/backups.adoc

@@ -45,6 +45,7 @@ for the data structures:
 * xref:data-structures:set.adoc#configuring-set[Sets], xref:data-structures:list.adoc#configuring-list[Lists]
 * xref:data-structures:ringbuffer.adoc#backing-up-ringbuffer[Ringbuffers]
 * xref:data-structures:cardinality-estimator-service.adoc[Cardinality Estimators]
+* xref:data-structures:vector-collections.adoc#configuration[Vector Collections]
 
 See xref:fault-tolerance:fault-tolerance.adoc[this section] for backup information about the Hazelcast jobs.
 

docs/modules/getting-started/pages/install-enterprise.adoc

@@ -60,7 +60,7 @@ Windows::
 +
 --
 ifdef::snapshot[]
-Download and extract the link:https://repository.hazelcast.com/snapshot/com/hazelcast/hazelcast-enterprise/hazelcast-enterprise-{full-version}.zip[Hazelcast ZIP file].
+Download and extract the Hazelcast ZIP file from link:https://hazelcast.jfrog.io/ui/native/snapshot/com/hazelcast/hazelcast-enterprise-distribution/{full-version}[the repository].
 endif::[]
 ifndef::snapshot[]
 Download and extract the link:https://repository.hazelcast.com/download/hazelcast-enterprise/hazelcast-enterprise-{full-version}.zip[Hazelcast ZIP file].

docs/modules/getting-started/pages/install-hazelcast.adoc

@@ -69,7 +69,7 @@ If you do not see a version number, download and install link:https://www.gnu.or
 ----
 wget -qO - https://repository.hazelcast.com/api/gpg/key/public | gpg --dearmor | sudo tee /usr/share/keyrings/hazelcast-archive-keyring.gpg > /dev/null
 echo "deb [signed-by=/usr/share/keyrings/hazelcast-archive-keyring.gpg] https://repository.hazelcast.com/debian stable main" | sudo tee -a /etc/apt/sources.list
-sudo apt update && sudo apt install hazelcast={last-oss-version}
+sudo apt update && sudo apt install hazelcast={os-version}
 ----
 +
 .RPM
@@ -135,13 +135,8 @@ Mac::
 Linux:: 
 + 
 --
-ifdef::snapshot[]
-Go to the link:https://oss.sonatype.org/content/repositories/snapshots/com/hazelcast/hazelcast-distribution/{full-version}/[snapshot repository] and click the download link for the TAR file that has the most up-to-date timestamp in the *Last Modified* columns.
-endif::[]
-ifndef::snapshot[]
 . Download a package from https://hazelcast.com/get-started/download/?utm_source=docs-website[hazelcast.com^].
 . Extract the downloaded package.
-endif::[]
 --
 Windows:: 
 +
@@ -200,7 +195,10 @@ The Java package includes both a member API and a Java client API. The member AP
 If you aren't using a build tool:
 
 ifdef::snapshot[]
-* link:https://oss.sonatype.org/content/repositories/snapshots/com/hazelcast/hazelcast/{last-oss-version}/[download the Hazelcast JAR file]
+* link:https://repo1.maven.org/maven2/com/hazelcast/hazelcast/{os-version}/hazelcast-{os-version}/[download the Hazelcast JAR file]
+endif::[]
+ifndef::snapshot[]
+* link:https://repo1.maven.org/maven2/com/hazelcast/hazelcast/{full-version}/hazelcast-{full-version}.jar[download the Hazelcast JAR file]
 endif::[]
 * add it to your classpath.
 

docs/modules/integrate/pages/cdc-connectors.adoc

@@ -62,12 +62,12 @@ PostgreSQL-specific connector:
 
 The Java API supports the following types of CDC source:
 
-* link:https://docs.hazelcast.org/docs/{full-version}/javadoc/com/hazelcast/enterprise/jet/cdc/DebeziumCdcSources.html[DebeziumCdcSources, window=_blank]:
+* link:https://docs.hazelcast.org/hazelcast-ee-docs/{full-version}/javadoc/com/hazelcast/enterprise/jet/cdc/DebeziumCdcSources.html[DebeziumCdcSources, window=_blank]:
   a generic source for all databases supported by Debezium
-* link:https://docs.hazelcast.org/docs/{full-version}/javadoc/com/hazelcast/enterprise/jet/cdc/mysql/MySqlCdcSources.html[MySqlCdcSources, window=_blank]:
+* link:https://docs.hazelcast.org/hazelcast-ee-docs/{full-version}/javadoc/com/hazelcast/enterprise/jet/cdc/mysql/MySqlCdcSources.html[MySqlCdcSources, window=_blank]:
   a specific, first class Jet CDC source for MySQL databases (also based
   on Debezium, but with the additional benefits provided by Hazelcast)
-* link:https://docs.hazelcast.org/docs/{full-version}/javadoc/com/hazelcast/enterprise/jet/cdc/postgres/PostgresCdcSources.html[PostgresCdcSources, window=_blank]:
+* link:https://docs.hazelcast.org/hazelcast-ee-docs/{full-version}/javadoc/com/hazelcast/enterprise/jet/cdc/postgres/PostgresCdcSources.html[PostgresCdcSources, window=_blank]:
   a specific, first class CDC source for PostgreSQL databases (also based
 on Debezium, but with the additional benefits provided by Hazelcast)
 

docs/modules/maintain-cluster/pages/rolling-upgrades.adoc

@@ -15,7 +15,7 @@ Before starting a rolling upgrade, consider the following best practices:
 * Ensure that <<hazelcast-members-compatibility,Upgrade Compatibility>> covers your requirements
 * If the minimum version of JDK has changed between releases, <<updated-jdk,upgrade the JDK version>> before starting a rolling upgrade
 * If you used Hazelcast Operator to deploy your cluster, refer to the link:https://docs.hazelcast.com/operator/latest/scaling-upgrading#upgrading[Operator documentation, window=_blank]
-* If using Hazelcast Management Center to trigger your rolling upgrade, upgrade Management Center before upgrading your members. For information on upgrading Management Center, refer to the link:https://docs.hazelcast.com/management-center/latest/getting-started/install[Management Center documentation, window=_blank], or if you deployed Management Center using the Hazelcast Operator, refer to the link:https://docs.hazelcast.com/operator/latest/deploy-management-center.adoc[Operator documentation, window=_blank]
+* If using Hazelcast Management Center to trigger your rolling upgrade, upgrade Management Center before upgrading your members. For information on upgrading Management Center, refer to the link:https://docs.hazelcast.com/management-center/latest/getting-started/install[Management Center documentation, window=_blank], or if you deployed Management Center using the Hazelcast Operator, refer to the link:https://docs.hazelcast.com/operator/latest/deploy-management-center[Operator documentation, window=_blank]
 * If using the `hz-cluster-admin` script to update your cluster, enable the `CLUSTER_WRITE` endpoints in the REST API. For further information on the {open-source-product-name} REST API, see xref:maintain-cluster:rest-api.adoc#using-the-rest-endpoint-groups[{open-source-product-name} REST API]. For further information on the {enterprise-product-name} REST API, see xref:maintain-cluster:enterprise-rest-api.adoc#dynamic-configuration-update-rest-endpoint[{enterprise-product-name} REST API]
 * Plan sufficient time for the migration and upgrade to complete, and remember that the members cannot change during the upgrade
 * Avoid issues where an automatic upgrade is triggered before the last member has been upgraded by setting a <<enabling-auto-upgrading,minimum cluster size>>

docs/modules/pipelines/pages/cdc.adoc

@@ -152,8 +152,13 @@ mysql> SELECT * FROM customers;
 +
 If you already have Hazelcast and you skipped the above steps, make sure to
 follow from here on.
-
-. Make sure the MySQL CDC plugin is in the `lib/` directory. You must manually download the MySQL CDC plugin from link:https://repo1.maven.org/maven2/com/hazelcast/jet/hazelcast-enterprise-cdc-mysql/{full-version}/hazelcast-enterprise-cdc-mysql-{full-version}-jar-with-dependencies.jar[Hazelcast's Maven repository, window=_blank] and then copy it to the `lib/` directory.
+ifdef::snapshot[]
+Download and extract the link:https://repository.hazelcast.com/snapshot/com/hazelcast/hazelcast-enterprise/hazelcast-enterprise-{full-version}.zip[Hazelcast ZIP file].
+. Make sure the MySQL CDC plugin is in the `lib/` directory. You must manually download the MySQL CDC plugin's `jar-with-dependencies` from link:https://hazelcast.jfrog.io/artifactory/snapshot/com/hazelcast/jet/hazelcast-enterprise-cdc-mysql/{full-version}[Hazelcast's Maven repository, window=_blank] and then copy it to the `lib/` directory.
+endif::[]
+ifndef::snapshot[]
+. Make sure the MySQL CDC plugin is in the `lib/` directory. You must manually download the MySQL CDC plugin from link:https://hazelcast.jfrog.io/artifactory/release/com/hazelcast/jet/hazelcast-enterprise-cdc-mysql/{full-version}/hazelcast-enterprise-cdc-mysql-{full-version}-jar-with-dependencies.jar[Hazelcast's Maven repository, window=_blank] and then copy it to the `lib/` directory.
+endif::[]
 +
 [source,bash]
 ----

docs/modules/pipelines/pages/spring-boot.adoc

@@ -3,7 +3,7 @@
 Spring Boot makes it easy to create and use third-party libraries, such
 as Hazelcast, with minimum configurations possible. While Spring
 Boot provides starters for some libraries, Hazelcast hosts its own
-link:https://github.com/hazelcast/hazelcast-jet-contrib/tree/master/hazelcast-jet-spring-boot-starter[starter].
+link:https://github.com/hazelcast/hazelcast-jet-contrib/tree/jet-4.5-maintenance/hazelcast-jet-spring-boot-starter[starter].
 
 Let's create a simple Spring Boot application which starts a Hazelcast
 member and auto-wires it.