Ability to manage versioning and generation of document IDs in the index separately

[ermakvlas]

I want to use the key of the message in the topic as the id, but at the same time use the internal versioning of the elasticsearch. Now this is impossible.

Currently property key.ignore determines the mapping of the document id in index

kafka-connect-elasticsearch/src/main/java/io/confluent/connect/elasticsearch/DataConverter.java

Lines 159 to 162 in f2fdc49

	final String id = config.shouldIgnoreKey(record.topic())
	? String.format("%s+%d+%d", record.topic(), record.kafkaPartition(), record.kafkaOffset())
	: convertKey(record.keySchema(), record.key());

and versioning type at the same time

kafka-connect-elasticsearch/src/main/java/io/confluent/connect/elasticsearch/DataConverter.java

Lines 251 to 254 in f2fdc49

	if (!config.isDataStream() && !config.shouldIgnoreKey(record.topic())) {
	request.versionType(VersionType.EXTERNAL);
	request.version(record.kafkaOffset());
	}

By setting the value of property key.ignore=false I refuse to use offset when generating the id. However, at the same time, it is forced to use external versioning, which without options uses offset as the version number. It looks illogical, maybe there are some reasons for this behavior?
Forcing the use of offset for versioning can be undesirable in cases where there is a possibility of resetting the offset on the topic that the connector reads. In such a situation, new versions of existing documents will have a smaller version (offset will reset to 0) and the update will fail. Further correct updating of documents in this case will require a complete re-indexing of the data.