http://cinhtau.net/2018/06/07/encrypted-communication-elasticsearch-java-rest-client/

[utterances-bot]

Encrypted Communication with the Elasticsearch Java Rest Client

Elasticsearch with the X-Pack extensions allows you to secure the communication with your Elasticsearch cluster. This post elaborates what actions are needed...

http://cinhtau.net/2018/06/07/encrypted-communication-elasticsearch-java-rest-client/

[ayushmathur86]

Hi,
I'm currently trying to setup Java SSL client to connect with my ES cluster with HTTP SSL enabled and version 6.2.3.
However, for truststore and keystore, I'm using P12 certs generated with certutil utility of Elasticsearch.
Can you please let me know how did you create watcher-truststore.jks initially ?
I'd like to modify the settings as per this doc but not sure how to generate JKS from my P12 and which P12 should be used: CA (certutil ca) or Certificate (certutil cert --ca) ?

Thanks in advance.

[cinhtau]

@ayushmathur86 - It is written in the article. keytool is a program within the JDK/JRE.

You can either add them to the default truststore or create a new one.
I prefer the last option. To create a new truststore:

keytool -import \
   -alias elasticCA \
   -file /opt/elasticsearch/config/ca.crt \
   -keystore truststore.jks

The format p12 does not matter. You can import with keytool the p12 format or just copy contents form keystore to keystore. See https://cinhtau.net/2016/03/11/copy-contents-of-java-keystore-to-another-keystore/

If the certs are in the keystore, just rename truststore.jks to watcher-truststore.jks.

[ayushmathur86]

@cinhtau thank you for your reply.
I tried that already using truststore, but it gives me error saying Invalid keystore, hence the question here.
Also, what about the certificates generated for ES:
ssl:
key: certs/node.key
certificate: certs/node.crt
certificate_authorities: "certs/ca.crt"

How those can be generated from P12 self signed certs generated from certutil utility of Elasticsearch ?

[cinhtau]

@ayushmathur86 Can you tell me what you are trying to do? What command doesn't work for you?

Do you see the contents of this keystore?

keytool -list -keystore <your-key-storename.jks>

From version 6.1 to 6.2, Elastic switch its certificate format from cert to p12. If you have the p12 certs just import them in p12 format.

An example:

keytool -importkeystore \
-deststorepass [password] \
-destkeystore [filename-new-keystore.jks] \
-srckeystore [filename-new-PKCS-12.p12] \
-srcstoretype PKCS12

[ayushmathur86]

@cinhtau Currently the http.ssl and transport.ssl is configured to use p12 certificates on Elasticsearch, but in your case here, you're using PEM formatted ones.

1. Should I use CA to create the PEM or Node certificate ?
2. I tried using the above command to create JKS from CA p12 certificate and here is the output:

Importing keystore elastic-stack-ca.p12 to client.jks...
Enter destination keystore password:
Re-enter new password:
Enter source keystore password:

*****************  WARNING WARNING WARNING  *****************
* The integrity of the information stored in the srckeystore*
* has NOT been verified!  In order to verify its integrity, *
* you must provide the srckeystore password.                *
*****************  WARNING WARNING WARNING  *****************

keytool error: java.lang.NullPointerException: invalid null input

[ayushmathur86]

Ahh, nevermind about 2nd point, I sorted out -srcstorepass should have been provided for blank/no passwords.
But which certificate should be used to generate PEM for ES is still the question.

[cinhtau]

But which certificate should be used to generate PEM for ES is still the question.

@ayushmathur86 Can you describe what you have tried with certutil?

See also https://www.elastic.co/guide/en/elasticsearch/reference/6.2/certutil.html

[ayushmathur86]

@cinhtau I have used just 2 commands to create my CA and node certificate:
certutil ca
certutil cert --ca ca_file_from_above.p12

From the link you mentioned, for Cert mode:
The cert mode generates X.509 certificates and private keys. By default, it produces a single certificate and key for use on a single instance.
Is there any possibility that same certificate can be used by multiple nodes by adding it via Dockerfile ?

[ayushmathur86]

Also, if I will create those PEM format certificates, can I use the same set for Kibana and Fluentd which use PEM format again ?

[cinhtau]

The idea behind self signed certs is you have the root ca. For each participant (client or server), you generate for each particpants its own certificate. Since your root ca issued the certificate, you can trust the certificate. Kibana and fluentd should get their own certificates. Generate for each its own certificate. Using one PEM for all clients is not recommended. Regardless what you generate with certutil, you can always convert p12 to pem and vice versa. Look for openssl.

[ayushmathur86]

Thank you @cinhtau.
For now, I've generated jks cert for my Java client which has both private key entry and trusted CA entry and can be listed using:
keytool -list -keystore truststore.jks
However, when I'm running my app, it throws PKIX build path failed :unable to find valid certification path to requested target.
Any idea what I could be doing wrong here or do I really have to start from scratch by generating the certificates in PEM directly ?

[cinhtau]

@ayushmathur86 What is your command on the terminal/shell, when you start the Java Application?

You don't need a private key in a trust store. I strongly advise against it. From my understanding, you have a client who wants to access Elasticsearch?

                           ┌───────────────┐
┌─────────────┐            │               │
│             │            │               │
│ Java Client │───────────▶│ Elasticsearch │
│             │            │               │
└─────────────┘            │               │
                           └───────────────┘

You need the certificate of the Elasticsearch Server and the public ca, in order to establish a connection.

[ayushmathur86]

@cinhtau thank you for guiding me through the steps. I'm now able to connect with Elastic using https scheme by following your blog, 3 cheers for that :) !
However, I'm not sure why client is returning me:
Caught HTTP rest client exception: 401 Unauthorized
although I can connect to my cluster using the same credentials on browser and also access Kibana.

[cinhtau]

@ayushmathur86 401is a http code, that tell you either your user or password is wrong. Maybe both.

Can you check with curl?
Add the -k option to ignore the SSL certificate. Replace localhost with your ES server name.

curl -k -u <your-username> https://localhost:9200

[ayushmathur86]

Hi, Already tried curl with that user and works fine. Looks like I had to include auth interceptor to pass the creds despite the fact I was using CredentialProvider in SSL Async Configurer. It's working fine now and I'm able to connect and search for logs in my citrus client. Thanks a ton for helping me out :)

…

On Tue, Aug 27, 2019, 19:04 Tan-Vinh Nguyen ***@***.***> wrote: @ayushmathur86 <https://github.com/ayushmathur86> 401is a http code, that tell you either your user or password is wrong. Maybe both. Can you check with curl? Add the -k option to ignore the SSL certificate. curl -k -u <your-username> https://localhost:9200 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5?email_source=notifications&email_token=ACVUDQIP564AXP7BFESMPFTQGVF7TA5CNFSM4IPBSDD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5IIFQQ#issuecomment-525370050>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACVUDQI7XJ5ZZFRLZPWKC5LQGVF7TANCNFSM4IPBSDDQ> .

[manojbvn4u]

Hi I tried all the above steps but getting the below excpetion
javax.net.ssl.SSLHandshakeException : General SSLEngine problem
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Can you please help