Skip to content

Encrypt with OpenSSL

Yujia Li edited this page Aug 3, 2020 · 10 revisions

caf provide following free function in caf::openssl to support SSL which have same signatures with free function in caf::io.

expected<uint16> publish(T, uint16, const char*, bool)
expected<void> unpublish(T x, uint16)
expected<T> remote_actor<T = actor>(actor_system&, string, uint16)

Following free function have no corresponding openssl version. If you want to encrypt, try to avoid using these functions. If you already publish actor(s) on one node, you don't need open. If you want a node_id for remote spawn, you can get it from remote actor handle actor->node().

expected<uint16> open(actor_system&, uint16, const char*, bool)
expected<node_id> connect(actor_system&, std::string host, uint16_t port)

Don't mix caf.io and caf.openssl, If you need encryption, use the openssl module consistently. Once the connections are established, CAF doesn't know whether a connection is encrypted or not and always discards extra connections to nodes it is already connected to.

Usage

To use these openssl functions, you should loading openssl and middleman module first.

CAF_MAIN(caf::io::middleman, caf::openssl::manager)

And include necessary header files caf/openssl/all.hpp.

caf use Mutual authentication, client and server both need certificate.

You can use following configuration options to use openssl:

--openssl.cafile='xxx.ca' // Certification Authority (CA) certificate (PEM-formatted)
--openssl.certificate='xxx.pem' // certificate for you app (PEM-formatted)
--openssl.key='xxx.pem' // private key for you app

[optional]--openssl.passphrase='xxx' // passphrase of your private key
[optional]--openssl.capath='xxx' // path to an OpenSSL-style directory of trusted certificates

Demo

server.cpp

#include "caf/all.hpp"
#include "caf/io/all.hpp"
#include "caf/openssl/all.hpp"

using namespace caf;

behavior make_pong_behavior() {
    return {
            [](int val) -> int {
                ++val;
                std::cout << "pong " << val << std::endl;
                return val;
            },
            [](std::string str) {
                std::cout << "receive msg: " << str << std::endl;
            }
    };
}

void caf_main(actor_system &system) {
    auto spong = system.spawn(make_pong_behavior);
    caf::openssl::publish(spong, 50602, "localhost");
    std::cout << "already publish spong" << std::endl;

    std::string dummy;
    std::getline(std::cin, dummy);
}

CAF_MAIN(caf::io::middleman, caf::openssl::manager)

client.cpp

#include "caf/all.hpp"
#include "caf/io/all.hpp"
#include "caf/openssl/all.hpp"
#include <string>
#include <iostream>

using namespace caf;

void caf_main(actor_system &system) {
    auto result = caf::openssl::remote_actor(system, "localhost", 50602);
    if (!result) {
        std::cout << "connect failed, reason: " << system.render(result.error()) << std::endl;
        return;
    }
    std::cout << "address: " << caf::to_string(result->address()) << std::endl;
    auto spong = std::move(*result);
    caf::anon_send(spong, 1);
    caf::anon_send(spong, "hello");

    std::string dummy;
    std::getline(std::cin, dummy);
}

CAF_MAIN(caf::io::middleman, caf::openssl::manager)

run server with:

server --openssl.cafile=docker/openssl/ca-noenc.crt --openssl.certificate=docker/openssl/node1.cert.pem --openssl.key=docker/openssl/node1.key.pem

run client with:

client --openssl.cafile=docker/openssl/ca-noenc.crt --openssl.certificate=docker/openssl/node2.cert.pem --openssl.key=docker/openssl/node1.key.pem

You can find certificate and key files in docker/openssl dir in source code.

openssl commands to check infos inside certificate:

openssl x509 -in ca-noenc.crt -text -noout

Demo Script to Generate Self-signed Certiticate

mkdir private
mkdir cert
mkdir csr

openssl genrsa -out private/ca-noenc.key 2048
openssl req -new -key private/ca-noenc.key -out csr/ca-noenc.csr -subj "/C=US/ST=CA/L=Berkeley/O=ThoughtWorks/OU=HPC/CN=ca"
openssl x509 -req -days 3650 -in csr/ca-noenc.csr -signkey private/ca-noenc.key -out cert/ca-noenc.crt

openssl genrsa -out private/node1.key.pem 2048
openssl req -new -key private/node1.key.pem -out csr/node1.csr -subj "/C=US/ST=CA/L=Berkeley/O=ThoughtWorks/OU=HPC/CN=node1"
openssl x509 -req -in csr/node1.csr -CA cert/ca-noenc.crt -CAkey private/ca-noenc.key -out cert/node1.cert.pem -CAcreateserial -days 3650 -sha256

openssl genrsa -out private/node2.key.pem 2048
openssl req -new -key private/node2.key.pem -out csr/node2.csr -subj "/C=US/ST=CA/L=Berkeley/O=ThoughtWorks/OU=HPC/CN=node2"
openssl x509 -req -in csr/node2.csr -CA cert/ca-noenc.crt -CAkey private/ca-noenc.key -out cert/node2.cert.pem -CAcreateserial -days 3650 -sha256

openssl genrsa -out private/node3.key.pem 2048
openssl req -new -key private/node3.key.pem -out csr/node3.csr -subj "/C=US/ST=CA/L=Berkeley/O=ThoughtWorks/OU=HPC/CN=node3"
openssl x509 -req -in csr/node3.csr -CA cert/ca-noenc.crt -CAkey private/ca-noenc.key -out cert/node3.cert.pem -CAcreateserial -days 3650 -sha256

Known Issue

  1. Client will hangs indefinitely when using caf::io::remote while server use caf::openssl::publish. Github Issue 1119 fix in version: 0.17.6
  2. If you mix caf.io and caf.openssl, actors communication may be encrypt or plain. You can try using demo/openssl. Github Issue 1128

Using tcpdump to verify communication are encrypt

Install

Debian

apt install tcpdump

Usage

First, run your app. I will run docker-compose -f yanghui_app.yml up -d as demonstration.

And run tcpdump to catch data. I run tcpdump with following args.

tcpdump src yanghui_root_v2 and port not 4445 -X

src yanghui_root_v2 : only catch data send from host yanghui_root_v2.

port not 4445 : ignore data on port 4445 which is gossip in cdcf.

-X : display data in hex

You will catch data like this if using non-SSL:

02:29:01.479383 IP 172.29.0.4.47322 > 172.29.0.5.56601: Flags [P.], seq 4731:4788, ack 3985, win 237, options [nop,nop,TS val 2033937571 ecr 110022286], length 57
	0x0000:  4500 006d 1ba2 4000 4006 c6a5 ac1d 0004  E..m..@.@.......
	0x0010:  ac1d 0005 b8da dd19 b0ff d48c 40d2 51b3  [email protected].
	0x0020:  8018 00ed 58a3 0000 0101 080a 793b 6ca3  ....X.......y;l.
	0x0030:  068e ce8e 0200 0000 0000 0019 1000 0000  ................
	0x0040:  0000 01fe 0000 0000 0000 000e 0000 0000  ................
	0x0050:  0000 000b 0000 000d 403c 3e2b 4069 3332  ........@<>+@i32
	0x0060:  2b40 6933 3200 0000 4700 0000 07         [email protected]....

Flags P: this is a data PUSH.

We can see this in stdout of worker1:

false slow calculator received add task. input a:71 b:7

Number 71 in decimal same as number 47 in hex. And We can see 47 and 07 in the tail of data.

You will catch data like this if using SSL:

02:14:47.988394 IP docker_yanghui_root_v2_1.docker_cdcf.53616 > d8b0af399c61.56601: Flags [P.], seq 9718:9804, ack 8702, win 302, options [nop,nop,TS val 3303185067 ecr 1285104605], length 86
	0x0000:  4500 008a a468 4000 4006 3dc4 ac1c 0004  E....h@.@.=.....
	0x0010:  ac1c 0005 d170 dd19 a7b4 b0ea 3c15 e2d9  .....p......<...
	0x0020:  8018 012e 58be 0000 0101 080a c4e2 9aab  ....X...........
	0x0030:  4c99 23dd 1703 0300 51c6 152b f9f2 e5a6  L.#.....Q..+....
	0x0040:  8658 7bd9 af6b cb35 796d 7563 e711 9376  .X{..k.5ymuc...v
	0x0050:  32dc cde6 08e6 3e76 9ab0 785d b4f1 8c50  2.....>v..x]...P
	0x0060:  92ad adcc 4d7d 4c61 6541 b132 8f9f 2dfe  ....M}LaeA.2..-.
	0x0070:  6c13 f571 4522 5206 ea9a 2b15 87da c545  l..qE"R...+....E
	0x0080:  3f1a 8383 8782 7b37 eb19                 ?.....{7..

You can not grasp anything readable information this time. 0303 in data means using TLS 1.2.

Reference

The serial article will help you know about certificate related terminologies and how to generate self-signed certificate.