EVEREST-400 encrypt secrets in secrets storage

[recharte]

EVEREST-400 encrypt secrets in secrets storage

Problem:
EVEREST-400

We were previously storing secrets in plain-text which obviously a really bad idea.

Solution:
Leverage hashicorp vault's AESGCM barrier module to encrypt the secrets in the postgres internal database.

CHECKLIST

Jira

• Is the Jira ticket created and referenced properly?

Tests

• [] Is an Integration test/test case added for the new feature/change?
• [] Are unit tests added where appropriate?

[ghost]

Generally looks good. Left a couple comments for discussion.

[ghost]

@recharte One other thing I just realized.
I was expecting we would connect to an external Vault instance - e.g. in case customers already run their own instance.

Do I understand it correctly that instead of running a separate instance we're running Vault as part of Everest backend?

[recharte]

@recharte One other thing I just realized. I was expecting we would connect to an external Vault instance - e.g. in case customers already run their own instance.

Do I understand it correctly that instead of running a separate instance we're running Vault as part of Everest backend?

Neither of those is what we want right now.

What you're describing is phase 2 of the Percona secrets management plan. In phase 2, the plan is to connect to a vault instance which can be external and managed by the user or bundled with a percona product.

What we are implementing right now is phase 1. This leverages the already existing backend DB (postgresql) and encrypts the secrets before persisting them in the DB.
We had many options for handling this encryption before storing but we decided on using the module used by vault to abstract this operation in simple Put, Get, Delete methods. So we are not using vault per se, we are using the same mechanism that vault uses to encrypt secret when using the postgresql physical backend.

[ghost]

lgtm

[oksana-grishchenko]

Great job! Pretty tricky way to migrate the secrets storage, thanks for adding the comments. It should work though, haven't found any misalignment. Concerns that appeared during the review are already discussed and resolved in the existing threads, nothing to add here.

If we plan to drop some of the migration logic in the later releases, I would suggest to create an Admin&Maintenance ticket to not forget about it.

[recharte]

Hey folks, I'm keeping this PR open until Thursday, as I'm waiting for a confirmation from legal regarding the BSL license of [email protected].