Skip to content

BIP APIs

Jump to bottom
Afsin Ustundag edited this page Feb 15, 2023 · 15 revisions

VRO uses two BIP APIs.

Information specific to the individual API can be found the above pages. This page, in addition to providing these links, describes the details of VRO Mutual TLS implementation.

Mutual TLS (mTLS)

BIP APIs require mTLS. It is necesssary for VRO to use a client certificate signed by a VA recognized Certificate Authority (CA) during https handshake. In addition VRO needs to use a VA recognized CA to validate BIP certificates during https handshake.

Certificates

Generated Certificates

Venabi is the VA's new Internal Management System which can be used to manage the PKI certificates. Within VRO Kubernetes cluster the process is further simplified and it is possible to generate the certificates within one of VRO Kubernetes namespaces. The client certificates that are being used currently by VRO has been generated using these instructions.

The certificates tls.key, tls.crt, and va.crt has been generated in va-abd-rrd-dev namespace and can be found in va-abd-rrd-dev-va-gov-tls secrets.

The private key tls.key is not encrypted so there are no passwords needed for steps described later in this document.

Details for tls.crt can be displayed using

openssl x509 -in tls.crt -text -noout

The certificate tls.crt expires in January 6th, 2024.

The certificate tls.crt contains both the client public key and an intermediate CA. Only the client public key is used and will be referred as tls_bip.crt.

The certificate va.crt is not currently used.

Production Use

Based on discussions with LHDI support, the considerations of production use of certificates generated in va-abd-rrd-dev are

  • Are there any concerns with using the same cert for both dev and prod environments?
  • Are there any concerns with management across the different environments? For example, if you're using auto cert renewal what does the process look like for grabbing the new cert and copying to prod vs using two different certs and having each managed/automatically updated?

At this time we do not have these concerns so we are using the same certificates in prodtest and prod.

Future Work

As discussed later in this document we do not yet know a way to use these certificates directly in the Java code and there are manual steps to make them usable. More discussions are needed with LHDI around the process on how to update them.

  • Should we switch to auto cert renewal? What is the trigger to perform manual steps if switch is made?
  • Should LHDI make these certificates available to containers without additional steps?
  • Can we forego container level https and move that complexity to API Gateway so that it is managed by LHDI?

At this time cert renewal is manual and steps in this document should be followed. We have time until January 6th, 2024 to make improvement.

Downloaded Certificates

During VRO implementation of mTLS we were not successful using the VA CA public keys in va.crt and tls.crt.

Instead we downloaded the public keys that were used by BIP APIs from Chrome on a GFE:

  • Open Chrome on your GFE
  • Open one of the end points in BIP Claims API or BIP Claim Evidence API. If you use a GET end point you should typically get a jwt missing error
  • Once you get the error click on the lock (View site information) in the url bar to bring up a pop-up window
  • Select "Connection is secure" menu item to bring up a second pop-up window. From this pop-up you can select "Certificate is valid" to bring up the "Certificate viewer".
  • Goto "Details" tab, select the certificate for root CA and "Export". Repeat for the intermediate CA

Two certificates downloaded are

  • VA-Internal-S2-RCA1-v1.crt (root)
  • VA-Internal-S2-ICA4.crt (intermediate)

These public keys are concatenated in a file named va_all.crt for later use.

Future Work

We have not retried to use the VA CA public keys in va.crt and tls.crt after we successfully used va_all.crt. This should be revisited in the future since it is possible that the steps we eventually ended up with could be succesful for public keys in va.crt and tls.crt as well.

VRO mTLS Implementation

PKCS #12 Files

As of now we do not know a simple way to consume the certificates tls.key, tls_bip.crt, and va_all.crt directly in the Java code to utilize them in https calls. Java uses either JKS files or PKCS #12 files. Since PKCS #12 files are not Java specific, we prefer them over JKS files in VRO.

To generate the PKCS #12 file keystore.p12 for client certificates we use openssl

openssl pkcs12 -export -in tls_bip.crt -out keystore.p12 -name keystore -nodes -inkey tls.key

This commands asks for an "export password". This password is to be recorded as it needs to be later used as a secret.

To generate the PKCS #12 file truststore.p12 for VA CA certificates we use keytool

keytool -import -file va_all.crt -alias all_cas -keystore truststore.p12

This command asks for a "keystore password". For simplicity VRO uses the same the value of "export password".

Future Work

It should be possible either to use keystore or openssl exclusively here. But these PKCS #12 files worked and we did not try either route any further.

Kubernetes Secrets

VRO uses three environment variables to store the content of the [PKCS #12] files and the "export password".

  • BIP_KEYSTORE
  • BIP_TRUSTSTORE
  • BIP_PASSWORD

Since PKCS #12 files are binary we convert them to Base 64 using openssl

openssl base64 -in keystore.p12 -out keystore.b64
openssl base64 -in truststore.p12 -out truststore.b64

What is stored in the environment variables BIP_KEYSTORE and BIP_TRUSTSTORE are the content of the Base 64 files.

In our environments these environment variables are made available to the app container using the following secrets

  • bip.bipKeystore
  • bip.bipTruststore
  • bip.bipPassword

Manual Steps

In summary the manual steps to make the certificates available to the Java code are

  • Convert certificates to PKCS #12 files
  • Convert PKCS #12 files to Base 64 files
  • Store content of the Base 64 files in Kubernetes secrets
  • Restart the pod to make secret changes effective

If https handshake implementation remains in the Java code (as opposed to say moved to API Gateway) these manual steps need to be automated if certificate renewal is automated.

Java implementation and Code Walkthrough

Java uses Keystore objects to store certificate information in PKCS #12 files. VRO

  • Reads in the content of the PKCS #12 files and the password from the environment variables BIP_KEYSTORE, BIP_TRUSTSTORE, and BIP_PASSWORD through application.yml
  • Converts Base 64 content to binary content
  • Creates the keystore and trusstore objects as Keystore instances from the binary content and the password
  • Creates a custom RestTemplate bean that can be used to make the https requests

The properties in application.yml that corresponds to the environment variables are

  • keystore for BIP_KEYSTORE
  • truststore for BIP_TRUSTSTORE
  • truststore_password for BIP_PASSWORD

The RestTemplate bean that is used to make the https requests is implemented in BipApiConfig. This file also includes the generation of the Keystore objects from the application.yml properties. This RestTemplate bean is autowired in the rest of the code using the Qualifier bipCERestTemplate.

Curl Validation of Certificates

Validation of the certificates using the Java code has been problematic since the BIP APIs are not available outside of VA Firewall

  • Our GFE's typically do not have Java installed
  • Extra permissions are necessary to install and maintain Java on the GFE's
  • Additional security features in GFE's makes it difficult to make https calls from Java

curl has been an invaluable tool to test the validity of the certificates. In principle it should be possible to run curl in your GFE but running it from the app container in one of our environment was easier. We copied the certificates tls.key, tls_bip.crt, and va_all.crt to the container using cat <<EOF and copy&paste in the /tmp directory. You should be able to use kubectl cp with enough permissions. Once the files are in the container you can run

curl -X POST <https://......> --cacert va_all.crt --cert tls_bip.crt --key tls.key --verbose

To see the https handshake to verify the validity of the certificates. The request can be made to any end point of interest.

Clone this wiki locally