The default back-end plugins that ship with Barbican are not really suitable for a production deployment. The simple storage plugin simply encrypts the secrets using a single symmetric key that is stored in a plain text file, and the snake oil plugin uses self signed certificates.
Over the last 10 years, I have been part of the Dogtag Certificate System team. Dogtag is the upstream project for the Red Hat Certificate System, which has been deployed and battle-hardened in some of the largest PKI deployments in the world. We use it to generate certificates, store private keys (encryption keys), issue certificates to tokens etc. It has also passed Common Criteria, works with the major HSMs, and provides robust HA and DR solutions.
Over the last few years, we have augmented one of its components, the Key Recovery Agent (KRA) to be able to securely store more than just encryption keys. Now we can store symmetric keys (for volume encryption, for example) and passphrases. This makes the KRA an ideal back-end for Barbican.
Moreover, we have added REST interfaces and python client code to the KRA and CA components. This makes these components ideal to act as backends for Barbican, and a Barbican plugin has been written (mostly by me) to interface with the KRA.
In this posting, I’m going to describe the simple steps needed to integrate
Dogtag with Barbican, so that you can create a production-backend-backed Barbican.
Dogtag and IPA
Over the last few years, we have also integrated Dogtag into FreeIPA (that is Identity Management or IdM). IdM is part of RHEL and includes the Dogtag CA and KRA as key components used for issuing server certificates and providing the back-end store for the IPA Vault respectively.
While it is certainly possible to configure Barbican with a stand-alone Dogtag CA and KRA, there are many benefits that a full IPA instance provides to an Openstack deployment. A lot of those benefits were explored in some of the demos we created for the Tokyo summit. <links>
In particular, we are able to:
- Register each of the openstack services and service users in the
undercloud with IPA providing them with kerberos credentials.
- Use these credentials to issue SSL certs for haproxy, so that the service endpoints would be secured.
- Secure communication with the database and the message queues.
- Use a nova plugin (written by Rich Megginson) to register new VMs to IPA, providing them with an identity and kerberos credentials. This solves the problem that Kevin Fox is trying to solve in (Fox spec). Certificates for each VM (and any other resources) can be retrieved using kerberos credentials. IPA can also be used to manage the host and provide useful features like access control.
- IPA can be tied into existing identity stores like Active Directory.
The demos provide some compelling use cases for deploying Barbican
and Openstack with IPA. In this blog, I’ll describe how to configure
Barbican with IPA.
Creating an IPA instance
To use IPA, it is highly recommended that you set up IPA in a separate VM. IPA uses some of the same ports needed by Openstack, including ports 80 and 443 (which might be used by horizon), as well as ports 8080 and 8443 (used by Dogtag). A separate VM just makes things clearer and easier to debug, and keeps a crisp separation of concerns.
Using a container
One of the easiest ways to deploy IPA is through the use of a container. Once the container is created, an configuration and data is stored separately so that the container can be started/stopped or created and recreated without loss of data.
This also allows one to treat the IPA instance essentially like an appliance. Just as you would not necessarily care to know about the internal workings of your stove or fridge – you would not need to know about the internals of your IPA instance. Just turn it on and configure Barbican to work with it.
This is useful also for folks who are not as familiar with – or do not want to run Fedora or RHEL machines. They could, for instance, run IPA on RHEL/Centos/Fedora in a container, and run Barbican on Ubuntu.
- Install and start docker.
sudo yum install docker systemctl start docker
- Pull down the IPA Docker image.
docker pull adelton/freeipa-server
- Create a directory in which will contain server info (/var/lib/ipa-data for example). This directory will be used during the initial install, and will be used to save any data needed to restore the container.
Create a file: ipa-server-install-options, which will contain command line parameters to the ipa-server-install command that is run after the container is created. You want at least:
- Run the container. This will create the container and will execute ipa-server-install to install IPA and the Dogtag CA, and will start a console in which you are logged in to the server.
docker run --name freeipa-server-container -ti \ -h ipa.example.test \ -v /var/lib/ipa-data:/data:Z freeipa-server
- The current IPA containers do not yet install the KRA. This is easy to do though. While logged onto the ipa container:
Using a separate VM
IPA can easily be installed on a RHEL 7 (Centos 7) or Fedora 22/23 VM. Specify RHEL 7 or Fedora 22/23 to ensure that the right version of Dogtag is installed. You need at least Dogtag 10.2 to ensure that the Dogtag REST API is present.
The steps are:
- Install a VM with RHEL 7 (Centos 7) or Fedora 23/22.
- Install the IPA software
sudo yum install freeipa-server (Fedora) OR sudo yum install ipa-server (RHEL/Centos)
- Install IPA with the CA
- Install the KRA:
Setting up Barbican to use Dogtag (in IPA)
Setting up Barbican to use Dogtag in IPA is straightforward.
- Copy over a PEM file containing the certificate and key for the IPA
CA and KRA agent from the IPA machine. Copy this file to
/etc/barbican/kra_agent.pem and set the ownership to root:barbican. On the IPA machine, this file is located at /etc/httpd/alias/kra-agent.pem. In the docker example above, this file can be found on the host machine at /var/lib/ipa-data/etc/httpd/alias/kra-agent.pem.
If integrating with Dogtag directly, you can create this file by executing the following OpenSSL commands to extract the agent certificate and key from the agent NSS database::
openssl pkcs12 -in ~/.dogtag/pki-tomcat/ca_admin_cert.p12 \ -out ~/admin_cert.pem -nodes
Note: Ideally one should use the Dogtag admin tools to create a new agent specifically for Barbican usage. Better yet, we should use the kerberos credentials provided by IPA to interact with Dogtag and create this user. This requires some changes to Dogtag (kra acls design), which are currently in design and are slated to go out in Dogtag 10.3.
- On the Barbican machine, install the Dogtag client code::
sudo yum install pki-base
Right now, this yum operation will pull in all the dependencies for both the Python and Java Dogtag clients. There is currently work underway to separate these clients, so that only the Python client would be installed. (Dogtag separation ticket). That work is expected to be completed soon.
For Debian systems, there should be Debian packages. Alternatively, one could try the pip -installable Dogtag client package at https://pypi.python.org/pypi/dogtag-pki/10.3.0.0.1 .
Also, note that at the time I wrote this, Centos 7 had not yet updated to reflect the release of RHEL 7.2. This means that Centos 7 pulled in RHEL 7.1 bits which includes pki-base 10.1. This version does not have the required python client library calls. We expect Centos 7 to update to pull in Dogtag 10.2 shortly. In the meantime, Centos 7 builds can be obtained from the IPA copr repo.
- Modify the /etc/barbican/barbican.conf configuration file to enable the Dogtag plugins for certificate generation and secret storage:
crudini --set /etc/barbican/barbican.conf secretstore enabled_secretstore_plugins dogtag_crypto crudini --set /etc/barbican/barbican.conf dogtag_plugin dogtag_host ipa_host crudini --set /etc/barbican/barbican.conf dogtag_plugin dogtag_port 8443 crudini --set /etc/barbican/barbican.conf dogtag_plugin pem_path /etc/barbican/kra-agent.pem crudini --del --list /etc/barbican/barbican.conf certificate enabled_certificate_plugins crudini --set /etc/barbican/barbican.conf certificate enabled_certificate_plugins dogtag systemctl restart openstack-barbican-api.service
The final version of barbican.conf will contain sections that look like : [secretstore] namespace = barbican.secretstore.plugin enabled_secretstore_plugins = dogtag_crypto [dogtag_plugin] pem_path = /etc/barbican/kra-agent.pem dogtag_host = ipa_host dogtag_port = 8443 nss_db_path = '/etc/barbican/alias' nss_db_path_ca = '/etc/barbican/alias-ca' nss_password = 'password123' simple_cmc_profile = 'caOtherCert' ca_expiration_time = 1 plugin_working_dir = '/etc/barbican/dogtag' [certificate] namespace = barbican.certificate.plugin enabled_certificate_plugins = dogtag
To test out Barbican/Dogtag, I usually just end up performing the tests
described in the Testing Barbican section above (previous post). The Dogtag plugin currently does not support subordinate CAs, but that should change soon.
To make sure the certificate and secret storage and retrieval requests
actually being sent to the Dogtag CA and KRA, I usually tail the debug
logs of the CA and KRA while executing the tests.
On the IPA VM,
tail -f /var/log/pki/pki-tomcat/kra/debug tail -f /var/log/pki/pki-tomcat/ca/debug