Monthly Archives: September 2016

Quick Note: NovaJoin in the Overcloud

In the last post, we discussed how to — starting from quickstart — get an undercloud which was enrolled with FreeIPA and which had the novajoin service up and running.  As part of the work to get that going, we had to create some puppet modules for novajoin.  We can reuse those same puppet modules to help us deploy overcloud controllers which are also enrolled with FreeIPA and on which novajoin is installed and configured.

As before, this is a quick note, so there is stuff here that will likely change as we iterate through this.  All the configuration steps below take place on an undercloud which has been registered with FreeIPA, and on which novajoin is running.  You could, for example, use the methods in the previous section to create this undercloud.

Prepare the Overcloud Image

The overcloud image needs to be customized in a number of ways:

  1. The image needs to have a recent enough cloud-init in order to retrieve the vendor metadata to register with IPA.
  2. By default, package installs are disabled during an overcloud install.  Essentially, any package operations are replaced by a no-op.  This means that all required packages (novajoin in particular) need to be installed in the image ahead of time.

As root, run the following:

cd ~stack
source ./stackrc

cat > novajoin.repo << EOF
name=Copr repo for novajoin owned by rcritten

chmod 777 /home/stack
virt-copy-in -a overcloud-full.qcow2 novajoin.repo /etc/yum.repos.d
virt-customize -a overcloud-full.qcow2 --install
virt-customize -a overcloud-full.qcow2 --install python-novajoin
openstack overcloud image upload --update-existing
chmod 700 /home/stack

Make sure DNS is set correctly on the undercloud

As the stack user,

source ./stackrc

# set nameserver to ipa server 
ID=$(openstack subnet list -f value -c ID)
openstack subnet set ${ID} --dns-nameserver  <ipa_server_address>

Add the relevant puppet modules

We need to add the puppet modules for IPA and novajoin to the overcloud image.  In addition, a new puppet manifest needs to be added to puppet-tripleo to call the novajoin puppet modules.  We’ll pull all these changes in from current gerrit reviews.

Rather than using virt-customize to copy in the relevant files, we will use the swift artifacts mechanism ( to deploy the puppet modules.

# set up puppet artifact mechanism
git config --global ""
git config --global "Ade Lee"

git clone
export PATH="$PATH:/home/stack/tripleo-common/scripts"

mkdir puppet-modules
cd puppet-modules
git clone tripleo

cd tripleo
git fetch \
  refs/changes/88/374288/1 && git cherry-pick FETCH_HEAD
cd ..
git clone ipa
git clone novajoin
cd ~

upload-puppet-modules -d puppet-modules

Get heat-templates

A new profile needs to be added to tripleo-heat-templates for the novajoin service.  This profile then needs to be included as an optional component.

git clone
cd tripleo-heat-templates
git fetch \
  refs/changes/85/374285/2 && git checkout FETCH_HEAD
cd ..

Create Environment Files

We create two environment files – one for joining IPA and one for novajoin.

cat >  /home/stack/tripleo-heat-templates/environments/ipa-join.yaml << EOF
    ipa_hostClass: app_server
    ipa_enroll: True

cat > novajoin.yaml << EOF
    OS::TripleO::Services::Novajoin: ./tripleo-heat-templates/puppet/services/novajoin.yaml
    IpaDomain: ''
    IpaPassword: 'redhat123'
    IpaPrincipal: 'admin'
    IpaServer: '<ipa server hostname>'
    NovaPassword: 'DxpwEd4bXxtCQgPan8QDHQQMT'

Deploy the Overcloud!

openstack overcloud deploy \
  --templates ./tripleo-heat-templates \
   -e ./tripleo-heat-templates/environments/ipa-join.yaml \
   -e novajoin.yaml



Quick Note: IPA Nova Join in the Undercloud

Note: The exact steps in this post have been superseded by this update.

Rob Crittenden has been working on a new nova microservice called novajoin that would instances created by nova to be automatically registered to IPA during cloud-init. (  It works using the newly provided vendor_metadata mechanism (   There is a short description of the design for this in the at the source link. This quick note is about steps that I took to integrate novajoin into an undercloud install as started by quickstart.  In particular, the goal here was to create an undercloud node which is registered to an IPA server, installs novajoin, configures nova to use novajoin and starts the novajoin service.  The undercloud novajoin service could then be used to deploy overcloud nodes that are automatically enrolled as IPA clients.

Create the undercloud image

First, it is necessary to create a new undercloud image.  This is required because the software we need is not yet present in the undercloud image.  The exact steps to create the undercloud image can be found here: In particular, we do the following:

    1. Start with an undercloud image.  In the script, I download from the triple-O master branch.
  • Add some code to /usr/share/instack-undercloud/puppet-stack-config/puppet-stack-config.pp to get the instack puppet code to call the puppet modules in (1) and (2).
  • Add a slightly modified novajoin-install script called ipa-novajoin-install-ipa.  This is a temporary step.  Right now, the novajoin install script does all the openstack configuration, as well as the IPA config.  We want to use the openstack puppet modules to do the required openstack configuration, and have the install script do the IPA configuration steps only.  Rob will re-factor the install scripts shortly and this step will no longer be necessary.

All of the above steps will in time become unnecessary as the relevant modules and code are merged into the Occata code base. The modified undercloud image can be downloaded from here:,

Add the undercloud node to IPA

Before invoking quickstart, we need to register the undercloud node to IPA providing it with an OTP.  On the IPA server, I did:

kinit admin

ipa host-add --password=MySecret --force

Modifying Quickstart

The puppet code that we added in instack-undercloud needs parameters which are provided by hieradata in quickstart-hieradata-overrides.yaml.  We need to make a small change to quickstart to allow it to pass these parameters to the instack-undercloud puppet modules.

We add the following to roles/tripleo/undercloud/templates/quickstart-hieradata-overrides.yaml.j2

{% if undercloud_ipa_client_install is defined %}
 enable_ipa_client_install: true
 ipa_domain: '{{ipa_domain}}'
 ipa_server: '{{ipa_server}}'
 ipa_otp: '{{ipa_otp}}'
 {% endif %}

{% if undercloud_novajoin_install is defined %}
 enable_novajoin_install: true
 nova::api::vendordata_jsonfile_path: '/etc/nova/cloud-config.json'
 nova::api::vendordata_providers: ['StaticJSON', 'DynamicJSON']
 nova::api::vendordata_dynamic_targets: ['join@']
 nova::notification_topics: 'notifications'
 nova::notify_on_state_change: 'vm_state'
 novajoin::api::hostname: "undercloud.%{hiera('ipa_domain')}"
 novajoin::api::ipa_domain: "%{hiera('ipa_domain')}"
 novajoin::api::ipa_password: "%{hiera('ipa_password')}"
 novajoin::api::ipa_principal: "%{hiera('ipa_principal')}"
 novajoin::api::ipa_server: "%{hiera('ipa_server')}"
 novajoin::api::keystone_identity_uri: "%{hiera('keystone_identity_uri')}"
 novajoin::api::keystone_auth_url: "%{hiera('keystone_auth_uri')}"
 novajoin::api::keystone_auth_uri: "%{hiera('keystone_auth_uri')}"
 novajoin::api::nova_password: "%{hiera('nova::keystone::authtoken::password')}"
 novajoin::api::transport_url: "%{hiera('nova::default_transport_url')}"
 ipa_principal: '{{ipa_principal}}'
 ipa_password: '{{ipa_password}}'
 {% endif %}

Then, we can create an environment file that specifies the required parameters (config/general_config/ha_ipa.yml)

undercloud_vcpu: 4

# Create three controller nodes and one compute node.
 - name: control_0
 flavor: control
 - name: control_1
 flavor: control
 - name: control_2
 flavor: control

- name: compute_0
 flavor: compute

# We don't need introspection in a virtual environment (because we are
 # creating all the "hardware" we really know the necessary
 # information).
 step_introspect: false

# Tell tripleo about our environment.
 network_isolation: true
 extra_args: >-
 --control-scale 3 --neutron-network-type vxlan
 --neutron-tunnel-types vxlan
 test_tempest: false
 test_ping: true
 enable_pacemaker: true

#ipa settings
 ipa_domain: ''
 ipa_server: ''
 ipa_otp: 'MySecret'

#novajoin settings
 ipa_principal: 'admin'
 ipa_password: 'password123'

undercloud_novajoin_install: true
 undercloud_ipa_client_install: true

The new stuff is really the part at the bottom – from IPA settings downwards.  As you can see, you just need to pass in the IPA domain, server and OTP.

Run Quickstart

We can now run quickstart as follows:

./ --config config/general_config/ha_ipa.yml -e undercloud_image_url='' -R master --no-clone