Dev Contrib: UT with AWS

Unit Tests execution requires an Openstack endpoint platform ; Tips with Amazon EC2.

Openstack platform creation

DevStack installation

1. On Amazon AWS, create a Ubuntu ~t2.large instance.
2. Install Openstack DevStack on it, using pike version (the last with keystone identity v2) ; add -b stable/pike at end of git clone command)
3. Update Security group of instance to authorize port HTTP 80 (Openstack main endpoint) and 9696 (Openstack neutron network API)
4. Optional - For working on #15 (cinder volume link) - Create a volume (~10Go) in same AWS region of instance and link them (AWS instance & volume)

Check if all is OK by retrieve a token through API using demo user (AWS URL & admin password should be updated):

curl --request POST --include --header "Content-Type: application/json" \
     --data '{"auth":{"identity":{"methods":["password"],"password":{"user":{"name":"demo","domain":{"name":"default"},"password":"secret"}}},"scope":{"project":{"name":"demo","domain":{"name":"default"}}}}}' \
     http://ec2-x.y.y.region.compute.amazonaws.com/identity/v3/auth/tokens

--> HTTP/1.1 201 Created

Public API access

By default, Openstack devstack is using eth0 ip for services URL. So this AWS private ip is in response of keystone-identity API ... and not accessible publicly. Neutron & Nova URLs service API should be updated with AWS public URL. The most simple way is toto that at startup.

Create the script sudo vi /etc/init.d/devstack-service-urls-update with:

#!/bin/sh
### BEGIN INIT INFO
# Provides:          Openstack DevStack service URLs update
# Required-Start:    $remote_fs $syslog $devstack
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:
# Short-Description: Update Openstack DevStack URLs service at boot time
# Description:       Cf upper.
### END INIT INFO

export AWS_URL=$(curl --silent http://169.254.169.254/latest/meta-data/public-hostname)
echo "Current AWS URL: $AWS_URL"
export OS_PASSWORD=$(cat /opt/stack/devstack/local.conf | grep "^ADMIN_PASSWORD=" | cut -d "=" -f 2)
export OS_IDENTITY_API_VERSION=3
export OS_AUTH_URL=http://localhost/identity/v3
export OS_DEFAULT_DOMAIN=default
export OS_USERNAME=admin
export OS_PROJECT_NAME=admin

echo "Waiting Openstack DevStack Keystone available ..."
until $(curl --output /dev/null --silent --head --fail $OS_AUTH_URL); do
    printf '.'
    sleep 5
done
echo ""

echo "Updating neutron URL"
NEUTRON_ID=`openstack endpoint list | grep neutron | cut -d "|" -f 2 | xargs`
openstack endpoint set --url "http://$AWS_URL:9696/" $NEUTRON_ID

echo "Updating nova URL"
NOVA_ID=`openstack endpoint list | grep nova | grep -v nova_legacy | cut -d "|" -f 2 | xargs`
openstack endpoint set --url "http://$AWS_URL/compute/v2.1" $NOVA_ID

exit 0

Add permissions and execution at startup:

sudo chmod 755 /etc/init.d/devstack-service-urls-update
sudo update-rc.d devstack-service-urls-update defaults

Execute the script, or reboot to verify.

Openstack platform configuration

Cinder volume

Optional - For working on #15 (cinder volume link)

You should link cinder volume group to the physical disk:

# Check that AWS volume has 'xvdf' name
lsblk

# Create physical volume
sudo pvcreate /dev/xvdf

# Link physical volume to cinder volume group
ID=$(cat /etc/cinder/cinder.conf | grep volume_group | cut -d " " -f 3)
echo $ID
sudo vgcreate $ID /dev/xvdf

# Verify cinder driver correctly linked to volume
sudo vgs

Test datas

On Openstack dasboard:

1. Create a keypair (ex: demo) to use in tests files
2. Allocate some floating ip to the project (required for some unit test and good code coverage)

These tasks could be done by script on DevStack host:

#!/bin/sh

export OS_PASSWORD=$(cat /opt/stack/devstack/local.conf | grep "^ADMIN_PASSWORD=" | cut -d "=" -f 2)
export OS_IDENTITY_API_VERSION=3
export OS_AUTH_URL=http://localhost/identity/v3
export OS_DEFAULT_DOMAIN=default
export OS_USERNAME=demo
export OS_PROJECT_NAME=demo

export KEY_PAIR_NAME=demo
echo "KeyPair check/creation..."
openstack keypair list | grep $KEY_PAIR_NAME
if [ $? -eq 1 ]; then
  echo "Creating keypair $KEY_PAIR_NAME"
  openstack keypair create $KEY_PAIR_NAME
else
  echo "Keypair $KEY_PAIR_NAME is exist, cancel creation"
fi

export FLOATING_IP_NUMBER=5
echo "Floating IP check/creation..."
while [ $(openstack floating ip list | grep -v "ID" | grep -v "\-\-\-\-\-" | wc -l) -lt $FLOATING_IP_NUMBER ]; do 
  openstack floating ip create public
done

TeamCity Openstack plugin tests files content

In TeamCity Openstack plugin source code, create tests files in directory cloud-openstack-server/src/test/resources (AWS URL & admin password should be updated).

test.v3.properties:

test.url=httphttps://ec2-x.y.y.region.compute.amazonaws.com/identity/v3
test.identity=default:demo:default:demo
test.password=secret
test.region=RegionOne

test.v3.yml

openstack-test-teamcity-plugin:
  image: cirros-0.3.5-x86_64-disk
  flavor: m1.tiny
  network: private
  security_group: default
  key_pair: demo
  auto_floating_ip: true

test.v2.properties:

test.url=https://ec2-x.y.y.region.compute.amazonaws.com/identity/v2.0
test.identity=default:demo:default:demo
test.password=secret
test.region=RegionOne

test.v2.yml

openstack-test-teamcity-plugin:
  image: cirros-0.3.5-x86_64-disk
  flavor: m1.tiny
  network: private
  security_group: default
  key_pair: demo