OpenNebula

I ran into an issue testing the Ceph drivers on a newer version of Libvirt (1.0.x in particular) and they are broken if CephX authentication is enabled (as noted in the documentation). However, it should work and function appropriately with CephX authentication, without requiring the ceph.conf and ceph.keyring present on each hypervisor, but we would need the following for each Ceph Datastore configured:

CEPHUUID= A generated UUID for a LibVirt secret (to hold the CephX authentication key in Libvirt on each hypervisor). This should be generated when creating the Ceph datastore in OpenNebula

CEPHUSER= The ceph client.user (this is 'admin' by default, but when configuring a cluster users can be added/removed, so ideally this should be defined)

CEPHKEY= A defined authentication Key for the user (extractable from the Ceph cluster)

CEPHMONS= The list of Monitors for the ceph cluster (either IP or hostnames)
CEPHPORT= The default port is '6789', this could be assumed, but it IS configurable in Ceph, so might need to define

• Generate a secret.xml file for defining the secret
• Check libvirt on the deployment hypervisor and verify the secret is defined. If not, define it.
• Inject the appropriate cluster information into the deployment.x file

The Secret XML would need to look like this (with the defined items replacing below)

<secret ephemeral='no' private='no'>
<uuid>$CEPHUUID</uuid>
<usage type='ceph'>
<name>anything here really</name>
</usage>
</secret>

The Disk Section of the XML would need to look like this (with the defined items replacing the variables below):

<disk type='network' device='disk'>
<source protocol='rbd' name='$SOURCE'>
<host name='$FIRST CEPHMON ENTRY' port='$CEPHPORT'/>
<host name='$SECOND CEPHMON ENTRY' port='$CEPHPORT'/>
<host name='$THIRD CEPHMON ENTRY' port='$CEPHPORT'/>
</source>
<auth username='$CEPHUSER'>
<secret type='ceph' uuid='$CEPHUUID'
<target dev='$DEVICE' bus='$BUS'/>

These steps could potentially be done with a TM/DS MAD driver

To verify the secret exists, you could run something similar on the following on the hypervisor:

virsh secret-list | grep -c $CEPHUUID

To add the secret and inject the key, you must do the following on the hypervisor (should be done upon deployment of VM (possibly with 'clone' and 'ln' TM drivers). This should only need to be done once per Cluster/Datastore

virsh secret-define --file secret.xml
virsh secret-set-value --secret $CEPHUUID --base64 $CEPHKEY

After that it should just be a matter of injecting the appropriate information into the deployment.x file for the virtual machine.

This is NOT anything I expect to be worked on for 4.0, just want to supply some information to better integrate Ceph and future proof VM deployments with newer versions of Libvirt. Ubuntu 12.04 and CentOS 6x still use older versions of Libvirt that are not affected by the authentication changes, so for most deployments the existing implementation should work well.