shared LVM driver
|Category:||Drivers - Storage|
We have developed at MTA SZTAKI Internet Technology Dept. (ITAK) a new datastore and transfer manager driver. It uses a shared block device, which could be iSCSI (also should work with certain AoE, Fibre Channel setups) etc but it must be accessible for rw on all nodes and frontends. There is a volume group on the block device, and the VM images are logical volumes, similar to the iSCSI driver, but it doesn't depend on tgtadm, it works with out-of-the-box iSCSI (or AoE) storages as well.
If you find it useful, feel free to use the code...
- If there is more than one frontend, the administrator should do something to avoid concurrent LVM metadata writes (e.g CLVM, lockfile on a shared space, etc)
- It doesn't support live migration, except if you use CLVM (and shared system driver of course), because the LV-s used by the VM arent active on the target node (with CLVM this is not an issue) - a hook before MIGRATE would be helpful in this situation (we are planning to send a patch to create MIGRATE hook).
- The administrator needs to make sure that the block device is available on all nodes and frontends, and make sure that the volume group shows up everywhere with same name as in shared_lvm.conf
- After you apply the attached patch, you should run the attached permissions script as well. It applies the necessary file permissions on the new driver scripts.
#4 Updated by SZTAKI ITAK about 8 years ago
I recently rewrote the LVM transfer manager according to this thread: http://lists.opennebula.org/pipermail/users-opennebula.org/2013-March/022305.html
You can download it from here: https://dl.dropboxusercontent.com/u/140123/lvm.tar.gz
- In case of Ubuntu >=12.10, the lvchange
an command doesn't work ( https://bugs.launchpad.net/ubuntu/+source/lvm2/+bug/1088081 ). A possible solution is changing the /lib/udev/rules.d/85-lvm2.rules file, and removing the "/sbin/lvm vgchange -a y" command. Because of disk caching, I need to execute a "sync;echo 3 > /proc/sys/vm/drop_caches" in the tm scripts, forcing the nodes to re-read the content of the partition. If anyone knows a better solution to this, please let me know.
- Live migration isn't working without CLVM, but here is a workaround (pre/postmigrate scripts for the shared system ds driver, and for the lvm tm driver): https://dl.dropboxusercontent.com/u/140123/livemigrate.tar.gz
- Although I tested the code, it might contain bugs, so it is not recommended for production.