Feature #32

ONE should support block devices

Added by Tino Vázquez over 10 years ago. Updated over 9 years ago.

Status:ClosedStart date:
Priority:HighDue date:
Assignee:Jaime Melis% Done:

100%

Category:Core & System
Target version:Release 1.4
Resolution:fixed Pull request:

Description

To, for example, support LVM, as asked for by Nuno Fernandes (http://lists.opennebula.org/pipermail/users-opennebula.org/2008-July/000078.html).

Another reason why management of LVMs could be far simpler that what we thought:

http://www.drbd.org/

one-diffs.tar.gz (1.15 KB) Ruben S. Montero, 07/27/2009 01:22 AM

tm_clone_lvm.sh Magnifier - Clone script for LVM by Sebastien Goasguen (2.41 KB) Ruben S. Montero, 07/27/2009 01:48 AM

tm_delete_lvm.sh Magnifier - Delete script for LVM by Sebastien Goasguen (1.95 KB) Ruben S. Montero, 07/27/2009 01:48 AM

tm_lvm_delete2.sh Magnifier - New version of delete by Sebastien (593 Bytes) Ruben S. Montero, 07/27/2009 12:59 PM

iscsi-v2.tar.gz - Transfer manager scripts supporting cow (2.28 KB) Sander Klous, 08/21/2009 03:06 PM

iscsi-v2.1.tar.gz (2.69 KB) Sander Klous, 08/25/2009 05:09 PM

one-diffs-v1.1.tar.gz (1.15 KB) Sander Klous, 08/25/2009 05:09 PM


Related issues

Duplicated by Request #154: External storage Closed 09/22/2009

Associated revisions

Revision fa21c11a
Added by Ruben S. Montero over 9 years ago

#32: This adds SIZE if defined to the MV and CLONE commands. Useful for creating LVM volumes and snapshots

git-svn-id: http://svn.opennebula.org/one/trunk@819 3034c82b-c49b-4eb3-8279-a7acafdc01c0

Revision 8d44e059
Added by Ruben S. Montero over 9 years ago

#32: Support for block devices. The target dev HAS TO BE linked to the
standard $VM_DIR/images/disk.0 and the DISK has to be defined as type="BLOCK"

git-svn-id: http://svn.opennebula.org/one/trunk@820 3034c82b-c49b-4eb3-8279-a7acafdc01c0

Revision 0b55468c
Added by Ruben S. Montero over 9 years ago

#32: Simple cloning example for LVM devices

git-svn-id: http://svn.opennebula.org/one/trunk@822 3034c82b-c49b-4eb3-8279-a7acafdc01c0

Revision 14e4a262
Added by Ruben S. Montero over 9 years ago

#32: Bug in when cheking the DISK type

git-svn-id: http://svn.opennebula.org/one/trunk@823 3034c82b-c49b-4eb3-8279-a7acafdc01c0

Revision e85cc06a
Added by Jaime Melis over 9 years ago

Fixed some bugs in the lvm tm_clone.sh script (#32)

git-svn-id: http://svn.opennebula.org/one/trunk@831 3034c82b-c49b-4eb3-8279-a7acafdc01c0

Revision 097e6fb9
Added by Jaime Melis over 9 years ago

Using pipes in lvm/tm_clone.sh so that we only need to read the source once (#32)

git-svn-id: http://svn.opennebula.org/one/trunk@832 3034c82b-c49b-4eb3-8279-a7acafdc01c0

Revision 5a891e76
Added by Jaime Melis over 9 years ago

added tm_delete and fixed get_lv_name helper function of tm_lvmrc (#32)

git-svn-id: http://svn.opennebula.org/one/trunk@837 3034c82b-c49b-4eb3-8279-a7acafdc01c0

Revision b7a29528
Added by Jaime Melis over 9 years ago

removed default VG_NAME in tm_lvmrc which was uploaded by mistake (#32)

git-svn-id: http://svn.opennebula.org/one/trunk@840 3034c82b-c49b-4eb3-8279-a7acafdc01c0

Revision 3fbfc970
Added by Jaime Melis over 9 years ago

Fixed lvm/tm_delete.sh so that it can delete multiple LVs (#32)

git-svn-id: http://svn.opennebula.org/one/trunk@852 3034c82b-c49b-4eb3-8279-a7acafdc01c0

Revision 369aea92
Added by Jaime Melis over 9 years ago

get_lv_name in tm_lvmrc now handles multiple disks (#32)

git-svn-id: http://svn.opennebula.org/one/trunk@853 3034c82b-c49b-4eb3-8279-a7acafdc01c0

Revision 0ea62ed2
Added by Jaime Melis over 9 years ago

Added lvm/tm_mv.sh and copied the rest of scripts from ssh (#32)

git-svn-id: http://svn.opennebula.org/one/trunk@860 3034c82b-c49b-4eb3-8279-a7acafdc01c0

Revision b92d70bb
Added by Jaime Melis over 9 years ago

lvm tm_mv.sh changed to support a more generic configuration (#32)

git-svn-id: http://svn.opennebula.org/one/trunk@873 3034c82b-c49b-4eb3-8279-a7acafdc01c0

History

#1 Updated by anonymous - over 10 years ago

second that, its essential missing thing in most other vmm's as well.

#2 Updated by anonymous - over 10 years ago

where is LVM ? I'm waiting

#3 Updated by Ruben S. Montero over 10 years ago

This ticket will be addressed in the next release of OpenNebula

#4 Updated by anonymous - about 10 years ago

Hi everyone,

I'm also very interested in seeing LVM/block device support in OpenNebula. I've checked through the change logs of v1.2, but I don't see any mention of it. Are there still plans to implement this?

#5 Updated by Javi Fontan about 10 years ago

Replying to [comment:5 anonymous]:

We have support for block devices in mind, in fact we think that the new transfer manager system is well suited to manage lvm partitions as they can be freely modified without touching the core. Unfortunately that feature is not in our priorities right now. I encourage anyone who wants to add this feature to give a try. We can also mirror svn into a git repository so cooperation is easier and provide help on the work.

The problem is that right now we have some milestones to meet and we have no resources for that very feature. Anyway we don't forget about your requests and we will reevaluate this functionality as soon as we can add resources to work on that.

#6 Updated by anonymous - about 10 years ago

that's great. i might have some time/resources to implement such a feature. would it be possible to mirror the repo in git?

Replying to [comment:6 jfontan]:

Replying to [comment:5 anonymous]:

We have support for block devices in mind, in fact we think that the new transfer manager system is well suited to manage lvm partitions as they can be freely modified without touching the core. Unfortunately that feature is not in our priorities right now. I encourage anyone who wants to add this feature to give a try. We can also mirror svn into a git repository so cooperation is easier and provide help on the work.

The problem is that right now we have some milestones to meet and we have no resources for that very feature. Anyway we don't forget about your requests and we will reevaluate this functionality as soon as we can add resources to work on that.

#7 Updated by Javi Fontan about 10 years ago

Replying to [comment:7 anonymous]:

that's great. i might have some time/resources to implement such a feature. would it be possible to mirror the repo in git?

Please, contact me jfontan AT fdi.ucm.es or to the mailing list so we can find the better way to do this collaboration. I think the git repo is going to be the be the best way to do the job as changes from svn are easy to mix. I will also check with my colleagues. Thank you for your interest.

#8 Updated by Ruben S. Montero over 9 years ago

  • Target version set to Release 1.4.2

After some discussion with the people at CERN we have now a clearer idea on how this feature can be implemented. In fact, they are currently using this approach, and have contributed the Transfer Manager scripts.

We will include some feature to ease the implementation:
  • Volume name generation
  • Specify the LVM size, volume group and the like in the DISK attribute

Thanks to Sebastien Goasguen for his kind contribution...

#9 Updated by Ruben S. Montero over 9 years ago

Another contribution to this issue. I personally support the approach by Sander, we can easily merge the patches with minor modifications. From the Mailing list:

I made some minimalistic changes to XenDriver.cc and TransferManager.cc to support LVM over iSCSI. Details of what I did can be found under "Implementation issues" on this page:

http://www.nikhef.nl/pub/projects/grid/gridwiki/index.php/Virtual_Machines_working_group

Basically the code checks for the first 5 characters of a path. If these characters match "/dev/" then it is assumed to be a block device and handled as "LVM over iSCSI". The changes should be backward compatible.

I attached the diffs for both src files. The tm scripts and configuration changes are available at the above mentioned web page.

The big advantage of using LVM over iSCSI is the possibility to work on locally stored copy-on-write clones from the LVM images. If this can be combined with a significant iSCSI read-cache (still need to investigate this), it would significantly reduce the overhead of image transfers between the image repository and the nodes.

Please feel free to contact me for comments or more information.
Thanks,
Sander

#10 Updated by Ruben S. Montero over 9 years ago

#12 Updated by Sander Klous over 9 years ago

I've been working a bit more on this (still in opennebula-1.2).

With the current implementation I have an iSCSI repository containing base virtual machine images. Once a virtual machine is deployed, opennebula creates a copy-on-write clone for it. This makes it possible to boot multiple virtual machines from the same base image. Once the VM is shutdown there are two options. The first one is "save=no", just throw the copy-on-write clone away. The second is "save=yes", the copy-on-write clone is saved in the image repository.

The technical details are a bit tricky. Because snapshots and clones on (c)LVM are not (yet) cluster aware. Fortunately, for this purpose they do not need to be (because the base virtual machine image doesn't change during usage). The workaround is to create a cluster aware normal partition in LVM intended for Copy-On-Write purposes. This partition can be enabled exclusively on the worker node and mapped together with the base (cluster aware) virtual machine image to a local snapshot with 'dmsetup'. When modifications have to be stored after a shutdown of the virtual machine, it is sufficient to remove the mapping and synchronize (i.e. disable) the cluster aware Copy-On-Write partition. Work can continue when the Copy-On-Write image is enabled on any worker node and the local mapping is reestablished. This last bit works when I do it by hand, but I am still trying to find a solution on how to do that from opennebula (it has to know it is booting a cow-image somehow).

Ruben S. Montero wrote:

Another contribution to this issue. I personally support the approach by Sander, we can easily merge the patches with minor modifications.

#13 Updated by Sander Klous over 9 years ago

Sander Klous wrote:

Work can continue when the Copy-On-Write image is enabled on any worker node and the local mapping is reestablished. This last bit works when I do it by hand, but I am still trying to find a solution on how to do that from opennebula (it has to know it is booting a cow-image somehow).

This is now working. In fact, COW images are recursive. So, you can modify the original image and save it. Using this image (which in fact is a COW clone with modifications of the original), will result in another COW clone, pointing to the original COW clone, pointing to the original image.

Why is this important? The use case I am trying to fulfill is the following: take a base image. Modify it to your liking and store it. Submit 100 VMs based on the modified image. All of this with minimal disk/network overhead and without access conflicts.

In the next version I will add caching with dm-cache to minimize the network overhead. Please find attached two tar-balls with minor modifications to the original patches I made in TransferManager.cc and the tm_commands.

#14 Updated by Ruben S. Montero over 9 years ago

  • Target version changed from Release 1.4.2 to Release 1.4

#15 Updated by Jaime Melis over 9 years ago

  • Status changed from New to Assigned
  • Assignee changed from Ruben S. Montero to Jaime Melis

#16 Updated by Jaime Melis over 9 years ago

  • % Done changed from 0 to 40

The core has been modified to accept block devices, tm scripts have been written to handle lvm partitions.

#17 Updated by Jaime Melis over 9 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 40 to 100
  • Resolution set to fixed

Transfer Manager LVM scripts have been tested. The LVM scenario has been documented.

Also available in: Atom PDF