OpenNebula

I've reviewed the commits and as far as I can see, the current iteration of this only allows snapshots to be made with the VM powered off?

I think it would be beneficial to include the ability to snapshot without powering off the VM. I am well aware of the implications of snapshotting a disk live outside the OS when there is no way to know if the data is flushed to disk, however there are a few good reasons why this is still something that is good to have:

• A live disk snapshot in an unknown state is better than no snapshot at all, specifically for cases where a VM cannot be powered off due to customer requirements
• In most cases a live snapshot of a disk for a VM using a journalled filesystem without a guest agent running is recoverable without issues (provided that there is no write back caching going on in qemu, which may not apply to CEPH but does apply to other potential storage systems that support snapshots)
• The fsfreeze feature in the guest agent can be leveraged to trigger a freeze in the guest to ensure sync status, then take a snapshot outside of the VM, and finally thaw it after the snapshot action has been taken (see http://wiki.qemu.org/Features/QAPI/GuestAgent for details)

I'm not aware of guest agent support, but that can be implemented as a new feature at a later date - live snapshotting does not explicitly require it and documentation can reflect the risks, with a safe default of not allowing it if so desired.

Gerben Meijer wrote:

I've reviewed the commits and as far as I can see, the current iteration of this only allows snapshots to be made with the VM powered off?

I think it would be beneficial to include the ability to snapshot without powering off the VM. I am well aware of the implications of snapshotting a disk live outside the OS when there is no way to know if the data is flushed to disk, however there are a few good reasons why this is still something that is good to have:

• A live disk snapshot in an unknown state is better than no snapshot at all, specifically for cases where a VM cannot be powered off due to customer requirements
• In most cases a live snapshot of a disk for a VM using a journalled filesystem without a guest agent running is recoverable without issues (provided that there is no write back caching going on in qemu, which may not apply to CEPH but does apply to other potential storage systems that support snapshots)
• The fsfreeze feature in the guest agent can be leveraged to trigger a freeze in the guest to ensure sync status, then take a snapshot outside of the VM, and finally thaw it after the snapshot action has been taken (see http://wiki.qemu.org/Features/QAPI/GuestAgent for details)

I'm not aware of guest agent support, but that can be implemented as a new feature at a later date - live snapshotting does not explicitly require it and documentation can reflect the risks, with a safe default of not allowing it if so desired.

Hi Gerben Meijer

Yes you are right, this is the initial code to test things, you know ;). Once most of the machinery is in place, we'll add the RUNNING state as possible and test things. For example, we want to stress libvirt when reverting to a snaphost (the original block or file will be replaced by another one and that may not play nicely with qemu/libvirt)...

THANKS FOR YOUR FEEDBACK :)

Needed modificactions:

Compatibility
- Clone cannot be perform on images with snapshots
- Image with snapshots can be only of OS and DATA type (chtype enforce that)
- Image with snapshots can be only persistent (persistent api enforce that)

Upgrade
- 3 new states for disk-snapshot operations
- New field <TARGET_SNAPSHOT> that marks active snapshot operation (smilar to CLONING_ID)
- New driver operations

- TM: snap_create, snap_delete, snap_revert + update of mvds and rm
- DS: snap_delete, snap_revert, snap_flatten + update of delete

- Migrator to add IMAGE element

API
- disk and image operations. OCA JAVA missing

Sunstone
- Refactor of VM disk tab
- Add snapshot tab for Images

Updates:

• level JAVA API with the new one.vm.disksaveas, this replaces the old disk_snapshot api call, and onevm.saveas method
• Check that JSON classes used by Sunstone are properly updated
• Sunstone needs to re-work the snapshot/saveas interface
• Datamodel, removed DISK fields SAVE_AS_SOURCE, SAVE_AS and SAVE_AS_ACTIVE are not in use anymore, code should not check against those.
• Check EC2 API's

• Applications using disk_snapshot API call needs to upgrade to the new functionality
• New API call, with optional snap_id parameter
• Deferred snapshots are superseded by the new snapshoting functionality
• CPDS has a new argument, positional parsing of arguments by custom scripts needs to be updated
• Removed cancelsaveas, not needed anymore

• VMs with snapshot deferred will not perform the saveas operation, needs to be manually triggered, and placeholder images maybe left in locked state (need to remove them) *

Evaluate migrators for VMs with SAVE_AS=YES (deferred)

libvirt/qemu currently does not allow reopening the disk to account for a disk change, which is what's happening in the snapshot-create operation. For the moment we will allow snapshot-create to be triggered in POWEROFF and SUSPENDED states. This will change once libvirt exposes an interface to make this change.

@Jaime Meles libvirt/qemu currently does not allow reopening the disk to account for a disk change, which is what's happening in the snapshot-create operation. For the moment we will allow snapshot-create to be triggered in POWEROFF and SUSPENDED states. This will change once libvirt exposes an interface to make this change.

^^ when snapshots are created using "virsh snapshot-create-as" there is no need to notify libvirt/qemu of a disk change, as this is done automatically (intrinsic action of snapshot-create-as operation").

virsh help snapshot-create-as
NAME
snapshot-create-as - Create a snapshot from a set of args

SYNOPSIS
    snapshot-create-as <domain> [--name <string>] [--description <string>] [--print-xml] [--no-metadata] [--halt] [--disk-only] [--reuse-external] [--quiesce] [--atomic] [--live] [--memspec <string>] [[--diskspec] <string>]..

According to this mail thread: https://www.redhat.com/archives/libvirt-users/2013-September/msg00146.html, this feature request: http://tracker.ceph.com/issues/2773 and the above output it should be possible to create a "--disk-only" snapshot on a live running VM. I have tested this on a qcow2 backed VM and this works. Unfortunately I do not (yet) have a CEPH cluster to test against but it's worth a try.

[--diskspec] <string> disk attributes: disk[,snapshot=type][,driver=type][,file=name].

^^ CEPH specific parameters shoud be filled in.

Hi,

Disk snapshot on a live running VM with Ceph backend does not seems to trigger the guest-fsfreeze-freeze/thaw commands on the guest. Is it supported in one-4.14.2?

Followed this procedure : http://www.sebastien-han.fr/blog/2015/02/09/openstack-perform-consistent-snapshots-with-qemu-guest-agent/

Works well if I trigger the command manually from the host (/tmp/freeze is filled with I'm frozen/I'm thaw). But nothing happen if I snapshot the disk manually from Sunstone.

Hi,

It's been called here

https://github.com/OpenNebula/one/blob/release-4.14.2/src/tm_mad/ceph/snap_create_live#L95

Could it be some problems with libvirtd/qemu version?

Guillaume Michaud wrote:

Hi,

Disk snapshot on a live running VM with Ceph backend does not seems to trigger the guest-fsfreeze-freeze/thaw commands on the guest. Is it supported in one-4.14.2?

Followed this procedure : http://www.sebastien-han.fr/blog/2015/02/09/openstack-perform-consistent-snapshots-with-qemu-guest-agent/

Works well if I trigger the command manually from the host (/tmp/freeze is filled with I'm frozen/I'm thaw). But nothing happen if I snapshot the disk manually from Sunstone.

Hi Ruben,

As I understand, the virsh command is called from the frontend but it is not installed my CentOS 7 frontend. I have to install the libvirt package to get it.
I installed my frontend using the official ONE repo and "yum install opennebula-server opennebula-sunstone" command.

Thanks.

Ruben S. Montero wrote:

Hi,

It's been called here

https://github.com/OpenNebula/one/blob/release-4.14.2/src/tm_mad/ceph/snap_create_live#L95

Could it be some problems with libvirtd/qemu version?

Guillaume Michaud wrote:

Hi,

Disk snapshot on a live running VM with Ceph backend does not seems to trigger the guest-fsfreeze-freeze/thaw commands on the guest. Is it supported in one-4.14.2?

Followed this procedure : http://www.sebastien-han.fr/blog/2015/02/09/openstack-perform-consistent-snapshots-with-qemu-guest-agent/

Works well if I trigger the command manually from the host (/tmp/freeze is filled with I'm frozen/I'm thaw). But nothing happen if I snapshot the disk manually from Sunstone.

Installing libvirt-client on the frontend does not make a difference. So I understand that the virsh command is called directly on the host.

Executing "virsh -c qemu:///system domfsfreeze one-546" on the host trigger the fshook correctly in the guest.

Just want to be sure, I'm talking about disk-snapshoting and not VM-snapshoting. On the Storage tab, I'm using the snapshot option for the vda disk (ID-0).

Thanks.