Backlog #4271
advanced Deploy of a VM to different SYSTEM_DS after Undeploy
| Status: | Pending | Start date: | 12/30/2015 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assignee: | - | % Done: | 0% | |
| Category: | Core & System | |||
| Target version: | - |
Description
Hello developers,
While testing our addon against SYSTEM_DS migrations I've spot this issue.
Tested via the sunstone interface and the OpenNebula version is 4.14.2.
I've made a quick check and can't find this issue reported. I am almost sure this issue is a bug not a feature :)
How to reproduce:
1. Define two SYSTEM_DS instances with same TM_MAD. My test is with ssh but I am confident that it is same for all TM_MAD.
2. Instantiate and run a VM. Check on which datastore is defined (in my case `system-ds-1`)
3. Undeploy the VM
4. Deploy the VM with selected `system-ds-2` in the "Advanced Options"
Result:
The VM is deployed on `system-ds-1`.
Expected result:
A VM deployed on the selected `systemd-ds-2`.
The cold migrate of a VM from one to another SYSTEM_DS is working.
I am not sure is it Core, Driver(Storage), or Synstone related so I am leaving empty Category field.
Kind Regards,
Anton Todorov
History
#1
Updated by Anton Todorov over 5 years ago
Quick update
Just tested to deploy with the shell tool onevm deploy <VM_ID> <HOST_ID> <DS_ID> ... same issue.
Kind Regards,
Anton Todorov
#2
Updated by Ruben S. Montero over 5 years ago
- Tracker changed from Bug to Backlog
- Category set to Core & System
Hi Anton,
this is a worksforme, in fact the core is imposing this restriction:
if (vm->hasHistory() &&
(vm->get_action() == History::STOP_ACTION ||
vm->get_action() == History::UNDEPLOY_ACTION ||
vm->get_action() == History::UNDEPLOY_HARD_ACTION))
{
ds_id = vm->get_ds_id();
}
We need to execute the migration steps, PROLOG/EPILOG_MIGRATE to move files and checkpoints across system datastores. Probably, we could introduce the logic to do so....
I am moving this the backlog tracker.
THANSK!
#3
Updated by Anton Todorov over 5 years ago
Hi Ruben,
Here is a brief description of my dev setup.
3 node cluster, frontedn on first one (s04):[root@s04 ~]# onehost list
ID NAME CLUSTER RVM ALLOCATED_CPU ALLOCATED_MEM STAT
0 s04 cluster0 0 0 / 1200 (0%) 0K / 31.2G (0%) on
1 s05 cluster0 0 0 / 1200 (0%) 0K / 31.2G (0%) on
2 s06 cluster0 1 100 / 1200 (8%) 1024M / 31.2G (3%) on
At the moment there are two system datastores attached to cluster0: system-ssh-1 and system-ssh-2. [root@s04 ~]# onedatastore list
ID NAME SIZE AVAIL CLUSTER IMAGES TYPE DS TM STAT
0 system 100G 91% - 0 sys - shared on
1 default 100G 91% - 0 img fs shared on
2 files 100G 91% cluster0 4 fil fs ssh on
100 storpool-r3-h 1.1T 98% cluster0 7 img storpoo storpoo on
101 system-ssh-1 121.1G 75% cluster0 0 sys - ssh on
105 system2 100G 91% - 0 sys - shared on
106 system-ssh-2 0M - cluster0 0 sys - ssh on
In the cluster0 I have DATASTORE_LOCATION=/var/lib/one-local (/var/lib/one is on s04 and shared over nfs to s05 and s06 but not used in this case)
The VM with VM_ID=115 is running on HOST_ID=2 (s06) VM home on system-ssh-1 (DS_ID=101), root disk on storpool (DS_ID=100)
I've added logging to syslog at the beginning of tm/ssh/mv(designated as `tm_mv`) and tm/storpool/mv(designated as `tm_sp_mv`)
First I do migrate to host 1 (s05) with SYSTEM_DS change from 101 to 106.[root@s04 ~]# onevm migrate 115 1 106
Dec 30 21:30:47 s04 tm_sp_mv: s06:/var/lib/one-local/101/115/disk.0 s05:/var/lib/one-local/106/115/disk.0 115 100
Dec 30 21:30:49 s04 tm_mv: s06:/var/lib/one-local/101/115 s05:/var/lib/one-local/106/115 115 106
^- It is clear tat the migration from 101 to 106 is done
Then I do simple migrate to move the VM back to host s06:[root@s04 ~]# onevm migrate 115 2
Dec 30 21:32:15 s04 tm_sp_mv: s05:/var/lib/one-local/106/115/disk.0 s06:/var/lib/one-local/106/115/disk.0 115 100
Dec 30 21:32:17 s04 tm_mv: s05:/var/lib/one-local/106/115 s06:/var/lib/one-local/106/115 115 106
Then lets undeploy the VM:[root@s04 ~]# onevm undeploy 115
Dec 30 21:33:52 s04 tm_sp_mv: s06:/var/lib/one-local/106/115/disk.0 s04:/var/lib/one-local/106/115/disk.0 115 100
Dec 30 21:33:54 s04 tm_mv: s06:/var/lib/one-local/106/115 s04:/var/lib/one-local/106/115 115 106
^- The VM is (as I name it)"parked" on the front-end (s04). Note that the VM is still on DS_ID=106
Deploy the VM to system_ds 101:[root@s04 ~]# onevm deploy 115 2 101
Dec 30 21:35:23 s04 tm_sp_mv: s04:/var/lib/one-local/106/115/disk.0 s06:/var/lib/one-local/106/115/disk.0 115 100
Dec 30 21:35:24 s04 tm_mv: s04:/var/lib/one-local/106/115 s06:/var/lib/one-local/106/115 115 106
^- Here is the issue that is baffling me! Note that the system_ds TM_MAD is called with DST and DS_ID arguments containing DS_ID=106 - not 101 as ordered?!
I've made one last migration from host 2 to host 1, and then here is the vm info. Note that in the history is logged migrate to 106 not 101 as requested with the deploy command:
[root@s04 ~]# onevm migrate 115 1
Dec 30 22:23:58 s04 tm_sp_mv: s06:/var/lib/one-local/106/115/disk.0 s05:/var/lib/one-local/106/115/disk.0 115 100
Dec 30 22:24:00 s04 tm_mv: s06:/var/lib/one-local/106/115 s05:/var/lib/one-local/106/115 115 106
[root@s04 ~]#
[root@s04 ~]# onevm show 115
VIRTUAL MACHINE 115 INFORMATION
ID : 115
NAME : CentOS 7.1 - KVM-115
USER : oneadmin
GROUP : oneadmin
STATE : ACTIVE
LCM_STATE : RUNNING
RESCHED : No
HOST : s05
CLUSTER ID : 100
CLUSTER : cluster0
START TIME : 12/30 10:21:11
END TIME : -
DEPLOY ID : one-115
VIRTUAL MACHINE MONITORING
CPU : 0.0
MEMORY : 0K
NETTX : 12K
NETRX : 237K
PERMISSIONS
OWNER : um-
GROUP : ---
OTHER : ---
VM DISKS
ID DATASTORE TARGET IMAGE SIZE TYPE SAVE
0 storpool-r sda CentOS-7 1.6G/10G bloc NO
1 - hda CONTEXT 0M/- - -
VM DISK SNAPSHOTS
AC ID DISK PARENT DATE SIZE NAME
0 0 -1 12/30 11:36:36 -/10G 1
=> 1 0 0 12/30 17:01:22 41M/10G 2
VM NICS
ID NETWORK VLAN BRIDGE IP MAC
0 vnet-10.2.8.0 no br0 10.2.8.107 02:00:0a:02:08:6b
SECURITY
NIC_ID NETWORK SECURITY_GROUPS
0 vnet-10.2.8.0 0
SECURITY GROUP TYPE PROTOCOL NETWORK RANGE
ID NAME VNET START SIZE
0 default OUTBOUND ALL
0 default INBOUND ALL
VIRTUAL MACHINE HISTORY
SEQ HOST ACTION DS START TIME PROLOG
0 s06 snap-create 101 12/30 10:21:28 0d 01h15m 0h00m06s
1 s06 migrate 101 12/30 11:36:36 0d 01h49m 0h00m00s
2 s05 disk-attach 106 12/30 13:25:46 0d 00h01m 0h00m09s
3 s05 migrate 106 12/30 13:27:11 0d 00h30m 0h00m00s
4 s06 delete-recreate 106 12/30 13:57:45 0d 00h01m 0h00m10s
5 s06 migrate 101 12/30 13:59:59 0d 00h00m 0h00m13s
6 s05 delete-recreate 101 12/30 14:00:47 0d 00h02m 0h00m10s
7 s06 migrate 101 12/30 14:03:29 0d 00h01m 0h00m09s
8 s05 delete-recreate 101 12/30 14:04:42 0d 00h02m 0h00m11s
9 s06 migrate 101 12/30 14:07:29 0d 00h00m 0h00m08s
10 s05 migrate 101 12/30 14:07:58 0d 00h02m 0h00m12s
11 s06 migrate 106 12/30 14:10:11 0d 00h08m 0h00m11s
12 s05 delete-recreate 106 12/30 14:18:35 0d 00h01m 0h00m11s
13 s06 migrate 101 12/30 14:20:29 0d 00h04m 0h00m08s
14 s05 delete-recreate 101 12/30 14:25:16 0d 00h01m 0h01m36s
15 s06 live-migrate 101 12/30 14:27:29 0d 00h04m 0h00m08s
16 s05 migrate 101 12/30 14:32:07 0d 00h00m 0h00m00s
17 s05 migrate 106 12/30 14:32:44 0d 00h20m 0h00m07s
18 s06 migrate 106 12/30 14:52:44 0d 00h01m 0h00m11s
19 s05 migrate 106 12/30 14:53:46 0d 00h02m 0h00m11s
20 s06 migrate 106 12/30 14:56:09 0d 00h02m 0h00m12s
21 s05 migrate 106 12/30 14:58:44 0d 00h01m 0h00m11s
22 s06 delete-recreate 106 12/30 15:00:16 0d 00h01m 0h00m11s
23 s06 live-migrate 101 12/30 15:02:29 0d 00h19m 0h00m09s
24 s05 live-migrate 101 12/30 15:21:27 0d 00h04m 0h00m00s
25 s06 migrate 101 12/30 15:26:17 0d 00h03m 0h00m00s
26 s05 migrate 101 12/30 15:29:30 0d 00h00m 0h00m11s
27 s06 migrate 106 12/30 15:30:19 0d 00h01m 0h00m11s
28 s05 undeploy 101 12/30 15:31:22 0d 00h01m 0h00m14s
29 s06 undeploy 101 12/30 15:33:16 0d 00h01m 0h00m07s
30 s06 migrate 101 12/30 15:35:31 0d 00h02m 0h00m06s
31 s05 undeploy 106 12/30 15:37:28 0d 00h00m 0h00m10s
32 s06 disk-attach 106 12/30 15:38:59 0d 00h01m 0h00m08s
33 s06 disk-attach 106 12/30 15:40:13 0d 00h00m 0h00m00s
34 s06 undeploy 106 12/30 15:40:52 0d 00h01m 0h00m00s
35 s05 undeploy 106 12/30 15:42:50 0d 00h07m 0h00m10s
36 s05 undeploy 106 12/30 16:15:20 0d 00h01m 0h00m10s
37 s06 migrate 106 12/30 16:17:35 0d 00h01m 0h00m10s
38 s05 migrate 101 12/30 16:18:42 0d 00h00m 0h00m16s
39 s05 undeploy 106 12/30 16:19:35 0d 00h01m 0h00m12s
40 s06 migrate 106 12/30 16:21:59 0d 00h27m 0h00m13s
41 s06 migrate 101 12/30 16:49:14 0d 00h01m 0h00m12s
42 s06 migrate 106 12/30 16:51:01 0d 00h05m 0h00m12s
43 s06 snap-create 101 12/30 16:56:00 0d 00h05m 0h00m13s
44 s06 disk-detach 101 12/30 17:01:23 0d 00h32m 0h00m00s
45 s06 disk-attach 101 12/30 17:34:20 0d 00h00m 0h00m00s
46 s06 disk-attach 101 12/30 17:35:01 0d 00h01m 0h00m00s
47 s06 disk-detach 101 12/30 17:36:31 0d 00h01m 0h00m00s
48 s06 disk-attach 101 12/30 17:38:19 0d 00h00m 0h00m00s
49 s06 disk-detach 101 12/30 17:38:52 0d 00h00m 0h00m00s
50 s06 disk-attach 101 12/30 17:39:46 0d 00h01m 0h00m00s
51 s06 undeploy 101 12/30 17:40:58 0d 03h33m 0h00m00s
52 s06 disk-detach 101 12/30 21:15:01 0d 00h00m 0h00m14s
53 s06 disk-detach 101 12/30 21:15:45 0d 00h00m 0h00m00s
54 s06 snap-create 101 12/30 21:15:56 0d 00h00m 0h00m00s
55 s06 disk-detach 101 12/30 21:16:07 0d 00h00m 0h00m00s
56 s06 disk-detach 101 12/30 21:16:42 0d 00h00m 0h00m00s
57 s06 undeploy 101 12/30 21:16:51 0d 00h05m 0h00m00s
58 s06 undeploy 101 12/30 21:25:12 0d 00h01m 0h00m04s
59 s06 migrate 101 12/30 21:28:04 0d 00h02m 0h00m05s
60 s05 migrate 106 12/30 21:30:41 0d 00h01m 0h00m08s
61 s06 undeploy 106 12/30 21:32:10 0d 00h01m 0h00m08s
62 s06 migrate 106 12/30 21:35:22 0d 00h48m 0h00m05s
63 s05 none 106 12/30 22:23:52 0d 00h00m 0h00m11s
USER TEMPLATE
ERROR="Wed Dec 30 14:18:52 2015 : Error restoring VM: Could not restore from /var/lib/one-local/106/115/checkpoint"
FROM_APP="53e7bf928fb81d6a69000002"
FROM_APP_NAME="CentOS 7.1 - KVM"
LOGO="images/logos/centos.png"
ROOT_PASSWORD="storpool"
USER_INPUTS=[
ROOT_PASSWORD="M|password|root Password" ]
VIRTUAL MACHINE TEMPLATE
AUTOMATIC_REQUIREMENTS="CLUSTER_ID = 100 & !(PUBLIC_CLOUD = YES)"
CONTEXT=[
DISK_ID="1",
ETH0_DNS="10.2.8.254",
ETH0_GATEWAY="10.2.8.254",
ETH0_IP="10.2.8.107",
ETH0_MAC="02:00:0a:02:08:6b",
ETH0_MASK="255.255.255.0",
ETH0_NETWORK="10.2.8.0",
FILES_DS="/var/lib/one//datastores/2/6eb619eec466b5c4e0a66dd01582b42f:'set-root-password.sh' ",
INIT_SCRIPTS="set-root-password.sh",
NETWORK="YES",
ROOT_PASSWORD="justpass",
SSH_PUBLIC_KEY="ssh-rsa AAAAB3NzaC1yc2EAAAABIwAABAEApj8xCnHNtJBAk0ojfn3Sg++tAQ4H7M3FkqaBQU8v9myanfz1/8yv8dZYOofqS21FAPsDBge/HT+gG2HKr0uvKpWCg6JRgKNln3YKbUKuu2TqM30K+rH6rlYxKa8rTc6BC/RHlmQgXL6HfblaRYNUxsZ2p/vcDH9/aMY0/PwmVWJK0Ew8mWQ7dSoJGwQDfuXdmEzdFmbJLLyZDWi5TTXK/9cUVF9vS9Ts8IbMMkNlf4evYKvzSCGe1IUev8VCsfa/ayeZG/27vFSZd0VrBqsqSUhuM64pxVLkyXtDH3JKqwBaDMMNntISs8FUKFzAbdVCIxOeSMsDz3SVpvMpZ3aJdG4PUapg0hVaeMj65x6zcrddNZUbEB+c1/IFTqBhSZKAaAmUQGzW7nbcPOItrSxkOyRi1hYj4okVsMWO5lCBJkadUiLybWRohrvp1aKws2Oy/wfHm9Jql6oIy15RuRXLXSoaUDDrSDL7xD8TMM+UQhon9XUuBlZ5Bjpbcbbn3UNwdSdZaZw1nbfrhP0M/lRCivbR81ETLaNeqDe1+B06mCy5ddHqRwjgfwrwrCJOGwpMpYnUxuoTZyIoUGlIgWK5C7bk8el+J5fx+nAw7OBvU8Jy+5Y5vSOPZcerQ+YrA3mg/Hho7U7YcVjDPKBL++++5cuBR+/nzy1wRQwm0nv/psPN8SMUecPFotd7hwzfsQUUs5sYuZnkoxVS3Qm6j+IeYB84q9HgrD1BsbpHKnKEra/RSTxG5nyIMwrnOb0LUJJwcNU1ZeEOZJl3v0IwsMol1KXbKCZ529dw1qNBrdyFkK/xQl3BKl//JLHKM3TFkE5V6hzo84vzJUA7H98+bceBb/OocqHLPxLWFOHm97RHYIqxPZGJw1SIUr/qCxkZ1IwBNV17Zb7Q648ZBsj2SDnZxMAP6HSdaYShAYE8Sb8rrbM9Z9lpa+O184qrey0sbZtSbdptIIzEwa0ZbAtESbdsSr4T2z/G/wFoRTHu6No9vecuFNyedtoC7MypwpkFV7a3mOCHOl5zJNXVRkfv9hZDsaNdxZwtb2rKb/ubDX9zqLLw6Bg+rdYrKpzzSwfCNl6zlQ+If/UczOWBHVN1pZpZK5CAyCRgVwZxjZJ63Vx7ZPXpHwFvvkgYKgLVpiHvhxXk+wKvFrX9NDDhmCCbE8Re2okvyq3QckHxBBFS1QpH7bwlbQ4wjV0b6aBobAyf7uX7cScMq98byoP/eICEhVRUE9Ut2RVb8IQ1Dz0jL9EhS5ikCLAIkKFdIDaqQz5KBd4Kf3+dLaATEo+T7KlUtrhM9D6LoIe4sFTeyw/pZ55NzkUMRjVm9bgn5ub2/vnKLFenm1B9liqoD+u1Ritj73+Phw== at_key
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCSiM61/M9bqLTqdg2HxsVMCoudy4TOWUnzP/imNeV1trU3bQikgi1TgQTOTEH4LqimGwW6C0GO5gI8u8zJmog3FTFH88YG16B2cawXxxw/AHCQClHMXPg9WLX53rEB16bHw/MExwdYMJ693BWjoCBT7wY9ofAXm0Jk66eNPfyyqxnL6Kg7QxXfeDhDVLSQ1XyXxm5BnRbzBnqmchVbkF6sKNjQt8s1GEMn1JBlrOAnNCEKKXdvB8ld4+jCQDTv8WomQ3sUrxD9OqaWcak/tCM4DvWT3+krrRm9B0OONvWIhbWNJdkiS9G7RaAiY8yfORPvJwK4McjuBWNceDA14qyF mgmt
",
TARGET="hda" ]
CPU="1"
GRAPHICS=[
LISTEN="0.0.0.0",
PORT="6015",
TYPE="vnc" ]
MEMORY="1024"
OS=[
ARCH="x86_64" ]
TEMPLATE_ID="5"
VCPU="2"
VMID="115"
Next Year I am planning to reinstall the nodes to CentOS 7.2 and do fresh install of OpenNebula and will check again.
Now it is time for happy celebrations of the new year!
Cheers,
Anton
#4
Updated by Ruben S. Montero over 5 years ago
Just a quick note, I confirm that this is not working (deploy with a different system ds from undeploy/undeploy hard/stop) as expected. I just was trying to spell out that the behavior is hardcoded in oned. Your observations are right, no need to reproduce the tests
#5
Updated by Ruben S. Montero over 5 years ago
An now! HAPPY NEW YEAR!