Bug #268

VM deleting goes nirvana

Added by Marlon Nerling over 10 years ago. Updated about 10 years ago.

Status:ClosedStart date:06/24/2010
Priority:HighDue date:
Assignee:-% Done:

100%

Category:Core & System
Target version:Release 2.0
Resolution:fixed Pull request:
Affected Versions:

Description

After a onevm delete 4775 the vm status stays forever deleting. It happens 30% of the time.

A look on the oned.log file shows me the symptoms:
(SNIP)
Thu Jun 24 10:24:25 2010 [ReM][D]: VirtualMachineAction invoked
Thu Jun 24 10:24:25 2010 [DiM][D]: Finalizing VM 4755
Thu Jun 24 10:24:26 2010 [ONE][E]: SQL command was: INSERT OR REPLACE INTO host_attributes (id,name,type,value) VALUES (49,'ARCH',0,'x86_64'), error: database is locked
Thu Jun 24 10:24:27 2010 [VMM][D]: Message received: LOG - 4755 Driver command for 4755 cancelled
Thu Jun 24 10:24:28 2010 [ONE][E]: SQL command was: INSERT OR REPLACE INTO vm_pool (oid,uid,name,last_poll,template_id,state,lcm_state,stime,etime,deploy_id,memory,cpu,net_tx,net_rx) VALUES (4755,0,'T200332DE',1277367851,4755,6,0,1277364732,1277367867,'one-4755',3072000,0,0,0), error: database is locked
Thu Jun 24 10:24:29 2010 [ONE][E]: SQL command was: DELETE FROM leases WHERE oid='4' AND ip='3232235527', error: database is locked
Thu Jun 24 10:24:29 2010 [TM][D]: Message received: LOG - 4755 tm_delete.sh: SOURCE: 172.22.0.4:/var/lib/one//4755/images
Thu Jun 24 10:24:29 2010 [TM][D]: Message received: LOG - 4755 tm_delete.sh: DESTINY: 172.22.0.4:/var/lib/one//4755/images
Thu Jun 24 10:24:29 2010 [TM][D]: Message received: LOG - 4755 tm_delete.sh: Executed "ssh 172.22.0.4 mkdir -p /var/lib/one//4755".
Thu Jun 24 10:24:29 2010 [TM][D]: Message received: LOG - 4755 tm_delete.sh: Deleting 172.22.0.4:/var/lib/one//4755/images
Thu Jun 24 10:24:29 2010 [TM][D]: Message received: LOG - 4755 tm_delete.sh: Executed "ssh 172.22.0.4 ionice -c3 rm -rf /var/lib/one//4755".
Thu Jun 24 10:24:29 2010 [TM][D]: Message received: TRANSFER SUCCESS 4755 -
Thu Jun 24 10:24:30 2010 [VMM][D]: Message received: CANCEL SUCCESS 4755 -
(SNIP)

This happen not only on delete, but more ofter then.
I have now 3 of them, in only one hour, in the week it goes to 30 of them.

Associated revisions

Revision bb33da70
Added by Jaime Melis over 3 years ago

Merge pull request #268 from juanmont/feature-hollow

removed hollow attribute

History

#1 Updated by Marlon Nerling over 10 years ago

This issue is reality since the version 1.2.
But now, with our infrastructure being used ofter It starts to hurt.

#2 Updated by Marlon Nerling over 10 years ago

From interess is that one forget the VM, as if it was not his job to look for it.
So, to diminish my headaches I stop opennebula one time in the week and delete the failed leases and history, set the correct values on vm_pool by hand:
DELETE FROM leases WHERE vid IN ( SELECT oid FROM vm_pool WHERE lcm_state=15 );
UPDATE vm_pool SET state=6, lcm_state=0 WHERE lcm_state=15;
DELETE FROM history WHERE vid NOT IN ( SELECT vid FROM leases ); ## I really don't need so much history!!
DELETE FROM vm_attributes WHERE id NOT IN ( SELECT vid FROM leases ) AND vid NOT IN ( ## LIST OF MY TEMPLATES ); ## I really don't need so much history!! My templates use part of the vm_attributes.

#3 Updated by Tino Vázquez over 10 years ago

  • Target version changed from Release 1.4 to Release 2.0
  • % Done changed from 0 to 100
  • Resolution set to fixed

This issue is caused by the "oned" and "mm_sched" process trying to access the DB simultaneously.

In 1.6 there had been a redesign, so now only the "oned" process accesses the DB. Also, we have added support to "mysql" DB. Therefore, this bug is being closed.

#4 Updated by Javi Fontan about 10 years ago

  • Status changed from New to Closed

Also available in: Atom PDF