Bug #1016

Performance issues when having lot of VMs

Added by Maxence Dunnewind over 9 years ago. Updated over 9 years ago.

Status:ClosedStart date:12/01/2011
Priority:NormalDue date:
Assignee:Tino Vázquez% Done:

100%

Category:Core & System
Target version:Release 3.4 - S0
Resolution:fixed Pull request:
Affected Versions:OpenNebula 3.0

Description

I try to create lot of VMs in a few minutes, and triggered an issue.

- ONE 3.0 with mysql

with a few (~5) vms in the pool, time to exec requests ('onevm list a') is ok
with more vms (~30), time to exec the same request is ~4s
with again more vm (~80), time is ~ 20 vms

I added some logs to OCCI/xmlrpc to find where it takes so much time :

"post_compute before VirtualMachineOCCI call : Thu Dec 01 11:55:34 +0100 2011" 
"post_compute after VirtualMachineOCCI call : Thu Dec 01 11:55:34 +0100 2011" 
"post_compute after to_one_template call : Thu Dec 01 11:55:34 +0100 2011" 
"post_compute before vm.allocate call : Thu Dec 01 11:55:34 +0100 2011" 
"Before rpc call to vm.allocate : Thu Dec 01 11:55:34 +0100 2011" 
"xmlrpc : call2 : before methodCall : Thu Dec 01 11:55:34 +0100 2011" 
"xmlrpc : call2 : after methodCall : Thu Dec 01 11:55:34 +0100 2011" 
"xmlrpc : call2 : after do_rpc: Thu Dec 01 11:55:42 +0100 2011" 
"xmlrpc : call2 : after parseMethodResponse: Thu Dec 01 11:55:42 +0100 2011" 
"after rpc call to vm.allocate : Thu Dec 01 11:55:42 +0100 2011" 
"post_compute after vm.allocate call : Thu Dec 01 11:55:42 +0100 2011"

The time is used in do_rpc, so inside the RPC.

I started oned -f with "valgrind --tool=callgrind", I attach the dump file.

History

#1 Updated by Maxence Dunnewind over 9 years ago

Ok,

after some more debug/analyze, it seems to be related to the quota driver. When we run the test using an user with only a few vms, it is fast. When we use an user with lot (50+) vms, it takes ages. Removing --authz quota, it is fast in all cases.

#2 Updated by Daniel Molina over 9 years ago

  • Status changed from New to Closed
  • Target version set to Release 3.4
  • % Done changed from 0 to 100
  • Resolution set to fixed

Hi,

We have included a new Nokogiri parser for XMLRPC (bug #1017) that improves the overall performance of the clients. Also we are moving the quota authorization to the core (feature #1099) to improve considerably the performance.

#3 Updated by Ruben S. Montero over 9 years ago

  • Target version changed from Release 3.4 to Release 3.4 - S0

Also available in: Atom PDF