Bug #124

Scalability problems with EC2 driver

Added by Johan Tordsson about 12 years ago. Updated almost 10 years ago.

Status:ClosedStart date:07/08/2009
Priority:NormalDue date:
Assignee:Ruben S. Montero% Done:

0%

Category:Drivers - Auth
Target version:Release 1.4
Resolution:fixed Pull request:
Affected Versions:

Description

When running "a few" (problem encountered for 10 to 15) VMs, the EC2 drivers hangs (uses all CPU/memory) during polling and fails to
produce updated information. As the EC2 API tools are Java-based, one Java VM is started for each VM ran by ONE.
Possible solutions to this could be to tweak hardware allocation to Java VMs, or to poll all VMs in one call (ec2-describe-instances allows this).

Associated revisions

Revision 1cbd5fe0
Added by Ruben S. Montero almost 12 years ago

#124: Added EC2_JVM_ARGS and custom number of concurrent EC2 operations

git-svn-id: http://svn.opennebula.org/one/trunk@797 3034c82b-c49b-4eb3-8279-a7acafdc01c0

Revision e99352f5
Added by Anton Todorov over 4 years ago

fix tm/qcow2/ln fix clean before attach (#124)

Following this forum thread
https://forum.opennebula.org/t/opennebula-5-0-2-attach-detach-images-datastore-folder-not-cleaned-up/3111

Revision 64a1a19f
Added by Ruben S. Montero over 4 years ago

Revert "fix tm/qcow2/ln fix clean before attach (#124)"

This reverts commit e99352f5d94043198087a8be6a48421221e5cf5c.

Revision 1d613a98
Added by Ruben S. Montero over 4 years ago

Revert "Revert "fix tm/qcow2/ln fix clean before attach (#124)""

This reverts commit 64a1a19f23335e69b7158773e622ed833013d4aa.

History

#1 Updated by Ruben S. Montero about 12 years ago

  • Category set to Drivers - Auth
  • Assignee set to Tino Vázquez
  • Target version set to Release 1.4

I've limit the number of simultaneous EC2 operations to 5, so there are no than 5 java instances at the same time. This has been committed to the development branch. Needs to be tested

#2 Updated by riv nil almost 12 years ago

1. I reverted back the changes committed by Ruben
2. I tried adding: EC2_JVM_ARGS="-Xms16m -Xmx64m" in [/etc/vmm_ec2/vmm_ec2rc]

No problem beyond 15 concurrent instances..

Evidently this is not an issue related to ONE rather EC2 setups..

Ruben, kindly revert-back the changes..

Thanks,

Pallab

#3 Updated by Ruben S. Montero almost 12 years ago

Thanks for the feedback!.

I think we can do the following:
  • move the number of threads to the configuration file
  • add the EC2_JVM_ARGS to the configuration file as you propose

In this way you can limit the number of java instances as well.

#4 Updated by riv nil almost 12 years ago

Hi Ruben,

Normally ec2 users are limited to a maximum of 20 concurrent instances.
Pls refer: http://developer.amazonwebservices.com/connect/entry.jspa?externalID=2157

This is fyi only..

Thanks

Pallab

#5 Updated by Ruben S. Montero almost 12 years ago

  • Status changed from New to 3
  • Assignee changed from Tino Vázquez to Ruben S. Montero
  • Resolution set to fixed

#6 Updated by Ruben S. Montero almost 10 years ago

  • Status changed from 3 to Closed

Also available in: Atom PDF