Bug #1017

Serious performance issue with onevm list

Added by Cyril Rohr about 8 years ago. Updated about 8 years ago.

Status:ClosedStart date:12/02/2011
Priority:NormalDue date:
Assignee:Tino Vázquez% Done:

100%

Category:CLI
Target version:Release 3.4 - S0
Resolution:fixed Pull request:
Affected Versions:OpenNebula 3.2

Description

In our local OpenNebula installation, after having submitted ~150 VMs the onevm list command becomes unusable:


-bash-3.2$ time onevm list a
    ID USER     GROUP    NAME         STAT CPU     MEM        HOSTNAME        TIME
  1710 crohr    users    VM-0         runn   0  252.3M paramount-2.ren 00 01:15:22
  1711 crohr    users    VM-1         runn   0  252.3M paramount-2.ren 00 01:15:22
  1712 crohr    users    VM-2         runn   0  252.3M paramount-2.ren 00 01:15:21
  1713 crohr    users    VM-3         runn   0  252.3M paramount-10.re 00 01:15:20
  1720 crohr    users    VM-0         runn   0  252.3M paramount-25.re 00 00:26:04
  1721 crohr    users    VM-1         runn   0  252.3M paramount-25.re 00 00:26:04
  1722 crohr    users    VM-2         runn   0  252.3M paramount-25.re 00 00:26:03
  1723 crohr    users    VM-3         runn   0  252.3M paramount-27.re 00 00:26:03
  1724 crohr    users    VM-1         runn   0  252.3M paramount-25.re 00 00:24:42
  1725 crohr    users    VM-0         runn   0  252.3M paramount-25.re 00 00:24:42
  1726 crohr    users    VM-5         runn   0  252.3M paramount-25.re 00 00:24:42
  1727 crohr    users    VM-6         runn   0  252.3M paramount-27.re 00 00:24:42
  1728 crohr    users    VM-8         runn   0  252.3M paramount-27.re 00 00:24:42
  1729 crohr    users    VM-3         runn   0  252.3M paramount-27.re 00 00:24:42
  1730 crohr    users    VM-4         runn   0  252.3M paradent-15.ren 00 00:24:42
  1731 crohr    users    VM-9         runn   0  252.3M paradent-15.ren 00 00:24:42
  1732 crohr    users    VM-7         runn   0  252.3M paradent-15.ren 00 00:24:42
  1733 crohr    users    VM-2         runn   0  252.3M paramount-25.re 00 00:24:42
  1734 crohr    users    VM-13        runn   0  252.3M paramount-25.re 00 00:24:39
  1735 crohr    users    VM-12        runn   0  252.3M paramount-25.re 00 00:24:39
  1736 crohr    users    VM-17        runn   0  252.3M paramount-27.re 00 00:24:38
  1737 crohr    users    VM-15        runn   0  252.3M paramount-27.re 00 00:24:38
  1738 crohr    users    VM-14        runn   0  252.3M paramount-27.re 00 00:24:38
  1739 crohr    users    VM-16        runn   0  252.3M paradent-15.ren 00 00:24:38
  1740 crohr    users    VM-10        runn   0  252.3M paradent-15.ren 00 00:24:38
  1741 crohr    users    VM-11        runn   0  252.3M paradent-15.ren 00 00:24:38
  1742 crohr    users    VM-18        runn   0  252.3M paramount-25.re 00 00:24:38
  1743 crohr    users    VM-19        runn   0  252.3M paramount-25.re 00 00:24:38
  1744 crohr    users    VM-20        runn   0  252.3M paramount-25.re 00 00:24:36
  1745 crohr    users    VM-21        runn   0  252.3M paramount-27.re 00 00:24:36
  1746 crohr    users    VM-22        runn   0  252.3M paramount-27.re 00 00:24:35
  1747 crohr    users    VM-23        runn   0  252.3M paramount-27.re 00 00:24:34
  1748 crohr    users    VM-25        runn   0  252.3M paradent-15.ren 00 00:24:34
  1749 crohr    users    VM-24        runn   0  252.3M paradent-15.ren 00 00:24:34
  1750 crohr    users    VM-26        runn   0  252.3M paradent-15.ren 00 00:24:34
  1751 crohr    users    VM-27        runn   0  252.3M paramount-25.re 00 00:24:33
  1752 crohr    users    VM-29        runn   0  252.3M paramount-25.re 00 00:24:33
  1753 crohr    users    VM-28        runn   0  252.3M paramount-25.re 00 00:24:33
  1754 crohr    users    VM-30        runn   0  252.3M paramount-27.re 00 00:24:32
  1755 crohr    users    VM-31        runn   0  252.3M paramount-27.re 00 00:24:31
  1756 crohr    users    VM-33        runn   0  252.3M paramount-27.re 00 00:24:31
  1757 crohr    users    VM-32        runn   0  252.3M paradent-15.ren 00 00:24:31
  1758 crohr    users    VM-34        runn   0  252.3M paradent-15.ren 00 00:24:30
  1759 crohr    users    VM-35        runn   0  252.3M paradent-15.ren 00 00:24:30
  1760 crohr    users    VM-36        runn   0  252.3M paramount-25.re 00 00:24:30
  1761 crohr    users    VM-37        runn   0  252.3M paramount-27.re 00 00:24:28
  1762 crohr    users    VM-38        runn   0  252.3M paramount-27.re 00 00:24:28
  1763 crohr    users    VM-39        runn   0  252.3M paramount-27.re 00 00:24:28
  1764 crohr    users    VM-40        runn   0  252.3M paradent-15.ren 00 00:24:27
  1765 crohr    users    VM-41        runn   0  252.3M paradent-15.ren 00 00:24:27
  1766 crohr    users    VM-42        runn   0  252.3M paradent-15.ren 00 00:24:27
  1767 crohr    users    VM-43        fail   0      0K                 00 00:00:39
  1768 crohr    users    VM-44        runn   0  252.3M paradent-15.ren 00 00:24:25
  1769 crohr    users    VM-46        runn   0  252.3M paradent-15.ren 00 00:24:25
  1770 crohr    users    VM-45        runn   0  252.3M paradent-15.ren 00 00:24:25
  1771 crohr    users    VM-47        runn   0  252.3M paradent-15.ren 00 00:24:24
  1772 crohr    users    VM-49        runn   0  252.3M paradent-15.ren 00 00:24:24
  1773 crohr    users    VM-48        runn   0  252.3M paradent-15.ren 00 00:24:24
  1774 crohr    users    VM-51        runn   0  252.3M paradent-15.ren 00 00:24:22
  1775 crohr    users    VM-52        fail   0      0K                 00 00:00:32
  1776 crohr    users    VM-53        fail   0      0K                 00 00:00:34
  1777 crohr    users    VM-50        runn   0  252.3M paradent-15.ren 00 00:24:22
  1778 crohr    users    VM-54        runn   0  252.3M paradent-15.ren 00 00:24:21
  1779 crohr    users    VM-55        runn   0  252.3M paradent-15.ren 00 00:24:21
  1780 crohr    users    VM-56        fail   0      0K                 00 00:00:32
  1781 crohr    users    VM-59        fail   0      0K                 00 00:00:31
  1782 crohr    users    VM-58        runn   0  252.3M paradent-15.ren 00 00:24:19
  1783 crohr    users    VM-57        runn   0  252.3M paradent-15.ren 00 00:24:19
  1784 crohr    users    VM-60        runn   0  252.3M paradent-15.ren 00 00:24:18
  1785 crohr    users    VM-61        runn   0  252.3M paradent-15.ren 00 00:24:18
  1786 crohr    users    VM-62        runn   0  252.3M paradent-15.ren 00 00:24:18
  1787 crohr    users    VM-63        runn   0  252.3M paradent-15.ren 00 00:24:18
  1788 crohr    users    VM-64        runn   0  252.3M paradent-15.ren 00 00:24:16
  1789 crohr    users    VM-66        pend   0      0K                 00 00:24:16
  1790 crohr    users    VM-65        pend   0      0K                 00 00:24:16
  1791 crohr    users    VM-68        pend   0      0K                 00 00:24:15
  1792 crohr    users    VM-67        pend   0      0K                 00 00:24:15
  1793 crohr    users    VM-70        pend   0      0K                 00 00:24:15
  1794 crohr    users    VM-69        pend   0      0K                 00 00:24:15
  1795 crohr    users    VM-71        pend   0      0K                 00 00:24:15
  1796 crohr    users    VM-73        pend   0      0K                 00 00:24:14
  1797 crohr    users    VM-72        pend   0      0K                 00 00:24:14
  1798 crohr    users    VM-74        pend   0      0K                 00 00:24:14
  1799 crohr    users    VM-76        pend   0      0K                 00 00:24:12
  1800 crohr    users    VM-75        pend   0      0K                 00 00:24:12
  1801 crohr    users    VM-78        pend   0      0K                 00 00:24:11
  1802 crohr    users    VM-81        pend   0      0K                 00 00:24:10
  1803 crohr    users    VM-80        pend   0      0K                 00 00:24:10
  1804 crohr    users    VM-79        pend   0      0K                 00 00:24:10
  1805 crohr    users    VM-77        pend   0      0K                 00 00:24:10
  1806 crohr    users    VM-84        pend   0      0K                 00 00:24:09
  1807 crohr    users    VM-82        pend   0      0K                 00 00:24:09
  1808 crohr    users    VM-83        pend   0      0K                 00 00:24:09
  1809 crohr    users    VM-85        pend   0      0K                 00 00:24:09
  1810 crohr    users    VM-86        pend   0      0K                 00 00:24:09
  1811 crohr    users    VM-87        pend   0      0K                 00 00:24:07
  1812 crohr    users    VM-89        pend   0      0K                 00 00:24:07
  1813 crohr    users    VM-90        pend   0      0K                 00 00:24:06
  1814 crohr    users    VM-88        pend   0      0K                 00 00:24:06
  1815 crohr    users    VM-91        pend   0      0K                 00 00:24:06
  1816 crohr    users    VM-92        pend   0      0K                 00 00:24:06
  1817 crohr    users    VM-94        pend   0      0K                 00 00:24:06
  1818 crohr    users    VM-93        pend   0      0K                 00 00:24:06
  1819 crohr    users    VM-95        pend   0      0K                 00 00:24:06
  1820 crohr    users    VM-96        pend   0      0K                 00 00:24:05
  1821 crohr    users    VM-97        pend   0      0K                 00 00:24:04
  1822 crohr    users    VM-98        pend   0      0K                 00 00:24:04
  1823 crohr    users    VM-99        pend   0      0K                 00 00:24:02
  1824 crohr    users    VM-100       pend   0      0K                 00 00:24:02
  1825 crohr    users    VM-101       pend   0      0K                 00 00:24:02
  1826 crohr    users    VM-102       pend   0      0K                 00 00:24:02
  1827 crohr    users    VM-103       pend   0      0K                 00 00:24:02
  1828 crohr    users    VM-104       pend   0      0K                 00 00:24:02
  1829 crohr    users    VM-105       pend   0      0K                 00 00:24:02
  1830 crohr    users    VM-106       pend   0      0K                 00 00:24:02
  1831 crohr    users    VM-107       pend   0      0K                 00 00:24:00
  1832 crohr    users    VM-108       pend   0      0K                 00 00:23:59
  1833 crohr    users    VM-109       pend   0      0K                 00 00:23:59
  1834 crohr    users    VM-111       pend   0      0K                 00 00:23:58
  1835 crohr    users    VM-110       pend   0      0K                 00 00:23:58
  1836 crohr    users    VM-112       pend   0      0K                 00 00:23:58
  1837 crohr    users    VM-113       pend   0      0K                 00 00:23:58
  1838 crohr    users    VM-114       pend   0      0K                 00 00:23:57
  1839 crohr    users    VM-115       pend   0      0K                 00 00:23:57
  1840 crohr    users    VM-116       pend   0      0K                 00 00:23:57
  1841 crohr    users    VM-117       pend   0      0K                 00 00:23:57
  1842 crohr    users    VM-118       pend   0      0K                 00 00:23:55
  1843 crohr    users    VM-119       pend   0      0K                 00 00:23:55
  1844 crohr    users    VM-120       pend   0      0K                 00 00:23:55
  1845 crohr    users    VM-122       pend   0      0K                 00 00:23:55
  1846 crohr    users    VM-121       pend   0      0K                 00 00:23:55
  1847 crohr    users    VM-123       pend   0      0K                 00 00:23:55
  1848 crohr    users    VM-124       pend   0      0K                 00 00:23:53
  1849 crohr    users    VM-125       pend   0      0K                 00 00:23:53
  1850 crohr    users    VM-126       pend   0      0K                 00 00:23:53
  1851 crohr    users    VM-127       pend   0      0K                 00 00:23:53
  1852 crohr    users    VM-128       pend   0      0K                 00 00:23:53
  1853 crohr    users    VM-131       pend   0      0K                 00 00:23:52
  1854 crohr    users    VM-130       pend   0      0K                 00 00:23:52
  1855 crohr    users    VM-129       pend   0      0K                 00 00:23:52
  1856 crohr    users    VM-132       pend   0      0K                 00 00:23:51
  1857 crohr    users    VM-133       pend   0      0K                 00 00:23:50
  1858 crohr    users    VM-134       pend   0      0K                 00 00:23:50
  1859 crohr    users    VM-135       pend   0      0K                 00 00:23:50
  1860 crohr    users    VM-137       pend   0      0K                 00 00:23:49
  1861 crohr    users    VM-136       pend   0      0K                 00 00:23:49
  1862 crohr    users    VM-138       pend   0      0K                 00 00:23:49
  1863 crohr    users    VM-139       pend   0      0K                 00 00:23:48
  1864 crohr    users    VM-140       pend   0      0K                 00 00:23:48
  1865 crohr    users    VM-141       pend   0      0K                 00 00:23:47
  1866 crohr    users    VM-143       pend   0      0K                 00 00:23:47
  1867 crohr    users    VM-142       pend   0      0K                 00 00:23:47
  1868 crohr    users    VM-144       pend   0      0K                 00 00:23:47
  1869 crohr    users    VM-145       pend   0      0K                 00 00:23:47
  1870 crohr    users    VM-146       pend   0      0K                 00 00:23:45
  1871 crohr    users    VM-149       pend   0      0K                 00 00:23:45
  1872 crohr    users    VM-147       pend   0      0K                 00 00:23:45
  1873 crohr    users    VM-148       pend   0      0K                 00 00:23:45
  1874 jenkins  users    fr-inria--VM pend   0      0K                 00 00:03:00

real    1m13.646s
user    1m12.723s
sys    0m0.696s

This is using a MySQL database (also tested with Sqlite).

After switching to NOKOGIRI, it takes only 10s less, so no real speedup.
Any chance you have a look?

Associated revisions

Revision 17e3ffc3
Added by Daniel Molina about 8 years ago

bug #1017: Add Nokogiri XMLRPCParser

Revision 575a2068
Added by Daniel Molina almost 8 years ago

bug #1017: Add Nokogiri XMLRPCParser(cherry picked from commit 17e3ffc31e20e17285c36de615862b147577d16a)

History

#1 Updated by Cyril Rohr about 8 years ago

OK, finally found the issue: the REXMLStreamParser XMLRPC is using by default is really slow. Switching to a Nokogiri based stream parser speeds up onevm list and compute listing from the OCCI interface by a factor of 100:

With default REXML Stream Parser:

-bash-3.2$ time onevm list a
    ID USER     GROUP    NAME         STAT CPU     MEM        HOSTNAME        TIME
  2062 crohr    users    VM-2         prol   0      0K paradent-15.ren 00 00:06:18
  2063 crohr    users    VM-5         prol   0      0K paradent-15.ren 00 00:06:18
  2064 crohr    users    VM-3         prol   0      0K paradent-15.ren 00 00:06:18
  2065 crohr    users    VM-9         prol   0      0K paradent-15.ren 00 00:06:18
  2066 crohr    users    VM-8         prol   0      0K paradent-15.ren 00 00:06:18
  2067 crohr    users    VM-0         prol   0      0K paradent-15.ren 00 00:06:18
  2068 crohr    users    VM-4         prol   0      0K paradent-15.ren 00 00:06:18
  2069 crohr    users    VM-6         prol   0      0K paradent-15.ren 00 00:06:18
  2070 crohr    users    VM-1         prol   0      0K paradent-15.ren 00 00:06:18
  2071 crohr    users    VM-7         prol   0      0K paradent-15.ren 00 00:06:18
  2072 crohr    users    VM-10        prol   0      0K paradent-15.ren 00 00:06:16
  2073 crohr    users    VM-14        prol   0      0K paradent-15.ren 00 00:06:15
  2074 crohr    users    VM-13        prol   0      0K paradent-15.ren 00 00:06:15
  2075 crohr    users    VM-11        prol   0      0K paradent-15.ren 00 00:06:15
  2076 crohr    users    VM-12        prol   0      0K paradent-15.ren 00 00:06:15
  2077 crohr    users    VM-15        prol   0      0K paradent-15.ren 00 00:06:14
  2078 crohr    users    VM-17        prol   0      0K paradent-15.ren 00 00:06:14
  2079 crohr    users    VM-19        prol   0      0K paradent-15.ren 00 00:06:14
  2080 crohr    users    VM-16        prol   0      0K paradent-15.ren 00 00:06:14
  2081 crohr    users    VM-18        prol   0      0K paradent-15.ren 00 00:06:14
  2082 crohr    users    VM-20        prol   0      0K paradent-15.ren 00 00:06:12
  2083 crohr    users    VM-21        prol   0      0K paradent-15.ren 00 00:06:12
  2084 crohr    users    VM-22        prol   0      0K paradent-15.ren 00 00:06:12
  2085 crohr    users    VM-23        prol   0      0K paradent-15.ren 00 00:06:12
  2086 crohr    users    VM-24        prol   0      0K paradent-15.ren 00 00:06:12
  2087 crohr    users    VM-25        prol   0      0K paradent-15.ren 00 00:06:12
  2088 crohr    users    VM-26        prol   0      0K paradent-15.ren 00 00:06:10
  2089 crohr    users    VM-29        prol   0      0K paradent-15.ren 00 00:06:10
  2090 crohr    users    VM-27        prol   0      0K paradent-15.ren 00 00:06:10
  2091 crohr    users    VM-28        prol   0      0K paradent-15.ren 00 00:06:10
  2092 crohr    users    VM-31        prol   0      0K paradent-15.ren 00 00:06:09
  2093 crohr    users    VM-30        prol   0      0K paradent-15.ren 00 00:06:09
  2094 crohr    users    VM-35        pend   0      0K                 00 00:06:08
  2095 crohr    users    VM-33        pend   0      0K                 00 00:06:08
  2096 crohr    users    VM-32        pend   0      0K                 00 00:06:08
  2097 crohr    users    VM-34        pend   0      0K                 00 00:06:08
  2098 crohr    users    VM-36        pend   0      0K                 00 00:06:07
  2099 crohr    users    VM-37        pend   0      0K                 00 00:06:07
  2100 crohr    users    VM-38        pend   0      0K                 00 00:06:07
  2101 crohr    users    VM-39        pend   0      0K                 00 00:06:07
  2102 crohr    users    VM-41        pend   0      0K                 00 00:06:06
  2103 crohr    users    VM-40        pend   0      0K                 00 00:06:06
  2104 crohr    users    VM-42        pend   0      0K                 00 00:06:05
  2105 crohr    users    VM-43        pend   0      0K                 00 00:06:04
  2106 crohr    users    VM-44        pend   0      0K                 00 00:06:04
  2107 crohr    users    VM-45        pend   0      0K                 00 00:06:04
  2108 crohr    users    VM-48        pend   0      0K                 00 00:06:04
  2109 crohr    users    VM-47        pend   0      0K                 00 00:06:03
  2110 crohr    users    VM-49        pend   0      0K                 00 00:06:03
  2111 crohr    users    VM-46        pend   0      0K                 00 00:06:03
  2112 crohr    users    VM-50        pend   0      0K                 00 00:06:03
  2113 crohr    users    VM-51        pend   0      0K                 00 00:06:03
  2114 crohr    users    VM-52        pend   0      0K                 00 00:06:02
  2115 crohr    users    VM-53        pend   0      0K                 00 00:06:01
  2116 crohr    users    VM-54        pend   0      0K                 00 00:06:01
  2117 crohr    users    VM-55        pend   0      0K                 00 00:06:01
  2118 crohr    users    VM-56        pend   0      0K                 00 00:06:01
  2119 crohr    users    VM-59        pend   0      0K                 00 00:06:00
  2120 crohr    users    VM-58        pend   0      0K                 00 00:06:00
  2121 crohr    users    VM-57        pend   0      0K                 00 00:06:00
  2122 crohr    users    VM-60        pend   0      0K                 00 00:06:00
  2123 crohr    users    VM-61        pend   0      0K                 00 00:06:00
  2124 crohr    users    VM-62        pend   0      0K                 00 00:05:59
  2125 crohr    users    VM-63        pend   0      0K                 00 00:05:59
  2126 crohr    users    VM-64        pend   0      0K                 00 00:05:58
  2127 crohr    users    VM-65        pend   0      0K                 00 00:05:58
  2128 crohr    users    VM-66        pend   0      0K                 00 00:05:58
  2129 crohr    users    VM-67        pend   0      0K                 00 00:05:56
  2130 crohr    users    VM-70        pend   0      0K                 00 00:05:56
  2131 crohr    users    VM-69        pend   0      0K                 00 00:05:56
  2132 crohr    users    VM-71        pend   0      0K                 00 00:05:56
  2133 crohr    users    VM-68        pend   0      0K                 00 00:05:56
  2134 crohr    users    VM-72        pend   0      0K                 00 00:05:55
  2135 crohr    users    VM-73        pend   0      0K                 00 00:05:55
  2136 crohr    users    VM-74        pend   0      0K                 00 00:05:54
  2137 crohr    users    VM-76        pend   0      0K                 00 00:05:54
  2138 crohr    users    VM-75        pend   0      0K                 00 00:05:54
  2139 crohr    users    VM-79        pend   0      0K                 00 00:05:53
  2140 crohr    users    VM-78        pend   0      0K                 00 00:05:53
  2141 crohr    users    VM-81        pend   0      0K                 00 00:05:53
  2142 crohr    users    VM-77        pend   0      0K                 00 00:05:53
  2143 crohr    users    VM-82        pend   0      0K                 00 00:05:52
  2144 crohr    users    VM-83        pend   0      0K                 00 00:05:52
  2145 crohr    users    VM-80        pend   0      0K                 00 00:05:52
  2146 crohr    users    VM-86        pend   0      0K                 00 00:05:51
  2147 crohr    users    VM-84        pend   0      0K                 00 00:05:51
  2148 crohr    users    VM-85        pend   0      0K                 00 00:05:50
  2149 crohr    users    VM-87        pend   0      0K                 00 00:05:49
  2150 crohr    users    VM-89        pend   0      0K                 00 00:05:49
  2151 crohr    users    VM-88        pend   0      0K                 00 00:05:49
  2152 crohr    users    VM-90        pend   0      0K                 00 00:05:49
  2153 crohr    users    VM-92        pend   0      0K                 00 00:05:48
  2154 crohr    users    VM-91        pend   0      0K                 00 00:05:48
  2155 crohr    users    VM-93        pend   0      0K                 00 00:05:48
  2156 crohr    users    VM-94        pend   0      0K                 00 00:05:47
  2157 crohr    users    VM-95        pend   0      0K                 00 00:05:47
  2158 crohr    users    VM-96        pend   0      0K                 00 00:05:47
  2159 crohr    users    VM-97        pend   0      0K                 00 00:05:47
  2160 crohr    users    VM-98        pend   0      0K                 00 00:05:47
  2161 crohr    users    VM-99        pend   0      0K                 00 00:05:47
  2162 crohr    users    VM-100       pend   0      0K                 00 00:05:46
  2163 crohr    users    VM-102       pend   0      0K                 00 00:05:45
  2164 crohr    users    VM-103       pend   0      0K                 00 00:05:44
  2165 crohr    users    VM-101       pend   0      0K                 00 00:05:44
  2166 crohr    users    VM-104       pend   0      0K                 00 00:05:44
  2167 crohr    users    VM-106       pend   0      0K                 00 00:05:44
  2168 crohr    users    VM-105       pend   0      0K                 00 00:05:44
  2169 crohr    users    VM-107       pend   0      0K                 00 00:05:43
  2170 crohr    users    VM-108       pend   0      0K                 00 00:05:43
  2171 crohr    users    VM-109       pend   0      0K                 00 00:05:43
  2172 crohr    users    VM-110       pend   0      0K                 00 00:05:43
  2173 crohr    users    VM-111       pend   0      0K                 00 00:05:42
  2174 crohr    users    VM-112       pend   0      0K                 00 00:05:41
  2175 crohr    users    VM-113       pend   0      0K                 00 00:05:41
  2176 crohr    users    VM-114       pend   0      0K                 00 00:05:41
  2177 crohr    users    VM-115       pend   0      0K                 00 00:05:41
  2178 crohr    users    VM-116       pend   0      0K                 00 00:05:41
  2179 crohr    users    VM-117       pend   0      0K                 00 00:05:41
  2180 crohr    users    VM-118       pend   0      0K                 00 00:05:41
  2181 crohr    users    VM-119       pend   0      0K                 00 00:05:39
  2182 crohr    users    VM-120       pend   0      0K                 00 00:05:39
  2183 crohr    users    VM-121       pend   0      0K                 00 00:05:39
  2184 crohr    users    VM-122       pend   0      0K                 00 00:05:38
  2185 crohr    users    VM-123       pend   0      0K                 00 00:05:37
  2186 crohr    users    VM-124       pend   0      0K                 00 00:05:37
  2187 crohr    users    VM-128       pend   0      0K                 00 00:05:37
  2188 crohr    users    VM-127       pend   0      0K                 00 00:05:37
  2189 crohr    users    VM-126       pend   0      0K                 00 00:05:37
  2190 crohr    users    VM-125       pend   0      0K                 00 00:05:37
  2191 crohr    users    VM-129       pend   0      0K                 00 00:05:36
  2192 crohr    users    VM-130       pend   0      0K                 00 00:05:36
  2193 crohr    users    VM-131       pend   0      0K                 00 00:05:36
  2194 crohr    users    VM-132       pend   0      0K                 00 00:05:35
  2195 crohr    users    VM-138       pend   0      0K                 00 00:05:33
  2196 crohr    users    VM-133       pend   0      0K                 00 00:05:33
  2197 crohr    users    VM-136       pend   0      0K                 00 00:05:33
  2198 crohr    users    VM-137       pend   0      0K                 00 00:05:33
  2199 crohr    users    VM-134       pend   0      0K                 00 00:05:33
  2200 crohr    users    VM-135       pend   0      0K                 00 00:05:33
  2201 crohr    users    VM-139       pend   0      0K                 00 00:05:33
  2202 crohr    users    VM-141       pend   0      0K                 00 00:05:33
  2203 crohr    users    VM-140       pend   0      0K                 00 00:05:32
  2204 crohr    users    VM-142       pend   0      0K                 00 00:05:32
  2205 crohr    users    VM-144       pend   0      0K                 00 00:05:30
  2206 crohr    users    VM-143       pend   0      0K                 00 00:05:30
  2207 crohr    users    VM-145       pend   0      0K                 00 00:05:30
  2208 crohr    users    VM-149       pend   0      0K                 00 00:05:29
  2209 crohr    users    VM-148       pend   0      0K                 00 00:05:29
  2210 crohr    users    VM-146       pend   0      0K                 00 00:05:29
  2211 crohr    users    VM-147       pend   0      0K                 00 00:05:29

real    0m58.317s
user    0m57.863s
sys    0m0.362s
crohr@parachute-wifi:~/bonfire-ssl $ time curl -K secure.curl https://frontend.integration.bonfire.grid5000.fr/compute
<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/0.8.55</center>
</body>
</html>

real    0m30.737s
user    0m0.026s
sys    0m0.008s

With Nokogiri Stream Parser:

-bash-3.2$ time onevm list a
    ID USER     GROUP    NAME         STAT CPU     MEM        HOSTNAME        TIME
  2062 crohr    users    VM-2         prol   0      0K paradent-15.ren 00 00:04:35
  2063 crohr    users    VM-5         prol   0      0K paradent-15.ren 00 00:04:35
  2064 crohr    users    VM-3         prol   0      0K paradent-15.ren 00 00:04:35
  2065 crohr    users    VM-9         prol   0      0K paradent-15.ren 00 00:04:35
  2066 crohr    users    VM-8         prol   0      0K paradent-15.ren 00 00:04:35
  2067 crohr    users    VM-0         prol   0      0K paradent-15.ren 00 00:04:35
  2068 crohr    users    VM-4         prol   0      0K paradent-15.ren 00 00:04:35
  2069 crohr    users    VM-6         prol   0      0K paradent-15.ren 00 00:04:35
  2070 crohr    users    VM-1         prol   0      0K paradent-15.ren 00 00:04:35
  2071 crohr    users    VM-7         prol   0      0K paradent-15.ren 00 00:04:35
  2072 crohr    users    VM-10        prol   0      0K paradent-15.ren 00 00:04:33
  2073 crohr    users    VM-14        prol   0      0K paradent-15.ren 00 00:04:32
  2074 crohr    users    VM-13        prol   0      0K paradent-15.ren 00 00:04:32
  2075 crohr    users    VM-11        prol   0      0K paradent-15.ren 00 00:04:32
  2076 crohr    users    VM-12        prol   0      0K paradent-15.ren 00 00:04:32
  2077 crohr    users    VM-15        prol   0      0K paradent-15.ren 00 00:04:31
  2078 crohr    users    VM-17        prol   0      0K paradent-15.ren 00 00:04:31
  2079 crohr    users    VM-19        prol   0      0K paradent-15.ren 00 00:04:31
  2080 crohr    users    VM-16        prol   0      0K paradent-15.ren 00 00:04:31
  2081 crohr    users    VM-18        prol   0      0K paradent-15.ren 00 00:04:31
  2082 crohr    users    VM-20        prol   0      0K paradent-15.ren 00 00:04:29
  2083 crohr    users    VM-21        prol   0      0K paradent-15.ren 00 00:04:29
  2084 crohr    users    VM-22        prol   0      0K paradent-15.ren 00 00:04:29
  2085 crohr    users    VM-23        prol   0      0K paradent-15.ren 00 00:04:29
  2086 crohr    users    VM-24        prol   0      0K paradent-15.ren 00 00:04:29
  2087 crohr    users    VM-25        prol   0      0K paradent-15.ren 00 00:04:29
  2088 crohr    users    VM-26        prol   0      0K paradent-15.ren 00 00:04:27
  2089 crohr    users    VM-29        prol   0      0K paradent-15.ren 00 00:04:27
  2090 crohr    users    VM-27        prol   0      0K paradent-15.ren 00 00:04:27
  2091 crohr    users    VM-28        prol   0      0K paradent-15.ren 00 00:04:27
  2092 crohr    users    VM-31        prol   0      0K paradent-15.ren 00 00:04:26
  2093 crohr    users    VM-30        prol   0      0K paradent-15.ren 00 00:04:26
  2094 crohr    users    VM-35        pend   0      0K                 00 00:04:25
  2095 crohr    users    VM-33        pend   0      0K                 00 00:04:25
  2096 crohr    users    VM-32        pend   0      0K                 00 00:04:25
  2097 crohr    users    VM-34        pend   0      0K                 00 00:04:25
  2098 crohr    users    VM-36        pend   0      0K                 00 00:04:24
  2099 crohr    users    VM-37        pend   0      0K                 00 00:04:24
  2100 crohr    users    VM-38        pend   0      0K                 00 00:04:24
  2101 crohr    users    VM-39        pend   0      0K                 00 00:04:24
  2102 crohr    users    VM-41        pend   0      0K                 00 00:04:23
  2103 crohr    users    VM-40        pend   0      0K                 00 00:04:23
  2104 crohr    users    VM-42        pend   0      0K                 00 00:04:22
  2105 crohr    users    VM-43        pend   0      0K                 00 00:04:21
  2106 crohr    users    VM-44        pend   0      0K                 00 00:04:21
  2107 crohr    users    VM-45        pend   0      0K                 00 00:04:21
  2108 crohr    users    VM-48        pend   0      0K                 00 00:04:21
  2109 crohr    users    VM-47        pend   0      0K                 00 00:04:20
  2110 crohr    users    VM-49        pend   0      0K                 00 00:04:20
  2111 crohr    users    VM-46        pend   0      0K                 00 00:04:20
  2112 crohr    users    VM-50        pend   0      0K                 00 00:04:20
  2113 crohr    users    VM-51        pend   0      0K                 00 00:04:20
  2114 crohr    users    VM-52        pend   0      0K                 00 00:04:19
  2115 crohr    users    VM-53        pend   0      0K                 00 00:04:18
  2116 crohr    users    VM-54        pend   0      0K                 00 00:04:18
  2117 crohr    users    VM-55        pend   0      0K                 00 00:04:18
  2118 crohr    users    VM-56        pend   0      0K                 00 00:04:18
  2119 crohr    users    VM-59        pend   0      0K                 00 00:04:17
  2120 crohr    users    VM-58        pend   0      0K                 00 00:04:17
  2121 crohr    users    VM-57        pend   0      0K                 00 00:04:17
  2122 crohr    users    VM-60        pend   0      0K                 00 00:04:17
  2123 crohr    users    VM-61        pend   0      0K                 00 00:04:17
  2124 crohr    users    VM-62        pend   0      0K                 00 00:04:16
  2125 crohr    users    VM-63        pend   0      0K                 00 00:04:16
  2126 crohr    users    VM-64        pend   0      0K                 00 00:04:15
  2127 crohr    users    VM-65        pend   0      0K                 00 00:04:15
  2128 crohr    users    VM-66        pend   0      0K                 00 00:04:15
  2129 crohr    users    VM-67        pend   0      0K                 00 00:04:13
  2130 crohr    users    VM-70        pend   0      0K                 00 00:04:13
  2131 crohr    users    VM-69        pend   0      0K                 00 00:04:13
  2132 crohr    users    VM-71        pend   0      0K                 00 00:04:13
  2133 crohr    users    VM-68        pend   0      0K                 00 00:04:13
  2134 crohr    users    VM-72        pend   0      0K                 00 00:04:12
  2135 crohr    users    VM-73        pend   0      0K                 00 00:04:12
  2136 crohr    users    VM-74        pend   0      0K                 00 00:04:11
  2137 crohr    users    VM-76        pend   0      0K                 00 00:04:11
  2138 crohr    users    VM-75        pend   0      0K                 00 00:04:11
  2139 crohr    users    VM-79        pend   0      0K                 00 00:04:10
  2140 crohr    users    VM-78        pend   0      0K                 00 00:04:10
  2141 crohr    users    VM-81        pend   0      0K                 00 00:04:10
  2142 crohr    users    VM-77        pend   0      0K                 00 00:04:10
  2143 crohr    users    VM-82        pend   0      0K                 00 00:04:09
  2144 crohr    users    VM-83        pend   0      0K                 00 00:04:09
  2145 crohr    users    VM-80        pend   0      0K                 00 00:04:09
  2146 crohr    users    VM-86        pend   0      0K                 00 00:04:08
  2147 crohr    users    VM-84        pend   0      0K                 00 00:04:08
  2148 crohr    users    VM-85        pend   0      0K                 00 00:04:07
  2149 crohr    users    VM-87        pend   0      0K                 00 00:04:06
  2150 crohr    users    VM-89        pend   0      0K                 00 00:04:06
  2151 crohr    users    VM-88        pend   0      0K                 00 00:04:06
  2152 crohr    users    VM-90        pend   0      0K                 00 00:04:06
  2153 crohr    users    VM-92        pend   0      0K                 00 00:04:05
  2154 crohr    users    VM-91        pend   0      0K                 00 00:04:05
  2155 crohr    users    VM-93        pend   0      0K                 00 00:04:05
  2156 crohr    users    VM-94        pend   0      0K                 00 00:04:04
  2157 crohr    users    VM-95        pend   0      0K                 00 00:04:04
  2158 crohr    users    VM-96        pend   0      0K                 00 00:04:04
  2159 crohr    users    VM-97        pend   0      0K                 00 00:04:04
  2160 crohr    users    VM-98        pend   0      0K                 00 00:04:04
  2161 crohr    users    VM-99        pend   0      0K                 00 00:04:04
  2162 crohr    users    VM-100       pend   0      0K                 00 00:04:03
  2163 crohr    users    VM-102       pend   0      0K                 00 00:04:02
  2164 crohr    users    VM-103       pend   0      0K                 00 00:04:01
  2165 crohr    users    VM-101       pend   0      0K                 00 00:04:01
  2166 crohr    users    VM-104       pend   0      0K                 00 00:04:01
  2167 crohr    users    VM-106       pend   0      0K                 00 00:04:01
  2168 crohr    users    VM-105       pend   0      0K                 00 00:04:01
  2169 crohr    users    VM-107       pend   0      0K                 00 00:04:00
  2170 crohr    users    VM-108       pend   0      0K                 00 00:04:00
  2171 crohr    users    VM-109       pend   0      0K                 00 00:04:00
  2172 crohr    users    VM-110       pend   0      0K                 00 00:04:00
  2173 crohr    users    VM-111       pend   0      0K                 00 00:03:59
  2174 crohr    users    VM-112       pend   0      0K                 00 00:03:58
  2175 crohr    users    VM-113       pend   0      0K                 00 00:03:58
  2176 crohr    users    VM-114       pend   0      0K                 00 00:03:58
  2177 crohr    users    VM-115       pend   0      0K                 00 00:03:58
  2178 crohr    users    VM-116       pend   0      0K                 00 00:03:58
  2179 crohr    users    VM-117       pend   0      0K                 00 00:03:58
  2180 crohr    users    VM-118       pend   0      0K                 00 00:03:58
  2181 crohr    users    VM-119       pend   0      0K                 00 00:03:56
  2182 crohr    users    VM-120       pend   0      0K                 00 00:03:56
  2183 crohr    users    VM-121       pend   0      0K                 00 00:03:56
  2184 crohr    users    VM-122       pend   0      0K                 00 00:03:55
  2185 crohr    users    VM-123       pend   0      0K                 00 00:03:54
  2186 crohr    users    VM-124       pend   0      0K                 00 00:03:54
  2187 crohr    users    VM-128       pend   0      0K                 00 00:03:54
  2188 crohr    users    VM-127       pend   0      0K                 00 00:03:54
  2189 crohr    users    VM-126       pend   0      0K                 00 00:03:54
  2190 crohr    users    VM-125       pend   0      0K                 00 00:03:54
  2191 crohr    users    VM-129       pend   0      0K                 00 00:03:53
  2192 crohr    users    VM-130       pend   0      0K                 00 00:03:53
  2193 crohr    users    VM-131       pend   0      0K                 00 00:03:53
  2194 crohr    users    VM-132       pend   0      0K                 00 00:03:52
  2195 crohr    users    VM-138       pend   0      0K                 00 00:03:50
  2196 crohr    users    VM-133       pend   0      0K                 00 00:03:50
  2197 crohr    users    VM-136       pend   0      0K                 00 00:03:50
  2198 crohr    users    VM-137       pend   0      0K                 00 00:03:50
  2199 crohr    users    VM-134       pend   0      0K                 00 00:03:50
  2200 crohr    users    VM-135       pend   0      0K                 00 00:03:50
  2201 crohr    users    VM-139       pend   0      0K                 00 00:03:50
  2202 crohr    users    VM-141       pend   0      0K                 00 00:03:50
  2203 crohr    users    VM-140       pend   0      0K                 00 00:03:49
  2204 crohr    users    VM-142       pend   0      0K                 00 00:03:49
  2205 crohr    users    VM-144       pend   0      0K                 00 00:03:47
  2206 crohr    users    VM-143       pend   0      0K                 00 00:03:47
  2207 crohr    users    VM-145       pend   0      0K                 00 00:03:47
  2208 crohr    users    VM-149       pend   0      0K                 00 00:03:46
  2209 crohr    users    VM-148       pend   0      0K                 00 00:03:46
  2210 crohr    users    VM-146       pend   0      0K                 00 00:03:46
  2211 crohr    users    VM-147       pend   0      0K                 00 00:03:46

real    0m0.853s
user    0m0.590s
sys    0m0.182s
crohr@parachute-wifi:~/bonfire-ssl $ time curl -I -K secure.curl https://frontend.integration.bonfire.grid5000.fr/compute
HTTP/1.1 200 OK
Server: nginx/0.8.55
Date: Fri, 02 Dec 2011 13:41:29 GMT
Content-Type: text/html;charset=utf-8
Connection: keep-alive
X-Frame-Options: sameorigin
X-XSS-Protection: 1; mode=block
Content-Length: 14488

real    0m0.572s
user    0m0.026s
sys    0m0.007s

So: major speedup (before the change the OCCI request failed because of our nginx timeout setting). I'll send a patch ASAP.

#2 Updated by Cyril Rohr about 8 years ago

The patch is here: http://flag.io/33998d87be6c9f8f3a6b0ae72c65fbb9.diff (to apply on path/to/lib/one/ruby/OpenNebula.rb -- version 3.0.0-bonfire). For reference, here is the full patch content:

85a86,92
>         NOKOGIRI_PARSER = begin
>             require 'stream_parser_mixin'
>             true
>         rescue LoadError
>             false
>         end
> 
132c139,141
<             if XMLPARSER
---
>             if NOKOGIRI_PARSER
>                 @server.set_parser(XMLRPC::XMLParser::NokogiriStreamParser.new)
>             elsif XMLPARSER
134a144
> 

This patch will trigger the use of the Nokogiri Stream Parser if the xmlrpc-streaming Ruby gem (https://github.com/washu/xmlrpc-streaming) is installed on the machine.

#3 Updated by Cyril Rohr about 8 years ago

Just putting here the full command to update a ONE installation until the patch is included upstream:

gem install xmlrpc-streaming nokogiri && curl http://flag.io/33998d87be6c9f8f3a6b0ae72c65fbb9 | patch /usr/lib/one/ruby/OpenNebula.rb -

#4 Updated by Tino Vázquez about 8 years ago

Thanks for the patch Cyril!

I want to point out that this is useful for ruby >= 1.9, what we recommend for other ruby versions is xmlparser (http://rubygems.org/gems/xmlparser), which is installed by default with the provided install_gems (http://opennebula.org/documentation:rel3.0:ignc#ruby_libraries_requirements_front-end).

We will evaluate this solution, it is promising to offer a solution for all ruby versions.

#5 Updated by Javi Fontan about 8 years ago

There's a bun in install_gems and xmlparser is only installed for ruby versions lesser than 1.8.7. To use this gem you can install it manually, the requirements are expat development libraries.

We will check your patch and try to use the most compatible solution for the next version.

#6 Updated by Daniel Molina about 8 years ago

  • Status changed from New to Closed
  • Target version set to Release 3.4 - S0
  • % Done changed from 0 to 100
  • Resolution set to fixed
  • Affected Versions OpenNebula 3.2 added

The performance issue should be fixed with this patch which includes a new Nokogiri parser. If Nokogiri is installed it will be used instead of the REXML parser.

Also available in: Atom PDF