Bug #1017
Serious performance issue with onevm list
| Status: | Closed | Start date: | 12/02/2011 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assignee: |  Tino Vázquez | % Done: | 100% | |
| Category: | CLI | |||
| Target version: | Release 3.4 - S0 | |||
| Resolution: | fixed | Pull request: | ||
| Affected Versions: | OpenNebula 3.2 |
Description
In our local OpenNebula installation, after having submitted ~150 VMs the onevm list command becomes unusable:
-bash-3.2$ time onevm list a
ID USER GROUP NAME STAT CPU MEM HOSTNAME TIME
1710 crohr users VM-0 runn 0 252.3M paramount-2.ren 00 01:15:22
1711 crohr users VM-1 runn 0 252.3M paramount-2.ren 00 01:15:22
1712 crohr users VM-2 runn 0 252.3M paramount-2.ren 00 01:15:21
1713 crohr users VM-3 runn 0 252.3M paramount-10.re 00 01:15:20
1720 crohr users VM-0 runn 0 252.3M paramount-25.re 00 00:26:04
1721 crohr users VM-1 runn 0 252.3M paramount-25.re 00 00:26:04
1722 crohr users VM-2 runn 0 252.3M paramount-25.re 00 00:26:03
1723 crohr users VM-3 runn 0 252.3M paramount-27.re 00 00:26:03
1724 crohr users VM-1 runn 0 252.3M paramount-25.re 00 00:24:42
1725 crohr users VM-0 runn 0 252.3M paramount-25.re 00 00:24:42
1726 crohr users VM-5 runn 0 252.3M paramount-25.re 00 00:24:42
1727 crohr users VM-6 runn 0 252.3M paramount-27.re 00 00:24:42
1728 crohr users VM-8 runn 0 252.3M paramount-27.re 00 00:24:42
1729 crohr users VM-3 runn 0 252.3M paramount-27.re 00 00:24:42
1730 crohr users VM-4 runn 0 252.3M paradent-15.ren 00 00:24:42
1731 crohr users VM-9 runn 0 252.3M paradent-15.ren 00 00:24:42
1732 crohr users VM-7 runn 0 252.3M paradent-15.ren 00 00:24:42
1733 crohr users VM-2 runn 0 252.3M paramount-25.re 00 00:24:42
1734 crohr users VM-13 runn 0 252.3M paramount-25.re 00 00:24:39
1735 crohr users VM-12 runn 0 252.3M paramount-25.re 00 00:24:39
1736 crohr users VM-17 runn 0 252.3M paramount-27.re 00 00:24:38
1737 crohr users VM-15 runn 0 252.3M paramount-27.re 00 00:24:38
1738 crohr users VM-14 runn 0 252.3M paramount-27.re 00 00:24:38
1739 crohr users VM-16 runn 0 252.3M paradent-15.ren 00 00:24:38
1740 crohr users VM-10 runn 0 252.3M paradent-15.ren 00 00:24:38
1741 crohr users VM-11 runn 0 252.3M paradent-15.ren 00 00:24:38
1742 crohr users VM-18 runn 0 252.3M paramount-25.re 00 00:24:38
1743 crohr users VM-19 runn 0 252.3M paramount-25.re 00 00:24:38
1744 crohr users VM-20 runn 0 252.3M paramount-25.re 00 00:24:36
1745 crohr users VM-21 runn 0 252.3M paramount-27.re 00 00:24:36
1746 crohr users VM-22 runn 0 252.3M paramount-27.re 00 00:24:35
1747 crohr users VM-23 runn 0 252.3M paramount-27.re 00 00:24:34
1748 crohr users VM-25 runn 0 252.3M paradent-15.ren 00 00:24:34
1749 crohr users VM-24 runn 0 252.3M paradent-15.ren 00 00:24:34
1750 crohr users VM-26 runn 0 252.3M paradent-15.ren 00 00:24:34
1751 crohr users VM-27 runn 0 252.3M paramount-25.re 00 00:24:33
1752 crohr users VM-29 runn 0 252.3M paramount-25.re 00 00:24:33
1753 crohr users VM-28 runn 0 252.3M paramount-25.re 00 00:24:33
1754 crohr users VM-30 runn 0 252.3M paramount-27.re 00 00:24:32
1755 crohr users VM-31 runn 0 252.3M paramount-27.re 00 00:24:31
1756 crohr users VM-33 runn 0 252.3M paramount-27.re 00 00:24:31
1757 crohr users VM-32 runn 0 252.3M paradent-15.ren 00 00:24:31
1758 crohr users VM-34 runn 0 252.3M paradent-15.ren 00 00:24:30
1759 crohr users VM-35 runn 0 252.3M paradent-15.ren 00 00:24:30
1760 crohr users VM-36 runn 0 252.3M paramount-25.re 00 00:24:30
1761 crohr users VM-37 runn 0 252.3M paramount-27.re 00 00:24:28
1762 crohr users VM-38 runn 0 252.3M paramount-27.re 00 00:24:28
1763 crohr users VM-39 runn 0 252.3M paramount-27.re 00 00:24:28
1764 crohr users VM-40 runn 0 252.3M paradent-15.ren 00 00:24:27
1765 crohr users VM-41 runn 0 252.3M paradent-15.ren 00 00:24:27
1766 crohr users VM-42 runn 0 252.3M paradent-15.ren 00 00:24:27
1767 crohr users VM-43 fail 0 0K 00 00:00:39
1768 crohr users VM-44 runn 0 252.3M paradent-15.ren 00 00:24:25
1769 crohr users VM-46 runn 0 252.3M paradent-15.ren 00 00:24:25
1770 crohr users VM-45 runn 0 252.3M paradent-15.ren 00 00:24:25
1771 crohr users VM-47 runn 0 252.3M paradent-15.ren 00 00:24:24
1772 crohr users VM-49 runn 0 252.3M paradent-15.ren 00 00:24:24
1773 crohr users VM-48 runn 0 252.3M paradent-15.ren 00 00:24:24
1774 crohr users VM-51 runn 0 252.3M paradent-15.ren 00 00:24:22
1775 crohr users VM-52 fail 0 0K 00 00:00:32
1776 crohr users VM-53 fail 0 0K 00 00:00:34
1777 crohr users VM-50 runn 0 252.3M paradent-15.ren 00 00:24:22
1778 crohr users VM-54 runn 0 252.3M paradent-15.ren 00 00:24:21
1779 crohr users VM-55 runn 0 252.3M paradent-15.ren 00 00:24:21
1780 crohr users VM-56 fail 0 0K 00 00:00:32
1781 crohr users VM-59 fail 0 0K 00 00:00:31
1782 crohr users VM-58 runn 0 252.3M paradent-15.ren 00 00:24:19
1783 crohr users VM-57 runn 0 252.3M paradent-15.ren 00 00:24:19
1784 crohr users VM-60 runn 0 252.3M paradent-15.ren 00 00:24:18
1785 crohr users VM-61 runn 0 252.3M paradent-15.ren 00 00:24:18
1786 crohr users VM-62 runn 0 252.3M paradent-15.ren 00 00:24:18
1787 crohr users VM-63 runn 0 252.3M paradent-15.ren 00 00:24:18
1788 crohr users VM-64 runn 0 252.3M paradent-15.ren 00 00:24:16
1789 crohr users VM-66 pend 0 0K 00 00:24:16
1790 crohr users VM-65 pend 0 0K 00 00:24:16
1791 crohr users VM-68 pend 0 0K 00 00:24:15
1792 crohr users VM-67 pend 0 0K 00 00:24:15
1793 crohr users VM-70 pend 0 0K 00 00:24:15
1794 crohr users VM-69 pend 0 0K 00 00:24:15
1795 crohr users VM-71 pend 0 0K 00 00:24:15
1796 crohr users VM-73 pend 0 0K 00 00:24:14
1797 crohr users VM-72 pend 0 0K 00 00:24:14
1798 crohr users VM-74 pend 0 0K 00 00:24:14
1799 crohr users VM-76 pend 0 0K 00 00:24:12
1800 crohr users VM-75 pend 0 0K 00 00:24:12
1801 crohr users VM-78 pend 0 0K 00 00:24:11
1802 crohr users VM-81 pend 0 0K 00 00:24:10
1803 crohr users VM-80 pend 0 0K 00 00:24:10
1804 crohr users VM-79 pend 0 0K 00 00:24:10
1805 crohr users VM-77 pend 0 0K 00 00:24:10
1806 crohr users VM-84 pend 0 0K 00 00:24:09
1807 crohr users VM-82 pend 0 0K 00 00:24:09
1808 crohr users VM-83 pend 0 0K 00 00:24:09
1809 crohr users VM-85 pend 0 0K 00 00:24:09
1810 crohr users VM-86 pend 0 0K 00 00:24:09
1811 crohr users VM-87 pend 0 0K 00 00:24:07
1812 crohr users VM-89 pend 0 0K 00 00:24:07
1813 crohr users VM-90 pend 0 0K 00 00:24:06
1814 crohr users VM-88 pend 0 0K 00 00:24:06
1815 crohr users VM-91 pend 0 0K 00 00:24:06
1816 crohr users VM-92 pend 0 0K 00 00:24:06
1817 crohr users VM-94 pend 0 0K 00 00:24:06
1818 crohr users VM-93 pend 0 0K 00 00:24:06
1819 crohr users VM-95 pend 0 0K 00 00:24:06
1820 crohr users VM-96 pend 0 0K 00 00:24:05
1821 crohr users VM-97 pend 0 0K 00 00:24:04
1822 crohr users VM-98 pend 0 0K 00 00:24:04
1823 crohr users VM-99 pend 0 0K 00 00:24:02
1824 crohr users VM-100 pend 0 0K 00 00:24:02
1825 crohr users VM-101 pend 0 0K 00 00:24:02
1826 crohr users VM-102 pend 0 0K 00 00:24:02
1827 crohr users VM-103 pend 0 0K 00 00:24:02
1828 crohr users VM-104 pend 0 0K 00 00:24:02
1829 crohr users VM-105 pend 0 0K 00 00:24:02
1830 crohr users VM-106 pend 0 0K 00 00:24:02
1831 crohr users VM-107 pend 0 0K 00 00:24:00
1832 crohr users VM-108 pend 0 0K 00 00:23:59
1833 crohr users VM-109 pend 0 0K 00 00:23:59
1834 crohr users VM-111 pend 0 0K 00 00:23:58
1835 crohr users VM-110 pend 0 0K 00 00:23:58
1836 crohr users VM-112 pend 0 0K 00 00:23:58
1837 crohr users VM-113 pend 0 0K 00 00:23:58
1838 crohr users VM-114 pend 0 0K 00 00:23:57
1839 crohr users VM-115 pend 0 0K 00 00:23:57
1840 crohr users VM-116 pend 0 0K 00 00:23:57
1841 crohr users VM-117 pend 0 0K 00 00:23:57
1842 crohr users VM-118 pend 0 0K 00 00:23:55
1843 crohr users VM-119 pend 0 0K 00 00:23:55
1844 crohr users VM-120 pend 0 0K 00 00:23:55
1845 crohr users VM-122 pend 0 0K 00 00:23:55
1846 crohr users VM-121 pend 0 0K 00 00:23:55
1847 crohr users VM-123 pend 0 0K 00 00:23:55
1848 crohr users VM-124 pend 0 0K 00 00:23:53
1849 crohr users VM-125 pend 0 0K 00 00:23:53
1850 crohr users VM-126 pend 0 0K 00 00:23:53
1851 crohr users VM-127 pend 0 0K 00 00:23:53
1852 crohr users VM-128 pend 0 0K 00 00:23:53
1853 crohr users VM-131 pend 0 0K 00 00:23:52
1854 crohr users VM-130 pend 0 0K 00 00:23:52
1855 crohr users VM-129 pend 0 0K 00 00:23:52
1856 crohr users VM-132 pend 0 0K 00 00:23:51
1857 crohr users VM-133 pend 0 0K 00 00:23:50
1858 crohr users VM-134 pend 0 0K 00 00:23:50
1859 crohr users VM-135 pend 0 0K 00 00:23:50
1860 crohr users VM-137 pend 0 0K 00 00:23:49
1861 crohr users VM-136 pend 0 0K 00 00:23:49
1862 crohr users VM-138 pend 0 0K 00 00:23:49
1863 crohr users VM-139 pend 0 0K 00 00:23:48
1864 crohr users VM-140 pend 0 0K 00 00:23:48
1865 crohr users VM-141 pend 0 0K 00 00:23:47
1866 crohr users VM-143 pend 0 0K 00 00:23:47
1867 crohr users VM-142 pend 0 0K 00 00:23:47
1868 crohr users VM-144 pend 0 0K 00 00:23:47
1869 crohr users VM-145 pend 0 0K 00 00:23:47
1870 crohr users VM-146 pend 0 0K 00 00:23:45
1871 crohr users VM-149 pend 0 0K 00 00:23:45
1872 crohr users VM-147 pend 0 0K 00 00:23:45
1873 crohr users VM-148 pend 0 0K 00 00:23:45
1874 jenkins users fr-inria--VM pend 0 0K 00 00:03:00
real 1m13.646s
user 1m12.723s
sys 0m0.696s
This is using a MySQL database (also tested with Sqlite).
After switching to NOKOGIRI, it takes only 10s less, so no real speedup.
Any chance you have a look?
Associated revisions
bug #1017: Add Nokogiri XMLRPCParser
bug #1017: Add Nokogiri XMLRPCParser(cherry picked from commit 17e3ffc31e20e17285c36de615862b147577d16a)
History
#1
Updated by Cyril Rohr over 9 years ago
OK, finally found the issue: the REXMLStreamParser XMLRPC is using by default is really slow. Switching to a Nokogiri based stream parser speeds up onevm list and compute listing from the OCCI interface by a factor of 100:
With default REXML Stream Parser:
-bash-3.2$ time onevm list a
ID USER GROUP NAME STAT CPU MEM HOSTNAME TIME
2062 crohr users VM-2 prol 0 0K paradent-15.ren 00 00:06:18
2063 crohr users VM-5 prol 0 0K paradent-15.ren 00 00:06:18
2064 crohr users VM-3 prol 0 0K paradent-15.ren 00 00:06:18
2065 crohr users VM-9 prol 0 0K paradent-15.ren 00 00:06:18
2066 crohr users VM-8 prol 0 0K paradent-15.ren 00 00:06:18
2067 crohr users VM-0 prol 0 0K paradent-15.ren 00 00:06:18
2068 crohr users VM-4 prol 0 0K paradent-15.ren 00 00:06:18
2069 crohr users VM-6 prol 0 0K paradent-15.ren 00 00:06:18
2070 crohr users VM-1 prol 0 0K paradent-15.ren 00 00:06:18
2071 crohr users VM-7 prol 0 0K paradent-15.ren 00 00:06:18
2072 crohr users VM-10 prol 0 0K paradent-15.ren 00 00:06:16
2073 crohr users VM-14 prol 0 0K paradent-15.ren 00 00:06:15
2074 crohr users VM-13 prol 0 0K paradent-15.ren 00 00:06:15
2075 crohr users VM-11 prol 0 0K paradent-15.ren 00 00:06:15
2076 crohr users VM-12 prol 0 0K paradent-15.ren 00 00:06:15
2077 crohr users VM-15 prol 0 0K paradent-15.ren 00 00:06:14
2078 crohr users VM-17 prol 0 0K paradent-15.ren 00 00:06:14
2079 crohr users VM-19 prol 0 0K paradent-15.ren 00 00:06:14
2080 crohr users VM-16 prol 0 0K paradent-15.ren 00 00:06:14
2081 crohr users VM-18 prol 0 0K paradent-15.ren 00 00:06:14
2082 crohr users VM-20 prol 0 0K paradent-15.ren 00 00:06:12
2083 crohr users VM-21 prol 0 0K paradent-15.ren 00 00:06:12
2084 crohr users VM-22 prol 0 0K paradent-15.ren 00 00:06:12
2085 crohr users VM-23 prol 0 0K paradent-15.ren 00 00:06:12
2086 crohr users VM-24 prol 0 0K paradent-15.ren 00 00:06:12
2087 crohr users VM-25 prol 0 0K paradent-15.ren 00 00:06:12
2088 crohr users VM-26 prol 0 0K paradent-15.ren 00 00:06:10
2089 crohr users VM-29 prol 0 0K paradent-15.ren 00 00:06:10
2090 crohr users VM-27 prol 0 0K paradent-15.ren 00 00:06:10
2091 crohr users VM-28 prol 0 0K paradent-15.ren 00 00:06:10
2092 crohr users VM-31 prol 0 0K paradent-15.ren 00 00:06:09
2093 crohr users VM-30 prol 0 0K paradent-15.ren 00 00:06:09
2094 crohr users VM-35 pend 0 0K 00 00:06:08
2095 crohr users VM-33 pend 0 0K 00 00:06:08
2096 crohr users VM-32 pend 0 0K 00 00:06:08
2097 crohr users VM-34 pend 0 0K 00 00:06:08
2098 crohr users VM-36 pend 0 0K 00 00:06:07
2099 crohr users VM-37 pend 0 0K 00 00:06:07
2100 crohr users VM-38 pend 0 0K 00 00:06:07
2101 crohr users VM-39 pend 0 0K 00 00:06:07
2102 crohr users VM-41 pend 0 0K 00 00:06:06
2103 crohr users VM-40 pend 0 0K 00 00:06:06
2104 crohr users VM-42 pend 0 0K 00 00:06:05
2105 crohr users VM-43 pend 0 0K 00 00:06:04
2106 crohr users VM-44 pend 0 0K 00 00:06:04
2107 crohr users VM-45 pend 0 0K 00 00:06:04
2108 crohr users VM-48 pend 0 0K 00 00:06:04
2109 crohr users VM-47 pend 0 0K 00 00:06:03
2110 crohr users VM-49 pend 0 0K 00 00:06:03
2111 crohr users VM-46 pend 0 0K 00 00:06:03
2112 crohr users VM-50 pend 0 0K 00 00:06:03
2113 crohr users VM-51 pend 0 0K 00 00:06:03
2114 crohr users VM-52 pend 0 0K 00 00:06:02
2115 crohr users VM-53 pend 0 0K 00 00:06:01
2116 crohr users VM-54 pend 0 0K 00 00:06:01
2117 crohr users VM-55 pend 0 0K 00 00:06:01
2118 crohr users VM-56 pend 0 0K 00 00:06:01
2119 crohr users VM-59 pend 0 0K 00 00:06:00
2120 crohr users VM-58 pend 0 0K 00 00:06:00
2121 crohr users VM-57 pend 0 0K 00 00:06:00
2122 crohr users VM-60 pend 0 0K 00 00:06:00
2123 crohr users VM-61 pend 0 0K 00 00:06:00
2124 crohr users VM-62 pend 0 0K 00 00:05:59
2125 crohr users VM-63 pend 0 0K 00 00:05:59
2126 crohr users VM-64 pend 0 0K 00 00:05:58
2127 crohr users VM-65 pend 0 0K 00 00:05:58
2128 crohr users VM-66 pend 0 0K 00 00:05:58
2129 crohr users VM-67 pend 0 0K 00 00:05:56
2130 crohr users VM-70 pend 0 0K 00 00:05:56
2131 crohr users VM-69 pend 0 0K 00 00:05:56
2132 crohr users VM-71 pend 0 0K 00 00:05:56
2133 crohr users VM-68 pend 0 0K 00 00:05:56
2134 crohr users VM-72 pend 0 0K 00 00:05:55
2135 crohr users VM-73 pend 0 0K 00 00:05:55
2136 crohr users VM-74 pend 0 0K 00 00:05:54
2137 crohr users VM-76 pend 0 0K 00 00:05:54
2138 crohr users VM-75 pend 0 0K 00 00:05:54
2139 crohr users VM-79 pend 0 0K 00 00:05:53
2140 crohr users VM-78 pend 0 0K 00 00:05:53
2141 crohr users VM-81 pend 0 0K 00 00:05:53
2142 crohr users VM-77 pend 0 0K 00 00:05:53
2143 crohr users VM-82 pend 0 0K 00 00:05:52
2144 crohr users VM-83 pend 0 0K 00 00:05:52
2145 crohr users VM-80 pend 0 0K 00 00:05:52
2146 crohr users VM-86 pend 0 0K 00 00:05:51
2147 crohr users VM-84 pend 0 0K 00 00:05:51
2148 crohr users VM-85 pend 0 0K 00 00:05:50
2149 crohr users VM-87 pend 0 0K 00 00:05:49
2150 crohr users VM-89 pend 0 0K 00 00:05:49
2151 crohr users VM-88 pend 0 0K 00 00:05:49
2152 crohr users VM-90 pend 0 0K 00 00:05:49
2153 crohr users VM-92 pend 0 0K 00 00:05:48
2154 crohr users VM-91 pend 0 0K 00 00:05:48
2155 crohr users VM-93 pend 0 0K 00 00:05:48
2156 crohr users VM-94 pend 0 0K 00 00:05:47
2157 crohr users VM-95 pend 0 0K 00 00:05:47
2158 crohr users VM-96 pend 0 0K 00 00:05:47
2159 crohr users VM-97 pend 0 0K 00 00:05:47
2160 crohr users VM-98 pend 0 0K 00 00:05:47
2161 crohr users VM-99 pend 0 0K 00 00:05:47
2162 crohr users VM-100 pend 0 0K 00 00:05:46
2163 crohr users VM-102 pend 0 0K 00 00:05:45
2164 crohr users VM-103 pend 0 0K 00 00:05:44
2165 crohr users VM-101 pend 0 0K 00 00:05:44
2166 crohr users VM-104 pend 0 0K 00 00:05:44
2167 crohr users VM-106 pend 0 0K 00 00:05:44
2168 crohr users VM-105 pend 0 0K 00 00:05:44
2169 crohr users VM-107 pend 0 0K 00 00:05:43
2170 crohr users VM-108 pend 0 0K 00 00:05:43
2171 crohr users VM-109 pend 0 0K 00 00:05:43
2172 crohr users VM-110 pend 0 0K 00 00:05:43
2173 crohr users VM-111 pend 0 0K 00 00:05:42
2174 crohr users VM-112 pend 0 0K 00 00:05:41
2175 crohr users VM-113 pend 0 0K 00 00:05:41
2176 crohr users VM-114 pend 0 0K 00 00:05:41
2177 crohr users VM-115 pend 0 0K 00 00:05:41
2178 crohr users VM-116 pend 0 0K 00 00:05:41
2179 crohr users VM-117 pend 0 0K 00 00:05:41
2180 crohr users VM-118 pend 0 0K 00 00:05:41
2181 crohr users VM-119 pend 0 0K 00 00:05:39
2182 crohr users VM-120 pend 0 0K 00 00:05:39
2183 crohr users VM-121 pend 0 0K 00 00:05:39
2184 crohr users VM-122 pend 0 0K 00 00:05:38
2185 crohr users VM-123 pend 0 0K 00 00:05:37
2186 crohr users VM-124 pend 0 0K 00 00:05:37
2187 crohr users VM-128 pend 0 0K 00 00:05:37
2188 crohr users VM-127 pend 0 0K 00 00:05:37
2189 crohr users VM-126 pend 0 0K 00 00:05:37
2190 crohr users VM-125 pend 0 0K 00 00:05:37
2191 crohr users VM-129 pend 0 0K 00 00:05:36
2192 crohr users VM-130 pend 0 0K 00 00:05:36
2193 crohr users VM-131 pend 0 0K 00 00:05:36
2194 crohr users VM-132 pend 0 0K 00 00:05:35
2195 crohr users VM-138 pend 0 0K 00 00:05:33
2196 crohr users VM-133 pend 0 0K 00 00:05:33
2197 crohr users VM-136 pend 0 0K 00 00:05:33
2198 crohr users VM-137 pend 0 0K 00 00:05:33
2199 crohr users VM-134 pend 0 0K 00 00:05:33
2200 crohr users VM-135 pend 0 0K 00 00:05:33
2201 crohr users VM-139 pend 0 0K 00 00:05:33
2202 crohr users VM-141 pend 0 0K 00 00:05:33
2203 crohr users VM-140 pend 0 0K 00 00:05:32
2204 crohr users VM-142 pend 0 0K 00 00:05:32
2205 crohr users VM-144 pend 0 0K 00 00:05:30
2206 crohr users VM-143 pend 0 0K 00 00:05:30
2207 crohr users VM-145 pend 0 0K 00 00:05:30
2208 crohr users VM-149 pend 0 0K 00 00:05:29
2209 crohr users VM-148 pend 0 0K 00 00:05:29
2210 crohr users VM-146 pend 0 0K 00 00:05:29
2211 crohr users VM-147 pend 0 0K 00 00:05:29
real 0m58.317s
user 0m57.863s
sys 0m0.362s
crohr@parachute-wifi:~/bonfire-ssl $ time curl -K secure.curl https://frontend.integration.bonfire.grid5000.fr/compute <html> <head><title>502 Bad Gateway</title></head> <body bgcolor="white"> <center><h1>502 Bad Gateway</h1></center> <hr><center>nginx/0.8.55</center> </body> </html> real 0m30.737s user 0m0.026s sys 0m0.008s
With Nokogiri Stream Parser:
-bash-3.2$ time onevm list a
ID USER GROUP NAME STAT CPU MEM HOSTNAME TIME
2062 crohr users VM-2 prol 0 0K paradent-15.ren 00 00:04:35
2063 crohr users VM-5 prol 0 0K paradent-15.ren 00 00:04:35
2064 crohr users VM-3 prol 0 0K paradent-15.ren 00 00:04:35
2065 crohr users VM-9 prol 0 0K paradent-15.ren 00 00:04:35
2066 crohr users VM-8 prol 0 0K paradent-15.ren 00 00:04:35
2067 crohr users VM-0 prol 0 0K paradent-15.ren 00 00:04:35
2068 crohr users VM-4 prol 0 0K paradent-15.ren 00 00:04:35
2069 crohr users VM-6 prol 0 0K paradent-15.ren 00 00:04:35
2070 crohr users VM-1 prol 0 0K paradent-15.ren 00 00:04:35
2071 crohr users VM-7 prol 0 0K paradent-15.ren 00 00:04:35
2072 crohr users VM-10 prol 0 0K paradent-15.ren 00 00:04:33
2073 crohr users VM-14 prol 0 0K paradent-15.ren 00 00:04:32
2074 crohr users VM-13 prol 0 0K paradent-15.ren 00 00:04:32
2075 crohr users VM-11 prol 0 0K paradent-15.ren 00 00:04:32
2076 crohr users VM-12 prol 0 0K paradent-15.ren 00 00:04:32
2077 crohr users VM-15 prol 0 0K paradent-15.ren 00 00:04:31
2078 crohr users VM-17 prol 0 0K paradent-15.ren 00 00:04:31
2079 crohr users VM-19 prol 0 0K paradent-15.ren 00 00:04:31
2080 crohr users VM-16 prol 0 0K paradent-15.ren 00 00:04:31
2081 crohr users VM-18 prol 0 0K paradent-15.ren 00 00:04:31
2082 crohr users VM-20 prol 0 0K paradent-15.ren 00 00:04:29
2083 crohr users VM-21 prol 0 0K paradent-15.ren 00 00:04:29
2084 crohr users VM-22 prol 0 0K paradent-15.ren 00 00:04:29
2085 crohr users VM-23 prol 0 0K paradent-15.ren 00 00:04:29
2086 crohr users VM-24 prol 0 0K paradent-15.ren 00 00:04:29
2087 crohr users VM-25 prol 0 0K paradent-15.ren 00 00:04:29
2088 crohr users VM-26 prol 0 0K paradent-15.ren 00 00:04:27
2089 crohr users VM-29 prol 0 0K paradent-15.ren 00 00:04:27
2090 crohr users VM-27 prol 0 0K paradent-15.ren 00 00:04:27
2091 crohr users VM-28 prol 0 0K paradent-15.ren 00 00:04:27
2092 crohr users VM-31 prol 0 0K paradent-15.ren 00 00:04:26
2093 crohr users VM-30 prol 0 0K paradent-15.ren 00 00:04:26
2094 crohr users VM-35 pend 0 0K 00 00:04:25
2095 crohr users VM-33 pend 0 0K 00 00:04:25
2096 crohr users VM-32 pend 0 0K 00 00:04:25
2097 crohr users VM-34 pend 0 0K 00 00:04:25
2098 crohr users VM-36 pend 0 0K 00 00:04:24
2099 crohr users VM-37 pend 0 0K 00 00:04:24
2100 crohr users VM-38 pend 0 0K 00 00:04:24
2101 crohr users VM-39 pend 0 0K 00 00:04:24
2102 crohr users VM-41 pend 0 0K 00 00:04:23
2103 crohr users VM-40 pend 0 0K 00 00:04:23
2104 crohr users VM-42 pend 0 0K 00 00:04:22
2105 crohr users VM-43 pend 0 0K 00 00:04:21
2106 crohr users VM-44 pend 0 0K 00 00:04:21
2107 crohr users VM-45 pend 0 0K 00 00:04:21
2108 crohr users VM-48 pend 0 0K 00 00:04:21
2109 crohr users VM-47 pend 0 0K 00 00:04:20
2110 crohr users VM-49 pend 0 0K 00 00:04:20
2111 crohr users VM-46 pend 0 0K 00 00:04:20
2112 crohr users VM-50 pend 0 0K 00 00:04:20
2113 crohr users VM-51 pend 0 0K 00 00:04:20
2114 crohr users VM-52 pend 0 0K 00 00:04:19
2115 crohr users VM-53 pend 0 0K 00 00:04:18
2116 crohr users VM-54 pend 0 0K 00 00:04:18
2117 crohr users VM-55 pend 0 0K 00 00:04:18
2118 crohr users VM-56 pend 0 0K 00 00:04:18
2119 crohr users VM-59 pend 0 0K 00 00:04:17
2120 crohr users VM-58 pend 0 0K 00 00:04:17
2121 crohr users VM-57 pend 0 0K 00 00:04:17
2122 crohr users VM-60 pend 0 0K 00 00:04:17
2123 crohr users VM-61 pend 0 0K 00 00:04:17
2124 crohr users VM-62 pend 0 0K 00 00:04:16
2125 crohr users VM-63 pend 0 0K 00 00:04:16
2126 crohr users VM-64 pend 0 0K 00 00:04:15
2127 crohr users VM-65 pend 0 0K 00 00:04:15
2128 crohr users VM-66 pend 0 0K 00 00:04:15
2129 crohr users VM-67 pend 0 0K 00 00:04:13
2130 crohr users VM-70 pend 0 0K 00 00:04:13
2131 crohr users VM-69 pend 0 0K 00 00:04:13
2132 crohr users VM-71 pend 0 0K 00 00:04:13
2133 crohr users VM-68 pend 0 0K 00 00:04:13
2134 crohr users VM-72 pend 0 0K 00 00:04:12
2135 crohr users VM-73 pend 0 0K 00 00:04:12
2136 crohr users VM-74 pend 0 0K 00 00:04:11
2137 crohr users VM-76 pend 0 0K 00 00:04:11
2138 crohr users VM-75 pend 0 0K 00 00:04:11
2139 crohr users VM-79 pend 0 0K 00 00:04:10
2140 crohr users VM-78 pend 0 0K 00 00:04:10
2141 crohr users VM-81 pend 0 0K 00 00:04:10
2142 crohr users VM-77 pend 0 0K 00 00:04:10
2143 crohr users VM-82 pend 0 0K 00 00:04:09
2144 crohr users VM-83 pend 0 0K 00 00:04:09
2145 crohr users VM-80 pend 0 0K 00 00:04:09
2146 crohr users VM-86 pend 0 0K 00 00:04:08
2147 crohr users VM-84 pend 0 0K 00 00:04:08
2148 crohr users VM-85 pend 0 0K 00 00:04:07
2149 crohr users VM-87 pend 0 0K 00 00:04:06
2150 crohr users VM-89 pend 0 0K 00 00:04:06
2151 crohr users VM-88 pend 0 0K 00 00:04:06
2152 crohr users VM-90 pend 0 0K 00 00:04:06
2153 crohr users VM-92 pend 0 0K 00 00:04:05
2154 crohr users VM-91 pend 0 0K 00 00:04:05
2155 crohr users VM-93 pend 0 0K 00 00:04:05
2156 crohr users VM-94 pend 0 0K 00 00:04:04
2157 crohr users VM-95 pend 0 0K 00 00:04:04
2158 crohr users VM-96 pend 0 0K 00 00:04:04
2159 crohr users VM-97 pend 0 0K 00 00:04:04
2160 crohr users VM-98 pend 0 0K 00 00:04:04
2161 crohr users VM-99 pend 0 0K 00 00:04:04
2162 crohr users VM-100 pend 0 0K 00 00:04:03
2163 crohr users VM-102 pend 0 0K 00 00:04:02
2164 crohr users VM-103 pend 0 0K 00 00:04:01
2165 crohr users VM-101 pend 0 0K 00 00:04:01
2166 crohr users VM-104 pend 0 0K 00 00:04:01
2167 crohr users VM-106 pend 0 0K 00 00:04:01
2168 crohr users VM-105 pend 0 0K 00 00:04:01
2169 crohr users VM-107 pend 0 0K 00 00:04:00
2170 crohr users VM-108 pend 0 0K 00 00:04:00
2171 crohr users VM-109 pend 0 0K 00 00:04:00
2172 crohr users VM-110 pend 0 0K 00 00:04:00
2173 crohr users VM-111 pend 0 0K 00 00:03:59
2174 crohr users VM-112 pend 0 0K 00 00:03:58
2175 crohr users VM-113 pend 0 0K 00 00:03:58
2176 crohr users VM-114 pend 0 0K 00 00:03:58
2177 crohr users VM-115 pend 0 0K 00 00:03:58
2178 crohr users VM-116 pend 0 0K 00 00:03:58
2179 crohr users VM-117 pend 0 0K 00 00:03:58
2180 crohr users VM-118 pend 0 0K 00 00:03:58
2181 crohr users VM-119 pend 0 0K 00 00:03:56
2182 crohr users VM-120 pend 0 0K 00 00:03:56
2183 crohr users VM-121 pend 0 0K 00 00:03:56
2184 crohr users VM-122 pend 0 0K 00 00:03:55
2185 crohr users VM-123 pend 0 0K 00 00:03:54
2186 crohr users VM-124 pend 0 0K 00 00:03:54
2187 crohr users VM-128 pend 0 0K 00 00:03:54
2188 crohr users VM-127 pend 0 0K 00 00:03:54
2189 crohr users VM-126 pend 0 0K 00 00:03:54
2190 crohr users VM-125 pend 0 0K 00 00:03:54
2191 crohr users VM-129 pend 0 0K 00 00:03:53
2192 crohr users VM-130 pend 0 0K 00 00:03:53
2193 crohr users VM-131 pend 0 0K 00 00:03:53
2194 crohr users VM-132 pend 0 0K 00 00:03:52
2195 crohr users VM-138 pend 0 0K 00 00:03:50
2196 crohr users VM-133 pend 0 0K 00 00:03:50
2197 crohr users VM-136 pend 0 0K 00 00:03:50
2198 crohr users VM-137 pend 0 0K 00 00:03:50
2199 crohr users VM-134 pend 0 0K 00 00:03:50
2200 crohr users VM-135 pend 0 0K 00 00:03:50
2201 crohr users VM-139 pend 0 0K 00 00:03:50
2202 crohr users VM-141 pend 0 0K 00 00:03:50
2203 crohr users VM-140 pend 0 0K 00 00:03:49
2204 crohr users VM-142 pend 0 0K 00 00:03:49
2205 crohr users VM-144 pend 0 0K 00 00:03:47
2206 crohr users VM-143 pend 0 0K 00 00:03:47
2207 crohr users VM-145 pend 0 0K 00 00:03:47
2208 crohr users VM-149 pend 0 0K 00 00:03:46
2209 crohr users VM-148 pend 0 0K 00 00:03:46
2210 crohr users VM-146 pend 0 0K 00 00:03:46
2211 crohr users VM-147 pend 0 0K 00 00:03:46
real 0m0.853s
user 0m0.590s
sys 0m0.182s
crohr@parachute-wifi:~/bonfire-ssl $ time curl -I -K secure.curl https://frontend.integration.bonfire.grid5000.fr/compute HTTP/1.1 200 OK Server: nginx/0.8.55 Date: Fri, 02 Dec 2011 13:41:29 GMT Content-Type: text/html;charset=utf-8 Connection: keep-alive X-Frame-Options: sameorigin X-XSS-Protection: 1; mode=block Content-Length: 14488 real 0m0.572s user 0m0.026s sys 0m0.007s
So: major speedup (before the change the OCCI request failed because of our nginx timeout setting). I'll send a patch ASAP.
#2
Updated by Cyril Rohr over 9 years ago
The patch is here: http://flag.io/33998d87be6c9f8f3a6b0ae72c65fbb9.diff (to apply on path/to/lib/one/ruby/OpenNebula.rb -- version 3.0.0-bonfire). For reference, here is the full patch content:
85a86,92 > NOKOGIRI_PARSER = begin > require 'stream_parser_mixin' > true > rescue LoadError > false > end > 132c139,141 < if XMLPARSER --- > if NOKOGIRI_PARSER > @server.set_parser(XMLRPC::XMLParser::NokogiriStreamParser.new) > elsif XMLPARSER 134a144 >
This patch will trigger the use of the Nokogiri Stream Parser if the xmlrpc-streaming Ruby gem (https://github.com/washu/xmlrpc-streaming) is installed on the machine.
#3
Updated by Cyril Rohr over 9 years ago
Just putting here the full command to update a ONE installation until the patch is included upstream:
gem install xmlrpc-streaming nokogiri && curl http://flag.io/33998d87be6c9f8f3a6b0ae72c65fbb9 | patch /usr/lib/one/ruby/OpenNebula.rb -
#4
Updated by Tino Vázquez over 9 years ago
Thanks for the patch Cyril!
I want to point out that this is useful for ruby >= 1.9, what we recommend for other ruby versions is xmlparser (http://rubygems.org/gems/xmlparser), which is installed by default with the provided install_gems (http://opennebula.org/documentation:rel3.0:ignc#ruby_libraries_requirements_front-end).
We will evaluate this solution, it is promising to offer a solution for all ruby versions.
#5
Updated by Javi Fontan over 9 years ago
There's a bun in install_gems and xmlparser is only installed for ruby versions lesser than 1.8.7. To use this gem you can install it manually, the requirements are expat development libraries.
We will check your patch and try to use the most compatible solution for the next version.
#6
Updated by Daniel Molina over 9 years ago
- Status changed from New to Closed
- Target version set to Release 3.4 - S0
- % Done changed from 0 to 100
- Resolution set to fixed
- Affected Versions OpenNebula 3.2 added
The performance issue should be fixed with this patch which includes a new Nokogiri parser. If Nokogiri is installed it will be used instead of the REXML parser.