Bug #2586: Optimize 'onedb' command (upgrade & fsck) - OpenNebula - OpenNebula Development pages

I found those changes in one-4.4 branch breaking stable release of OpenNebula 4.4.1.
By using Nokogiry::XML all user_pool and group_pool body columns are rewrote to indented presentation of XML and breaks other parts in opennebula core and user cli utilities. For e.g. usually body in user_pool is represented as one long line (except if data inside have line breaks):

<USER><ID>34</ID><GID>0</GID><GROUPS><ID>0</ID><ID>1</ID></GROUPS><GNAME>oneadmin</GNAME>...</USER>

but now they look like:

<USER>
  <ID>34</ID>
  <GID>0</GID>
  <GROUPS>
    <ID>0</ID>
    <ID>1</ID>
  </GROUPS>
  <GNAME>oneadmin</GNAME>
  ...
</USER>

#5 Updated by Rolandas Naujikas about 7 years ago

File patch-fsck.rb added

I found a solution for fsck.rb. We can use

doc.root.xpath('//text()[not(normalize-space())]').remove

to remove all formatting space (I'm not sure about space in values) see http://stackoverflow.com/questions/1155293/removing-anything-between-xml-tags-and-their-content
After we can put xml in one line with

:body       => doc.root.serialize(:save_with => 0),

see http://stackoverflow.com/questions/4611403/need-to-remove-newlines-from-object-embed-tags-only-using-nokogiri

Regards

#6 Updated by Ruben S. Montero about 7 years ago

Tracker changed from Feature to Bug
Status changed from Closed to New
Resolution deleted (~~fixed~~)
Affected Versions Development added

Hi Rolands, Thanks for your detailed feedback!. I've moved this to bug tracker and assign to 4.6, so we can include this before the final release of 4.6

#7 Updated by Carlos Martín about 7 years ago

Hi Rolandas, could you please tell us where was OpenNebula crashing because of the xml formatting?
thanks!

#8 Updated by Carlos Martín about 7 years ago

I guess the problem was that this fix c026120a wasn't cherry-picked to one-4.4. This problem should not happen in the 4.6 branch.

Thanks a lot for your patch. I removed the xpath, because if I understood it right it only removes ATTRIBUTE = " ", which should be valid in OpenNebula. Did I miss any other reason why it was needed?

The documentation is also updated with instructions to update the fsck file from the version in the repo.

Best regards.

#9 Updated by Rolandas Naujikas about 7 years ago

Carlos Martín wrote:

Hi Rolandas, could you please tell us where was OpenNebula crashing because of the xml formatting?
thanks!

When user_pool/group_pool body contains pretty XML formatting (after bug in onedb fsck), then there were many strange errors related to ACLs and permission denied. oneuser show crashing on secondary groups. Strange permission problems. After all spaces and newlines were removed - problems disappeared.

Rolandas

#10 Updated by Rolandas Naujikas about 7 years ago

Carlos Martín wrote:

I guess the problem was that this fix c026120a wasn't cherry-picked to one-4.4. This problem should not happen in the 4.6 branch.

Thanks a lot for your patch. I removed the xpath, because if I understood it right it only removes ATTRIBUTE = " ", which should be valid in OpenNebula. Did I miss any other reason why it was needed?

That was quick way to correct XML formating, but not good in general and not needed if you don't hit bug in onedb fsck. Nokogiri is preserving original XML formatting with right .serialize() call.

The documentation is also updated with instructions to update the fsck file from the version in the repo.

Best regards.

#11 Updated by Carlos Martín about 7 years ago

Related to Backlog #2845: Improve fsck performance added

#12 Updated by Carlos Martín about 7 years ago

Status changed from New to Closed
Resolution set to fixed

I think this ticket is ready for the 4.6 release.

Rolandas, I decided to fix the empty lines in Nokogiri using this commit 45d60eb8. Although it wasn't strictly necessary, since master and release-4.4.1 were working fine, the problem was only in the one-4.4 head. Anyway, thank you for your feedback.

Also available in: Atom PDF

OpenNebula

Issues

Custom queries

Bug #2586

Optimize 'onedb' command (upgrade & fsck)

Associated revisions

History

#1 Updated by Ruben S. Montero over 7 years ago

#2 Updated by Jaime Melis over 7 years ago

#3 Updated by Ruben S. Montero over 7 years ago

#4 Updated by Rolandas Naujikas about 7 years ago