Bug #4088

Pagination issue with info calls on pools

Added by Boris Parak over 5 years ago. Updated over 5 years ago.

Status:ClosedStart date:10/25/2015
Priority:NormalDue date:
Assignee:Ruben S. Montero% Done:

0%

Category:Client API & Library
Target version:Release 4.14.2
Resolution:fixed Pull request:
Affected Versions:OpenNebula 4.12, OpenNebula 4.14

Description

Pagination uses the following 'info' arguments

[user_id, start_id, end_id, state]

to retrieve objects with ID between 'start_id' and 'end_id'. If the underlying pool is continuous (no IDs missing), this works well and I can iterate over the entire pool. However, if there is a gap in the pool (sets of IDs missing), this stops working as soon as the loop encounters the first gap wider than the page size. It will return an empty batch and terminate the loop (=> there are no more objects to retrieve).

This is an issue on long-running OpenNebula sites where the administrators (rather unwisely) attempted to clean up the database by dropping old records manually ... therefore creating large gaps in the VM pool. There is no way to get the last issued object ID on the given pool so I cannot use that to keep iterating over gaps until 'end_id' > 'last_id'.

Is there a way to work around this? Could you add a new call to the next version of the API allowing the retrieval of the last issued object ID for the given pool?

History

#1 Updated by Ruben S. Montero over 5 years ago

  • Status changed from Pending to New

THANKS for the feedback!

#2 Updated by Ruben S. Montero over 5 years ago

  • Target version changed from Release 5.0 to Release 4.14.2

#3 Updated by Ruben S. Montero over 5 years ago

  • Category changed from Core & System to Client API & Library
  • Assignee set to Ruben S. Montero

#4 Updated by Ruben S. Montero over 5 years ago

  • Status changed from New to Closed
  • Resolution set to fixed
  • Affected Versions OpenNebula 4.12 added

Hi Boris,

It seems that there was a bug in the SAX parser. I've tested this with gaps greater than the page size and it works.

After re-reading your email, I think you may be referring to the core API call, that just needs a number of rows from a given offset. If that's the issue you need to implement a logic like the following in your client:

   while true                                                                       
       a=@client.call("#{@pool_name.delete('_').downcase}.info",                    
                      @user_id, current, -size, -1)                                            
        return a if OpenNebula.is_error?(a)

        a_array=parser.parse(a)

        array   += a_array
        current += size

        break if !a || a_array.length<size                                           
    end

I'm closing this issue, we can reopen it if needed.

BTW if the SAX parser was your issue, just apply the patch in this commit to the lib file

Also available in: Atom PDF