Whats the best way to handle this

Wappler General How To
#1

I have an API I am getting data from, and as usual it has a limit, this time 250 per call however I do get some data to help repeat through.
This is what that data looks like

data": {
      "totalSize": 3289,
      "done": false,
      "nextRecordsUrl": "/services/data/v53.0/query/01g3K123456KqzJMQAZ-250",
      "records": Array[250][
        {...}
      ]
}

I really do not want to have to duplicate all my steps 14 times to get to the final 3289 so I can add all the records to my internal database.

totalSize is the same throughout
done changes to true when there is no nextRecordsUrl
nextRecordsUrl ends with -250 and the entire code before that remains the same for every page

Surely there must be a smart way to repeat through all these

Currently my steps look like this
2021-10-02_00-21-48

Pretty simple, call the API, repeat through each record of the Array, check if the record already exists in the database, if it does update it, if it doesn’t insert it.

Anyone got any bright maths ideas

#2

I think you’d be best off using a while loop. Add a set value that’s updated when done is true and use that as the basis of the ‘while’.

I found the while loop difficult to use with the ui, it works perfectly manually adding the code. You just can’t see the fields in the data pickers (I’ve not revisited since so it may have been updated).

This post should show some relevant info:

2 Likes
#3

Yep, a while loop is exactly what you need. But also, consider the overall execution time and make sure you are well within your timeouts.

1 Like
#4

Thanks guys, that worked perfectly.

2021-10-02_08-39-10
2021-10-02_08-39-10674×608 45.7 KB

I created a Set Value step first of getNextRecordsUrl (local & global) and manually gave it the first starting URL for the first API call that initiates the process.

Then a While loop with a condition of the Set Value step, meaning if it has no value then its going to stop.

In the While loop I placed my API Action, with a URL of the Set Value step, then a second Set Value step to feed the first Global Set Value with a new URL.

When all the records were done, the last run had no new URL, so it stopped.

1 Like