Parallel Repeat

An updated version with concurrency option (thanks to @Apple for the idea and example, have a bit different implementation)

experimental.zip (857 Bytes)

1 Like

Hi @patrick

I've placed the experimental.js file in lib/modules, and renamed module from core to experimental but it's just displaying as an unknown action in the UI:

Screenshot 2024-11-20 at 12.30.42 PM

I am on Wappler 7 latest beta if that may be causing some issues?

It's shown as Unknown because Wappler doesn't know about the experimental module, the action will work correctly.

1 Like

Hey Patrick, what are all the rules for using 'parallel' and 'parallel + repeat (experimental)' in Wappler? I have several cases where I need to execute the function 'lib/my-function', but 'parallel' doesn't seem to execute it correctly.

What is the issue you have with the parallel action? With parallel all actions run simultaneous, and you can't access any result of another action that runs in parallel, that is a limitation that you get by running in parallel. The experimental repeat is a variant on the standard repeat with the difference that it runs some steps in parallel.

1 Like

I understand, the current execution was of the 'nodejs' functions using the wappler library. When I did this, the parallel execution does not work correctly, executing the functions in parallel. Instead of executing in parallel, it is executed asynchronously (default nodejs). When I changed the api logic to directly use API/Database calls (insert/update/read) the parallel execution is done correctly.

View this example 'succeeded parallel action'. 'Without 'exec (asyncAuditLog)''
image

Traces with succeeded parallel actions:

So look at this 'parallel execution failure' example. 'With 'exec (asyncAuditLog)''
image

Traces without succeeded parallel actions:

The steps inside the audit-logs will not run in parallel, they will run sequential but the database and api action step should run parallel. If you want to run the steps in the audit-logs also in parallel you have to put a parallel step in that server action.

Also not all actions are async and will not run in parallel, it works best with actions like for database, api and filesystem. Parsing of large datasets will block the thread and will not work in parallel.

It's strange — I added the 'exec' action in parallel, but after its execution, the other actions no longer run in parallel.

Has Parallel Repeat become available in the stable v7.x.x.?

If it’s available, how do I use it? I have an API with 4 nested repeat steps. Can I just place the complete block of nested repeats inside the Parallel step?

An example of how this can be used to speed up execution of queries would be very helpful.

Can you show a screenshot of what you're doing? Only the outermost steps will run in parallel. If you have nested stuff it will run sequentially inside the outermost parallel iteration

Hi Apple,

The screenshot below is example of an API with multiple repeats and nested repeats. How will Parallel Repeat work in this example?

You would need to use Group steps as the outermost steps, and inside each DB query + Repeat. Each Group would run in parallel and execute the steps inside like in their own “lane”

So the API steps will look like this? Is it different to using Parallel as an outermost group set?

<Group1>
  <Repeat 1 + DB>
</Group1>
<Group2>
  <Repeat 2 + DB>
</Group2>

Yes, that's the structure. If you didn't use the Group steps the DB query and Repeat would run in parallel - the Repeat wouldn't wait for the DB query to complete, so you would Repeat on empty data