I have a workflow that repeats data from the database and then on each row calls an external API that can take 1 - 2 seconds to respond. This is causing some timeout issues when I want to handle more records at once.
It looks lie the Parallel module won’t help here, as it can’t handle working with an array. I’m wondering how everyone here handles running multiple actions/api calls etc. in parallel with Wappler?
This will drastically improve the speed and performance of 3 core api’s in my new project.
Thanks @kfawcett unfortunatelly that custom extension in particular has too many bugs/issues with it to be confident in using it. BQ’s are were I headed initially, perhaps i’ll be forced to build my own extension on the package if no other solution presents itself.
Adding a working api (that works 100% perfectly using Wappler’s scheduler) to a job will in 90% of cases just sit at delay, and then eventually fail or just sit on delay.
Recurring jobs don’t seem to work at all.
{status: “500”, message: “jobState is not defined”,…} error, when the job is clearly defined in the UI. This happens across most of the modules/functions provided in the full extension.
Seems to be some caching issues (as discussed in release post).
Incorrect, as the job should not be delayed, it is an active job that should be run immediately as there are no competing jobs or workers and is defaulting to ‘delay’ and just gets stuck there. The ‘action’ works standalone, or in a scheduler with Wappler but not with BullQ’s. I’ve attempted to add as a single job to a q, and as a recurring job.
Nope, not for me during during my testing.
Yeah, so it’s a bug.
I am unable to run simple test ‘jobs’ which is the core purpose of the extension, right? I guess I’ll leave the interpretation of ‘major’ bug up to others.
I’m familiar with the documentation and how the bullq system should work, thank you, and I also have my trusty ChatGPT to dumb things down for me.
I’m not too interested in getting into a discussion around it, my point was that I have been unable to use this solution due to he bugs that I found, which is unfortunate due to the amount of time and effort that no doubt went into the initial build. I’m glad others have found it to be more reliable and useful as that’s good for the community.
It’s been a while, but from memory I was with every job creating the queu again, as if the queue is created already it will just ignore that and add it. I played with both autostart and non autostart.
The implementation followed the guidance from the documentation - I mainly had issues with jobs being added, and then just getting stuck at delay, with no reason for it - that plus some caching issues. I could work around everything else - I just needed to reliably create queues and add jobs with pre-existing working actions.
It’s clearly the right solution so i’ll have to return to it next week and either build fresh, or, try to make improvements to get this working. I appreciate your responses though!
I don’t know about the caching issues as I don’t use that, but I’m sure you can make it work as I’m using it reliably in a lot of server actions, good luck!
Just a tip to get you going:
Your 1. and 2. issues both sound like there was no worker attached. So make sure a worker is attached when adding the jobs.