How to involve multiple cores in processing one heavy backend action? (NodeJS)

[nickneustroev]

Hi, everyone!
Yet again, I am looking for some advice about performance improvement.

I know that Wappler has a great build-in option for load-balancing using Docker Swarm. You just set a number of replicas, and it works.

But is it possible to do the similar with one backend action?
I just had a long operation that one user (administrator) starts sometimes and I want to speed it up by utilizing the multiple cores.

Or another option, I can try to cut this operation in several parts and start those parts independently from frontend. (still on the behalf of one user) So it goes to different containers.
But as I understand, that would not be fully reliable because Traefik routes requests based on round robin, and not current load level of nodes.

So, in what direction should I approach?

[sid]

@nickneustroev

You just set a number of replicas, and it works.

Don’t have any advice for the question here… but a question of my own.
This replica option - do you know if it also replicates the scheduler SAs?

[tbvgl]

@nickneustroev I split these kind of tasks by assigning them to bull queues. Then I set the limit for each queue to one concurrent job. I have worker containers which then pick up the iobs from these queues. So 10 worker containers would process 10 jobs at the same time and never process the same job. That way a second iob is only taken by each container when the previous job is done and there are no load issues.

@sid yes, replicas replicate the scheduler as well. So if you have a job running every 5 minutes and two replicas then the iob will run twice every 5 minutes.

[sid]

Thanks for confirming.

I was going to propose Bull Queue (we recently used it in a project), but I am not sure if it utilizes multiple-physical-cpu-cores or not. And the question was more around passing parts of the task to other Docker instances - which did not make sense to me.

[tbvgl]

Bull utilizes a single core but bull combined with replicas and a bull worker/manager application configuration works very well for multi threaded/multi core/multi instance tasks while it allows you to fine tune how many jobs each container/instance takes etc.

[nickneustroev]

Hi, @sid!
Yes, I am aware of that. There is a FR I started with the intention of this being altered. Make the scheduler work with many replicas

I wasn’t thinking about using it to my advantage here, but now when you mentioned it, I think it maybe can help me.
But FR is already assigned, meaning this behavior is gonna be changed in the future. So I better build my solution around another approach.

And the question was more around passing parts of the task to other Docker instances - which did not make sense to me.

My bad, I may have sounded silly here. The main goal here is to speed up the processing of a big operation. Operation itself could be cut in parts or paralleled. But I don’t know which possible ways are available here for this so I may have assumed weird things.

.

@tbvgl, thank you for your suggestions and detailed explanation! It looks like bull queues are indeed the right way to handle this.
A little downside for me though that this requires some programming.

I was also told to look into the Worker threads. https://nodejs.org/api/worker_threads.html#worker-threads

[tbvgl]

@nickneustroev the bull solution doesn’t require any coding. You can just use the bull extension Ken and I worked on and then do everything via UI.

[Apple]

What does this “long operation” do? Are these database queries? Data processing?

If it’s data processing, you could speed it up by using raw JS (requires coding) instead of Wappler steps

[nickneustroev]

@tbvgl

You can just use the bull extension Ken and I worked on and then do everything via UI.

Wow, indeed. I just haven’t paid attention before. I will check it out, much appreciated!

@Apple

What does this “long operation” do? Are these database queries? Data processing?
If it’s data processing, you could speed it up by using raw JS (requires coding) instead of Wappler steps

Database queries mostly. But I would keep in mind you tip anyway, thanks!

[nickneustroev]

Hi, Tobias! @tbvgl

I was distracted from this task for a couple of months. But now I come back to it and try to implement bull queues in my project.

And the first results are awesome!
My test action that includes processing 7000 of records speed up almost 10 times: from 45 seconds to only 5.

Only this improvement on its own helps me incredibly. It basically solves my issue.
So first of all I wanna thank you for your work here! That’s amazing.

But I’m interested, if I can push it further.
Because it seems like my current setup still does not utilize multiple cores or multiple instances.
I have tried to change those parameters but it doesn’t affect the speed of processing – it is still at 5 seconds no matter what.

Currently I have 4 app replicas and 6 CPUs.
My Create Queue step parameters like this:

My Add Job step parameters are default.

Can you please give me advice here, what should I check or change?

[tbvgl]

Hi @nickneustroev

It’s good to hear the extension is helping you!

The queue processor is defined as a sandboxed processor, which is good for utilizing multiple cores, but you need to increase the concurrency. The concurrency uses one new thread for each increment. So if you have four replicas and concurrency set to 5, you will process 20 jobs simultaneously and up to 5 cores on each replica.

Your current configuration should run four concurrent jobs, one on each replica.

Best experiment with how much concurrency your cores can handle. I have configurations where I process 1 million records in 300ms.

Your queue is set to autostart. So once you made the changes, run the create queue action once and restart your replicas or the full app so they can pick up the new config. You can also edit the configuration in Redis directly after making the changes in the UI if you are comfortable with that.

[nickneustroev]

Thank you for the feedback!

According to your explanations I assume I have a correct understanding of how it works.

But it still seems like an issue here.
Because when I increase replicas and cores number twice or more , it doesn’t change the overall processing time.

But interesting, that if I increase the “Number of concurrent jobs”, that on the contrary decreases the speed of processing. This is strange.

My one suggestion is that autostart somehow doesn’t apply.
Maybe I am missing something in this part “So once you made the changes, run the create queue action once and restart your replicas or the full app so they can pick up the new config”.
Or later, when you say “You can also edit the configuration in Redis directly after making the changes in the UI…”. Tbh I don’t understand why I’m gonna need that.

[tbvgl]

When you enable the autostart option for a queue, my extension stores the configuration in Redis to look up the config when the app starts.

The key format is: autostartqueues:queuename so if your queue name is answersQueues then the config is stored under the key autostartqueues:answersQueues.

The data stored in Redis looks like this:

To verify your queues are starting when the app starts, best enable logging. You can do this by adding the following ENV variables:

PRETTY_LOG=enabled
LOG_LEVEL=debug

Then when your app starts, you should see something like this:

The log contains the queue config so you can verify the concurrency.

When you change the queue settings, the config data in Redis must be updated. That’s done by running the create queue action. Restarting your app will then pick up the new config.

I recommend decreasing your replicas to 1 and adjusting the concurrency to find a good value. Then you can increase the replica count.

[nickneustroev]

Got it! Thank you for the details.
I am gonna investigate.

[nickneustroev]

Hi, Tobias!

I have figured out my issue.
Turns out, I was just a bit silly last time.
I wrote “I have tried to change those parameters but it doesn’t affect the speed of processing – it is still at 5 seconds no matter what.”
In fact, I was looking at the end of the server action. I didn’t consider that jobs are still running in the background. I know, so lame.
Sorry for all the confusion.

Now I see that queues indeed get a boost when increasing the number of replicas, or concurrent jobs, or CPU.
I have run a lot of tests of my case and found the best config, for me it has 4 concurrent jobs, 8 replicas, 10 CPU.
Further upgrades don’t make changes. I assume the bottleneck is Database now.
Anyway, it made my processing 3-4 times faster than regular processing without queues.
So once again: thank you!
It helped me a lot, and this extension now stays in my tools list.