Hi, @sid!
Yes, I am aware of that. There is a FR I started with the intention of this being altered. Make the scheduler work with many replicas
I wasn’t thinking about using it to my advantage here, but now when you mentioned it, I think it maybe can help me.
But FR is already assigned, meaning this behavior is gonna be changed in the future. So I better build my solution around another approach.
And the question was more around passing parts of the task to other Docker instances - which did not make sense to me.
My bad, I may have sounded silly here. The main goal here is to speed up the processing of a big operation. Operation itself could be cut in parts or paralleled. But I don’t know which possible ways are available here for this so I may have assumed weird things.
.
@tbvgl, thank you for your suggestions and detailed explanation! It looks like bull queues are indeed the right way to handle this.
A little downside for me though that this requires some programming.
I was also told to look into the Worker threads. Worker threads | Node.js v22.1.0 Documentation