Wappler currently doesn’t provide hashing mechanisms for nodejs. Since I believe there’s no support for async formatters, you will have to run your argon/bcrypt and what have you synchronously.
Unless you got 10 users on different timezones, that’s simply unacceptable for any nodejs because it will block every other http request while the CPU is running the hashing in the event loop.
This is a little concerning to hear!
Not sure why the Wappler team changed my title from “Wappler cannot scale nodejs apps”. It’s a valid concern that should be addressed by the team that’s why I started the topic.
This is not strictly related to async formatters though it’s one example, but everything that Wappler is executing on the backend that could potentially block the event loop. Other examples include reading files, saving files, and all the server actions.
@Teodor care to explain your edit of the title? This affects how we structure our apps from the get go so I think it’s useful to kick off a discussion.
Your title saying Wappler cannot scale NodeJS apps was not really clear and it is not correct.
You are talking about bcrypt/argon and async formatters which are not available for NodeJS.
Please check the following discussion about async formatters and the replies posted by Patrick and George:
@Teodor Support for async formatters is just one example to kick start the discussion. I am talking about the whole Node.js server model so that includes all the server actions and DB actions that Wappler is running. Therefore, I think my title is fair.
None of these are performed with formatters. The standard formatters use so little CPU that main thread blocking is probably negligible.
Of course, if you create an upload action with a custom formatter instead of a custom module(these are async) then you are misusing the framework.
Hashing algorithms are indeed concerning though.
@JonL This affects your modules as well. If they are blocking the event loop and not running on worker pools, Wappler will never scale.
Once again I am not talking about a formatter or bcrypt. I am talking about Wappler nodejs model.
Please read: https://nodejs.org/ja/docs/guides/dont-block-the-event-loop/
As mentioned previously modules are async.
@JonL nonsense. It’s sync. Just because you stick a async in front of your exports doesn’t make Wappler async. They are calling sync methods all over the place. A quick folder search will tell you all there is:
That's even before getting into the fact that your awaits also run in the event loop, and you need workers to make it scalable. I mean this is straight from the Node official website
Suppose your server must read files in order to handle some client requests. After consulting the Node.js File system APIs, you opted to use fs.readFile() for simplicity. However, fs.readFile() is (currently) not partitioned: it submits a single fs.read() Task spanning the entire file. If you read shorter files for some users and longer files for others, fs.readFile() may introduce significant variation in Task lengths, to the detriment of Worker Pool throughput.
I am not at my laptop at the moment. Isn’t the fs module using promises then?
require('fs').promises
Or even the older promisify?
You should make sure you never block the Event Loop. In other words, each of your JavaScript callbacks should complete quickly. This of course also applies to your
await's, yourPromise.then's, and so on.
@JonL to answer your question no. Straight syncs.
Well it seems there is a clear intentionality to use sync versions from the first two(the last two are your own I guess). So better to ask @patrick why this decision.
actually not having async formatters is by design on purpose.
Having async formatters will slow down everything very significantly.
As for simple loop processing of your array you will have to do each iteration in a separate tick.
Formatters are also never meant for heavy duty processing, we have separate actions steps for this.
So for more advanced hash processing, we will add separate action steps.
If you have special needs - you can create a custom extension.
@George formatters are indeed used for heavy duty if you are using them in Wappler tutorials to hash passwords with argon2. If crypto is not heavy duty for thousands of users pinging the wappler server I don’t know what is.
Also your answer doesn’t touch on my main point on the nodejs model in general which includes modules being synchronous.
That tutorial is for PHP model. Argon2 is not supported officially for node server model.
That is not true, please stop making comments and starting unnecessary discussions if you don't understand the underlying technology.
All Server Action steps ARE asynchronous!
Just the formatters of expressions aren't! And that is on purpose as explained above,
So please keep the discussions to the point and stop making false accusations.
