When receiving data via the API, I ran into a similar problem. @Teodor, please tell me how this problem can be solved in NodeJS?
Seems you are posting quite a large chunk of data to remote api. What are you posting?
I use the HTTP interface to connect to the ClickHouse database. I need to send a lot of data there.
Initially, I tried to import from a CSV file, but it didn’t work out. Then I tried to transfer the data from the received file directly in the request and got this error.
Getting data in TSV format
Sending data to the database
So how much data are you posting? I see that the default max is set to 10MB, you really should think twice if you want to post larger objects in a simple server request.
Does it have something to do with security or with the stability of work? Why is this not the best solution?
I need to receive packages of 100 MB each.
@George , tell me more, please, how can I save the received data to a file? They come to me via the API in TSV format, but then they are converted to JSON. The “Export CSV” module does not help to save them to the hard disk. Is it really necessary to use a custom module for this?
@George, I’m really looking forward to your answer. If it is impossible to increase the limit on the received request size, then please tell me how I can save the CSV that I receive in the response.
Please test this update, it should increase the limit. Unzip to lib/modules.
api.zip (1.3 KB)