I recently updated one of my projects to run with multiple replicas (2 replicas in this case). The customer reported this morning that when exporting/downloading files (CSV), the process had to be run more than once for the files to be downloaded on the local PC.
Currently, the files are exported and downloaded from this path /public/assets/exports/file_name.
I think the reason for the downloads to fail is that when using multiple replicas, the download path may be looking for the file in a different replica than where it has been originally exported to.
Is this the reason for the download to fail on multiple replicas? How can I set up the export and download paths to look for the files in the same replica or the same path?
Any help and guidance for this setup is greatly appreciated.
Hi.
Here's what I can suggest given my limited knowledge about this:
With multiple instances of the same app, the server/load-balancer decides which request is resolved by which instance.
As you have identified, that is the reason for your issue.
A simple solution is S3 - acting as a central storage. Upload the exported file to S3, and the download request will then get the file from S3.
Additional note: Anything you keep in the public folder is accessible directly from the base URL of your app - so its not recommended to use public folder for sensitive/private data. You may create a folder in the root of the application, and use that. Such folders/files are accessible from back-end, but not directly in the front-end.
Thanks for suggesting the use of S3 for storing the exported files and download from there. In this case, the exported / downloaded files are used by the client to import sales information in their accounting package. So, these are once off exports and do not require a long-term storage for access after the initial download.
Thanks for highlighting the general accessibility of the public folder. I'm trialling with /uploads folder in the root directory to stop a direct access to these files.
I have created a new uploads folder in the root directory and attached it as User Uploads Folder under Project Settings. My understanding is that this folder is created as a separate volume on the remote server and should be available to all the replicas. Can this solution work with multiple replicas?
Are you using docker? If so, then this user uploads folder is created as a volume in Wappler-Docker configuration. This acts like the central storage, so can be another solution.
I did not suggest this since I didn't know if you are using Docker.
You can run a scheduler every 5 mins or so, depending on how big your files can be, to clear S3 storage periodically. S3 here is just external storage to solve the multi-instance problem.