Filemanager similar to cPanel but for Docker Volume

Wappler General How To
#21

I will try simplify the steps for you, although I am no expert myself, but this is the way I see it.

Think of docker in 4 parts just so it makes sense
Docker Machine
Docker Container
Docker Database Volume
Docker Volume

The Docker Machine is the virtual machine at the Hosting provider, this machine handles the processing, the web server software such as Apache / PHP / MySQL.
You can connect to the Docker machine using any SSH application you like.

The Docker Container is stored on the Docker machine, and is like a disk image, that is a self contained unit, and once created by Wappler, any alterations you make, do not edit this container but replace and regenerate a completely new container, even for the smallest of edits. This gives you peace of mind if you are hosting a single website over multiple web servers to handle massive traffic, so it makes sense that it was created in this way.
Inside this docker container are all you website files, including any static images, your style sheets, javascripts, etc.

Because each alteration you make in Wappler creates a completely new container, there needs to be a way that your database updates, as well as your user uploads are not overwritten with each new container created.

Docker Database Volume, holds the database data and schema, and can be managed directly in Wappler or by any third party database tool.

Docker Volume, holds all user uploaded assets, be it pictures, movies, pdfs, etc. and again is persistent storage, that does not get overwritten when a new container is created.

Just like you can manage your database data and structure with an external application, and those alterations will not be overwritten, we need a way to manage user uploaded files, in a file manager style interface, where you can download / upload / delete / rename files as you want. This is where you need the free Cyberduck application or any SFTP compatible client.

To find this folder in Cyberduck you will browse from to base directory of / as your second screenshot shows, and then start opening the following folders.
var
lib
docker
UnknownName__UnknownName_user_uploads
_data

Once you have that final folder open you will find copies of all the files your user has uploaded.

If you dislike the system as I am describing, or you have greater security, or backup needs, then you could also just use the S3 environment instead.

I hope this helps a little.

2 Likes
#22

Correct me if I am wrong, but this persistent/non-overwriting folder is defined by the “User Uploads Folder” in Wappler project settings.
All files & folders apart from this will get overwritten on each deploy.
And cyberduck/WinSCP can be used to manually manage this persistent folder.
Is this understanding correct?

1 Like
#23

Indeed users upload folder is created in special docker volume that gets preserved in deploys, just like the database that gets also its own volume.

Only if you completely shutdown and clear the docker project those will be cleared as well.

1 Like
#24

Presumably this means that if the system crashed for any reason, your database and the contents of your uploads folder would disappear.

#25

this is the problem that scares me. if for some reason Docker restarts all folders containing images and PDF are lost?

#26

I have PHP sites that contain thousands of images inserted, managed and cataloged by the remote user. If for some reason they were deleted, without the comfort of a backup, it would be a disaster.Therefore a nodejs project developed with Docker on Cloud servers (like Heztner) is fantastic for many reasons but it does not give you the security of continuity, unless you pay a cost. additional monthly, and maybe not even that?

#27

That is why an S3 is next to file delivery also an excellent low cost backup service. It is just file system in the cloud. Many different providers support it with various plans for backup, CDN etc.

Much better than taking care of the backups yourself.

#28

thanks George. but I have chosen Hetzner and I am satisfied for the moment, but I have not yet understood if with Hetzner it is possible to use S3 and possibly how, since it is not proposed by Wappler

#29

You can use any s3 service, you are not bound to hetzner.

#30

I have not been concerned about this because, for this to happen I would need to go to project settings, manage the docker machine and shutdown the machine, then clear it manually.
I can not really see this happening by mistake.

1 Like
#31

but if it were possible to use the same provider I would prefer

#32

I ask a more direct question. having a Cloud on Hetzner it is possible to associate an EXT4 or XFS volume. can you use it as S3?

#33

Yes(block storage) and nope. Hetzner doesn’t offer object storage(only block).

You can install an open source object storage in your Herzner VPS though if you wanted but I think it’s too much of a hassle for your use case.

#34

I am not sure Marzio, however I did not have to do anything in the hosting provider manually to create the volume.

So if I were you, before doing anything inside Hetzner, I would do a quick test directly inside Wappler to see if it can create the User Uploads Volume for you automatically.

Open your Wappler project
With your target set to your Hetzner remote Docker Machine, select Host Server SSH

Screenshot 2021-04-09 at 11.28.20
Screenshot 2021-04-09 at 11.28.202342×188 90.7 KB

Wait for the SSH to log in to the Hetzner machine a few seconds, then type in docker volume ls ENTER
Check if any items in the outpul show a Volume Name that ends with _user_uploads like this
Screenshot 2021-04-09 at 11.39.53

If it is there, then great your Project settings are already correct, and Wappler has already created it all for you and you do not need to do anything more inside the Hetzner control panel.

If not, then open your Project Settings in Wappler

Screenshot 2021-04-09 at 11.43.07
Screenshot 2021-04-09 at 11.43.07820×663 137 KB

Click the folder and create a new folder inside your local projects /public/ folder named whatever you like, i prefer all lowercase, no spaces, as per the screenshot. click save, it may take a few moments to create the volume for you.

Once saved, go back to the Wappler SSH area and try the same steps again docker volume ls
If you now have an entry ending in _user_uploads then you are done, and the volume has been created inside Wappler and you have no need to manually do more inside Hetzner.

#35

Schermata 2021-04-09 alle 12.30.34

#36

And what do your project settings look like, mainly the “User Uploads Folder” in the general section.

Also did you just add the “User Uploads Folder” and save, if so then please deploy before checking again.

#37

unfortunately a new problem arises, probably with the new version of Wappler. I can no longer choose the upload folder in the project configuration.

#38

What do you mean you cannot select the folder? Can you please be a little more specific?
Also you are jumping from topic to topic - what the exact issue here is, i am really confused by all your different questions?

So to summarize all the posts here - you have 2 options:

  • define user uploads folder and store your files there
  • use an s3 storage provider and store them there

that’s all.

#39

however I cannot select the upload folder in the project. perhaps because of MacOS BigSur

#40

What do you mean? Please explain the issue more detailed!
I am on Big Sur 11.2.3 and i can select a folder in the project settings without any problem.