Make rsync deployment method (SFTP faster alternative)

Make rsync deployment method (SFTP faster alternative)

Using Wappler FTP/FTPS/SFTP is slow, my previous deployment I used rsync and it feels much faster.

P.S.: This feature request requires the shared hosting of your choice to support SSH access

Edit: This also means skipping the remote file listing and stuff, or using rsync to get the file listing...

SFTP deployment is insanely slow

we might have found a way to speed it up.

the slowness was related to the terminal calls we do and no so much to LFTP or RSYNC (those are very comparable in speed when ran from terminal)

The new improvements are already implemented in the SSH access in the Resource Manager and we will see if we can push the same also for LFTP sync that we use for SFTP connections

2 Likes

Bump

This has been integrated in Wappler 7 beta 9

Are you sure this has been integrated? An rsync command still runs many times faster, I don't think there was any noticeable improvement. I'm on Wappler 7 beta 18

@George has this performance improvement applied to SFTP?

It's pretty unclear what changes exactly have been made to SFTP deployment to make this faster. File listing in the Publish dialog still takes an eternity (uncheck Commit to Git, you'll see the eternity of file listing)

Edit: And file uploads also seem inefficient with unnecessary deletes before overwrite?

1 Like

We are using LFTP for the file transfers and it is the industry fastest and most s feature rich. See

https://lftp.yar.ru/

As for the improvements in calling we have optimized the shell execution on Windows, but it was partially rolled back because of windows file locking problems.

My rsync command runs like 10x faster, I don't know what the hell you're doing within Wappler to make it so slow

I'm on MacOS, uploading to a Linux server

Edit:

Was this fixed?

Edit 2:
When I say my rsync runs 10x faster, I mean my rsync takes like 5 seconds, and your Wappler publish easily takes a full minute or two

Well if you have rsync setup and it is working fine for you, then just use it. For the publishing it doesn't matter if you use ftp, built-in or rsync - as long as the files get over.

It's additional mental effort to switch the target and then use the rsync command. Imagine I forget to switch the target and run the rsync command, then the production setup is screwed up. It's better to use the Publish button...

Alternatively, I did came up with some creative .rsyncignore, and I use it so I don't have to switch targets, but it's not fool-proof.

So, it's best you just improve your Publish... Especially like a previous feature request to e.g. run a custom Bash script to deploy were discarded in your mind. You don't really make it easy in any shape or form, whatever deploy method I use... And I've tested it all