How to download a file from s3 (getfile? downloadfile?)

[karh]

Trying to download a file from s3 so I can then send it to another API.

I thought this would be simple to do but I’m failing to understand how to use ‘Get file’ and ‘Download file’.

I have a path that holds my file:

/school/1/invoices/factuur-A.pdf

I have a destination for this file:

/tmp/factuur-A.pdf

Whne I try to use getFile:

It throws the error:

The path I see is confusing me, it seems to join the ‘path’ with the ‘key’.

If I split up the key and the path:

it throws the same error

which makes sense, as the file is not in /opt/node_app but it’s on my s3 storage.

Am I completely missing the point here of getFile?
Since ‘putFile’ puts a file on s3 storage. I assume ‘getFile’ takes it off?

[karh]

Bump.

I’m pretty sure the ’ get file’ server action is broken - would love a fix

Changing this topic to a bug

[otavionestares]

hello @karh, I use “Sign Download URL”, it will generate a temporary link to your document.

On API:

On front end: dmx-on:click="download1.download(your_link_generated by the API)" using the download component.

[scalaris]

It’s not that easy. I recommend using a wappler custom file action- this library works well for me.

GitHub - mebeingken/wappler_modules: Custom Modules used to extend Wappler

Here’s an action shot

[mebeingken]

And this post has a newer version of the module

Get File from API Action - NodeJS [OpenSource] (v0.2)

I actually did run into issues with the request library a while back. I can’t recall the specifics, but it was obviously time to move on from that library. Here is a version of the action that now uses Axios: exports.write_binary = async function (options) { //retrieves a file via remote http request and saves the file to the local server //names the file using a uuid //for converting file paths provided in UI const { toSystemPath } = require('../../../lib/core/path'); …

I really need to update the repository!

[karh]

Awesome! Thanks @scalaris @mebeingken and @jellederijke

Will try it out soon.

@otavionestares Thanks, sadly I don’t need to download it client side. I need it downloaded server side. But it looks like Jelle’s extension can help with it.

Regarding this topic: I still believe that ‘get file’ is broken and should allow me to get files to my server using s3

[sid]

@karh

Regarding this topic: I still believe that ‘get file’ is broken and should allow me to get files to my server using s3

There bug here is that the path needs to be exactly the same as the KEY’s path.
So, before GetFile step, you need to do a create folder step, and create the folder:
/school/1/invoices/
Then, in path, just put /.

We used this just last week, and ended up copying Wappler’s code to create a custom module where we could set whatever path we need.

@patrick Bump.

[karh]

Thanks sid! Sounds not ideal to have so many folders created but at least something to work with for now!!

[karh]

Bump. Really need this fixed

[patrick]

Say I have files on the S3 like /user/patrick/avatar.png and /user/george/avatar.png.

If I save the avatar from patrick to the path /tmp/ it will generate /tmp/user/patrick/avatar.png. It will keep the path which was included in the key. If I then save the avatar from george it will be creating a new file. When we would strip the path the file would be overwritten.

Seeing the error in the topic post it seems that it doesn’t create the needed folders dynamically. Will have a look at that.

Would there be a use-case where the path needs to be stripped and file being saved as /tmp/avatar.png then let me know. I think we should keep the path in and just fix the dynamic folder creation.

[karh]

@patrick

/user/patrick/avatar.png

To make sure we’re aligned.
This is what you’re saying, correct?

@patrick

it doesn’t create the needed folders dynamically. Will have a look at that.

Yes, correct

@patrick

Would there be a use-case where the path needs to be stripped and file being saved as /tmp/avatar.png then let me know.

For my use case right now I am okay with /tmp/school/1/invoices/invoice.pdf
However, I still would feel like we would have more freedom and flexibility if the code isn’t ‘biased’ in this way.

To save /user/patrick/avatar.png as /tmp/user/patrick/avatar.png I could set it up like this:

Which has the benefits of:

1. Being more clear for the user (IMO) because there’s no mention of how they key and Path are being joined in the UI right now
2. More flexibility, because I could also save the files like
   I would then remove these files immediately (I download them to send them away again, then remove them from my server). So if for some reason these files aren’t removed and I’m seeing this folder 2 months later… I get more context with my own naming scheme.

If you do keep it the same, I highly recommend some info in the tooltips

[patrick]

The path and key are joined together, in most cases the key is only the filename and the path is some folder on your server.

I have an update that should fix the issue for creating the folders defined in the key when they don’t exists.

s3.zip (1.3 KB) Upzip it to lib/modules.

[sid]

@patrick

I think we should keep the path in and just fix the dynamic folder creation.

An ideal solution would be to have a separate checkbox to “create path” or not.

[patrick]

What would the create path checkbox exactly do. Would you rather have an error when the path doesn’t exist and the checkbox is not checked? Perhaps better then is to have an option to strip the path from the key.

[bpj]

I would prefer the option to strip the path

[scalaris]

I feel stripping the path by default is what’s generally expected - an option would be nice but the remote path would generally have little bearing on what I want t do with the file post-download

[sid]

@patrick

Perhaps better then is to have an option to strip the path from the key.

That’s another way of looking at it. What I meant is in the Path input, whatever path is given, the file should just get downloaded there.
If folders in the given path do not exists, they would need to be created. Which means having “create path” option does not sound right.
So “Strip Key Path” is a better name for it I suppose.

[Teodor]

An option to strip key path has been added to Wappler 5.3.2