Problem uploading using S3 file upload

Wappler General Bugs
#1

I’m trying to set up the S3 file upload to DigitalOcean Spaces. I haven’t used this feature before and I’m probably doing something wrong.

I can list buckets and files etc. so I think the settings are essentially correct. I’ve added the CORS settings according to the docs. What might be wrong are the settings I used in the ‘S3 sign upload template’ step. As far as I can see, the S3 Provider Properties should be the same as the settings already entered in Globals > S3 Storage - so I wouldn’t expect to have to enter them again. If different values should be entered, what should they be?

If I run the server connect/api file itself, I get:

“message”: “Found 2 errors while validating the input provided for the PutObject operation:\n[Key] is missing and is a required parameter\n[Key] expected string length to be >= 1, but found string length of 0”,<

If I select a file to upload, and click the upload button, the upload starts (progress bar moves etc), but then it fails. After some retries ‘Network error, perhaps no CORS set’ appears. the request URL includes sha256=UNSIGNED-PAYLOAD.

I would be grateful for any suggestions.

Wappler 4.7.2 Released
2020: The Year in Review
#2

There have been some changes with the new version (3.5.6). It seemed odd that it was necessary to repeat the details already entered into the S3 Storage Glocal. This issue seems to have been fixed. However I still can’t get it to work.

Now, having selected files to upload, a message appears - ‘Ready to Upload’ (this didn’t appear before). Then, after clicking the upload button, this appears:

image

After the upgrade, the template now creates steps like this:

image

Are the steps in the wrong order? I tried reording them but that didn’t help.

I might well be doing something wrong but I can’t get it to work. Any suggestions would be much appreciated.

#3
#4

Seems that the template is indeed in the wrong order (reversed). You only need the Sign Upload Url step if you have the S3 Provider globally, just change the name to url.

#5

Thanks, but I’m not sure what you mean. I had tried reversing the order, like this:

image

… but this didn’t help. What else should I change?

Where?

One other thing I changed is the code which is generated in App Connect:

image
(I removed the extra spaces)

#6

That is indeed the correct order, what error do you get when the action is called?

#7

I get the same error as above - 'Failed to execute ‘setRequestHeader’ etc.

The error message in the response is: “Endpoints must be full URIs and include a scheme and host”

(On the same page as the upload control, I’m displaying a list of buckets, so the global S3 Provider must be working correctly.)

#8

Looks like something on the client, if you open the server action url directly in the browser, does it work?

#9

No, I see this message above (‘Endpoints must be full URIs…’). The message refers to line 701 in ClientResolver.php.

#10

Check your S3 Provider settings, is the endpoint set?

#11

Yes:

image

… and I can view images which are stored in these buckets/spaces in a browser.

#12

Sorry, I think you meant this:

image

… but I’m not sure where I should add the endpoint. The bucket/space is specified in the Sign Upload Url step.

#13

If you directly use a Service it should be generated for you, so that should be fine. Remove the S3 Provider step in the sign upload action file, check the Sign Download Url step if it points to the correct provider.

#14

I removed the step and this time it seemed to work - but unfortunately it didn’t. There was a delay and the progress bar updated, and after a little delay:

image

Opening the server connect file in a browser:
“message”: “Found 2 errors while validating the input provided for the PutObject operation:\n[Key] is missing and is a required parameter\n[Key] expected string length to be >= 1, but found string length of 0”,

In the DigitalOcean settings:

image
image857×195 11.3 KB

#15

The server action expects a name parameter, that’s why it shows the error in the browser. Calling it with ?name=file.ext added to the url should work.

Does the origin exactly match? (you could test by temporary allowing all domains with *)

What does the page show in the network tab of devtools?

#16

Calling the server parameter doesn’t give any errors but it doesn’t work. (Wouldn’t it need the whole path to know which file to upload?). It returns this:

“url”: “https://bucketname.ams3.digitaloceanspaces.com/112-1234_IMG.JPG?x-amz-acl=public-read&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=XXXXXX%2F20201127%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20201127T140433Z&X-Amz-SignedHeaders=host%3Bx-amz-acl&X-Amz-Expires=300&X-Amz-Signature=d77bee442550da2f47c09017cc815a75aa7663e1f5612b0beb3c8b139369d34b”

It looks the problem migtht be: Sha256=UNSIGNED-PAYLOAD
… the region looks wrong: us-east-1

This is part of the global S3 setting:

image

I forgot to mention that I had already tried using the * setting the CORS configuration:

image

#17

The X-Amz-Content-Sha256=UNSIGNED-PAYLOAD is that it has no hash to validate the data integrity. The region us-east-1 is the default, it isn’t used by Digital Ocean and it I have tested it and it didn’t matter.

The url looks fine, what fails is the actual upload from the client. Does the network tab of devtools show anything, connection being disconnected or error?

#18

I was using chrome and the failed PUTs showed Status: ‘failed’ and Response: ‘Failed to load response data’.

I’ve just tried with Firefox and the failures return a 403 and this Response message:

<?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch</Code><RequestId>tx0000000000000a2d9da8d-005fc138c1-2c412d2-ams3b</RequestId><HostId>2c412d2-ams3b-ams3-zg02</HostId></Error>

#19

I’ve just set up an account with Lineode. I thought it might be a problem with DigitalOcean but I have the same problem. Again, I’ve checked the account details by listing buckets etc. without any problem.

The console errors include:
Access to XMLHttpRequest... has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

I tried with CORs enabled and disabled (an option on Lineode).

#20

This last error could actually be related to the region. Some Object Storage providers don’t take into account the region(like DO) in the credential url parameter but others do(like scaleway). It might be the case for linode.

Additional info: S3 custom provider -> Add region field and make code a bit more custom provider friendly