MinIO S3 + Wappler

Wappler General Need Help
1

I have deployed S3 file storage (https://github.com/minio/minio) on my server. I’m trying to work with him in Wappler. However, errors occur at certain steps. For example, when trying to Put a file to the repository (Put File), the following error occurs:

21
211656×510 209 KB

At the same time, the step of getting a list of files in the buckets (S3 List Files) is working fine. The step of getting a buckets list (S3 List Buckets) also works fine.

Is there any way to fix the work with minio in those steps where errors occur?

2

@Mr.Rubi I don’t get it - why do you need some middle-ware to work with S3 buckets if Wappler already have all you need?

image
image1018×703 31.1 KB

3

There is no middleware. Most likely you didn’t fully understand what I’m talking about. MinIO is not an intermediate software for working with S3, it is S3, which can be installed on your server. For connection and requests, I use the Wappler steps that you gave above.

4

As per the error, minio could not parse the request body. Maybe its not XML or not what minio is expecting?

Try using an external tool to connect and upload files - like S3 browser for Windows. For mac I think cyberduck could work.
If it works there, then it could be a Wappler specific problem.

5

I did the testing as you recommended. This is a specific Wappler problem. Everything works fine through S3 browser for Windows. Creating new buckets, uploading files:

22
221208×920 115 KB

6

In your first screenshot you are trying to access bucket name “56.jpg” but in your last screenshot bucket name is “test”.

7

For what reason did you draw such conclusions?

My queries have the correct bucket name:

23
231663×531 198 KB

24
24885×419 39.3 KB

8

image
image1656×510 305 KB

9

This is the output of Wappler, it does not depend on me.

10

I’ve installed minio in docker next to Wappler and was able to connect to bucket with S3 Browser but only if I uncheck Use secure transfer (SSL/TLS)

image
image899×970 158 KB

But when I’m trying to connect with Wappler to same S3 bucket I got this error:

  server-connect:server Got error? Error: write EPROTO 139993102608320:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332:
  server-connect:server 
  server-connect:server     at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:94:16) {
  server-connect:server   errno: -71,
  server-connect:server   code: 'EPROTO',
  server-connect:server   syscall: 'write',
  server-connect:server   '$metadata': { attempts: 1, totalRetryDelay: 0 }
  server-connect:server } +19s

I’m not sure, but most likely Wappler expects to have valid certificate on S3 endpoint.

11

It’s true, that’s the way it is. The wappler only works with valid SSL from the host. I released it, my tail has an SSL certificate.

12

With the help of a Wappler, some of the steps, as I wrote above, work absolutely correctly. I can get a list of buckets, I can get a list of files in a specific bucket.

13

Then please ignore my previous messages - I misread your first message and thought that none of S3 actions are working.

14

@patrick, perhaps you can recommend something in this matter?

15

Have you considered running an older version of MinIO by any chance?

16

Default configuration of min.io is the use “Path-style” request and not “Virtual-hosted–style”. And looks like that Wappler use the method Virtual-hosted.

This should be made first for a proper Virtual-hosted–style inside min.io:
https://github.com/minio/minio/tree/master/docs/config#domain

Obtaining buckets doesn’t affect how the component query with the right credentials and endpoint, but for obtain files looks like a Wappler error.

I have already testing with a min.io installation using cloudron.io this allow directly install mini.io as an app that comes ready to use.

Obtaining buckets not a problem in Wappler, but listing files cause the error, and that is what I suppose the problem comes from virtual-hosted method that Wappler use to connected in s3 API.

Amazon deprecate the use of path style, but I really don’t know how Wappler it. Maybe allow to choose between Virtual hosted and Path style.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html

Using the same software S3 Browser for testing.
Menu Account->Manage accounts.
Then Edit the account->Then advanced settings.

In Addressing model if choose Virtual hosted no files are retrieving. Using Path Style the files are listed.

image
image1251×1203 222 KB

Can you please help us with this?
Any chance to allow choose the method of addressing buckets.

@wappler_ambassadors
@patrick @George

17

A temporal workaround is to add “forcePathStyle” into app.js file. This will force to work with default min.io configuration with Path Style and not Virtual-Hosted:

Wappler version 5.5.3

File

/lib/core/app.js

In line 433 after signatureVersion add forcePathStyle to true.

Original:
this.s3[name] = new S3({ endpoint: 'https://' + endpoint, credentials: { accessKeyId, secretAccessKey }, region, signatureVersion: 'v4' });

Modification:
this.s3[name] = new S3({ endpoint: 'https://' + endpoint, credentials: { accessKeyId, secretAccessKey }, region, signatureVersion: 'v4', forcePathStyle: true });

Now Wappler will force to addressing using “Path Style” and all works.

Of course after next Wappler update this modification to app.js file will dissapear.

The correct way is to proper config min.io to allow the use of virtual-hosted and not path style, because this is deprecated.
But looks like is a bit complicated to made minio work with virtual-hosted, at least for me, I have tried differents configuration but I can’t made it wo work.

3 Likes
18

Hello Alexander! I confirm that all of the above is true and you have found a great workaround! Although I try not to touch or change the core files, but it turned out to be much more efficient than building a separate server extension for MinIO based on the library https://github.com/minio/minio-js With your solution, just one property makes all the steps of the Wappler with MinIO work correctly.

19

I guess the most feasible option is for the Wappler Team to implement it in native way and allow to choose from virtual-hosted and path style. This change is really minimum and I don’t think that could affect to other s3 connections.

Maybe allow choosing during this configuration:
image

2 Likes
20

Bump!. @patrick maybe any chance to implement this natively?