Has anyone got the s3 connecter to work with Google Cloud Storage?

Just wondering. I’ve hit some errors on my end but this is my first foray into to object storage so i’m thinking its user/configuration issues.

I can list the bucket names, but can’t go a next step down and list bucket contents, kepp getting an error like:

{"status":"500","code":"NotImplemented","message":"A header or query you provided requested a function that is not implemented.","stack":"NotImplemented: A header or query you provided requested a function that is not implemented.\n    
at Request.extractError (/opt/node_app/node_modules/aws-sdk/lib/services/s3.js:700:35)\n    
at Request.callListeners (/opt/node_app/node_modules/aws-sdk/lib/sequential_executor.js:106:20)\n   
at Request.emit (/opt/node_app/node_modules/aws-sdk/lib/sequential_executor.js:78:10)\n    
at Request.emit (/opt/node_app/node_modules/aws-sdk/lib/request.js:688:14)\n    
at Request.transition (/opt/node_app/node_modules/aws-sdk/lib/request.js:22:10)\n   
at AcceptorStateMachine.runTo (/opt/node_app/node_modules/aws-sdk/lib/state_machine.js:14:12)\n   
at /opt/node_app/node_modules/aws-sdk/lib/state_machine.js:26:10\n    
at Request.<anonymous> (/opt/node_app/node_modules/aws-sdk/lib/request.js:38:9)\n    
at Request.<anonymous> (/opt/node_app/node_modules/aws-sdk/lib/request.js:690:12)\n    
at Request.callListeners (/opt/node_app/node_modules/aws-sdk/lib/sequential_executor.js:116:18)"}

Is Google Cloud Storage S3 compatible?

To list the object in the bucket we use the ListObjectsV2 which is the newer version of ListObjects. Google seems not to support V2.

I believed it was, they spoke about interoperability with AWS S3 storage.

Maybe we can add support for google cloud as S3 revideren and @patrick can implement some fallbacks for it.

1 Like

Naturally that would be great :grin:. A big report from @JonL was talking about how the custom S3 connector doesn’t allow for a region to be selected and the code in the s3.js file defaulting to us-east-1 when none detected. He hard coded a work around.

I don’t know if it’s connected at all to my issue though as Patrick mentioned the lisObjectsV2 that Google doesn’t support. I did check though and they definitely have a set of S3 compatible APIs that should work…so maybe it’s just the lisObjectsV2…I’ll need to test more