CODEWITHSUNDEEP
node.jsamazon-s3minio
Furman
Furman

Reputation: 2395

AWS SDK S3 node.js connect to local MinIO server

I have application server written in Node.js which upload files to AWS S3 storage. For that I'm using https://www.npmjs.com/package/aws-sdk and when I'm connecting to and uploading to my AWS production storage it is working fine. However during development I want to upload files to local MinIO server (create from docker image https://hub.docker.com/r/minio/minio/):

docker run -p 9000:9000 -p 9001:9001 \
  quay.io/minio/minio server /minio --console-address ":9001"

MinIO server itself is working fine, I'm able to login there locally, create buckets, etc. However when I tried to upload file to my local MinIO storage using AWS SDK for node.js I'm unable to do so:

import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';

#s3Client = new S3Client({
    credentials: {
        accessKeyId: config.getAwsStorageAccessKeyId(), // from env variables
        secretAccessKey: config.getAwsStorageSecretAccessKey(),
    },
    ...(globalConfig.isReleaseVersion() && {
        endpoint: config.getAwsStorageEndpoint(), // from env "http://localhost:9001"
    }),
});

And later I'm trying to upload object:

await this.#s3Client.send(new PutObjectCommand({
    Bucket: config.getAwsStorageWebflowBucketName(), // from env "test" value
    Key: 'example.json',
    Body: body, // a buffer
}));

I'm receiving error:

ERROR EndpointError: Custom endpoint `test.127.0.0.1://9001/` was not a valid URI

However I'm able to reach address http://test.localhost:9001/ from my browser. When I try to pass directly value 'http://localhost:9001/' as an endpoint to S3Client configuration, I'm getting error:

ERROR Error: getaddrinfo ENOTFOUND test.localhost

Looks like port value is ignored my SDK. How can this be fixed?

<Yes, I know there is separate MinIO SDK - I want to use one provided by AWS, existence of endpoint parameter in configuration implies that it should be possible>

Upvotes: 6

Views: 11343

Answers (5)

OhadR
OhadR

Reputation: 8839

I saw here correct answers (@gladix and @p2t2p - I have upvoted, of course). But I think we should make it a bit clearer:

AWS SDK V3

In the question I see:

import { S3Client } from '@aws-sdk/client-s3';

It means he uses V3 of the AWS SDK. This is important, because in V3 the config passed to the constructor of the client is of type S3ClientConfig.

import { S3ClientConfig } from "@aws-sdk/client-s3";

And there, there is a member forcePathStyle: boolean.

AWS SDK V2

However, if user uses V2 (like import * as AWS from 'aws-sdk'), then the config passed to the constructor of the client is of a different type, ClientConfiguration.

import { ClientConfiguration } from "aws-sdk/clients/s3";

And there, there is a member s3ForcePathStyle: boolean.

Hope that makes things clearer :)

Upvotes: 2

p2t2p
p2t2p

Reputation: 312

All you need to do is set s3ForcePathStyle config option to true. Alternatively (in sdk V3) it can have name forcePathStyle. In my code I do it this way:

const awsConfig: S3.Types.ClientConfiguration = {
  s3ForcePathStyle: true
};

I understand that there's a similar answer but it did resolve my issue.

Additional options that I also supplied:

sslEnabled: false
endpoint: http://localhost:port/
accessKeyId: dummy
secretAccessKey: dummydummy

Upvotes: 6

yash shah
yash shah

Reputation: 56

http://localhost:9001 is not the correct URL

  • 9001 is the one where you access the minio console
  • 9000 is the one where S3 API is served

Console output when running minio

Try following & it should work

const s3Client = new S3Client({
  region: "region_set_in_minio",
  credentials: {
    accessKeyId: "minio_access_key",
    secretAccessKey: "minio_secret_access_key",
  },
  endpoint: "http://127.0.0.1:9000",
});

You can set minio region from minio console.

Go to Administrator -> Settings -> Region

Change minio region

Upvotes: 4

V.Andri
V.Andri

Reputation: 12

the listener endpoint for S3 API is port 9000, 9001 is for dashboard, also remove region when using Minio, because I think Minio doesn't need it. by so, sdk will not give a prefix 'test' for the endpoint.

Upvotes: 0

gladix
gladix

Reputation: 402

Have you tried the option "s3ForcePathStyle" ? I had to use this config to make it work on a local environment

  options.s3ForcePathStyle = true

See https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Config.html#s3ForcePathStyle-property

So in your code

...(globalConfig.isReleaseVersion() && {
    endpoint: config.getAwsStorageEndpoint(), // from env "http://localhost:9001",
    s3ForcePathStyle: true,
}),

Upvotes: 0

Related Questions