CODEWITHSUNDEEP
node.jsamazon-s3
JulienM
JulienM

Reputation: 134

AWS S3 parse URI error with docker containers

On my local environment, I develop a file upload feature with a express/node backend and an AWS S3 server (using scality/S3). Both services are dockerized in their own container, and the communication works fine. My problem is that the S3 server seems not to understand the upload request. Here is the code I try to use :

const s3 = new S3({
  accessKeyId: 'accessKey1',
  secretAccessKey: 'verySecretKey1',
  endpoint: 's3server:8000',
  sslEnabled: false,
  s3ForcePathStyle: true,
});

function uploadFile(file) {
  const params = {
    Body: file,
    Bucket: 'testbucket',
    Key: 'testkey',
    ACL: 'public-read',
  };
  s3.upload(params, (err, data) => {
    if (err) console.log(err, err.stack); // an error occurred
    else console.log(data); // successful response
  });
}

The S3 server receives the request but send back the error:

{"name":"S3","clientIP":"::ffff:172.18.0.5","clientPort":45066,"httpMethod":"PUT","httpURL":"/testbucket/testkey","time":1502458550488,"req_id":"7f4fac280644b5cf203c","level":"info","message":"received request","hostname":"faf8cb0b47d4","pid":103}
{"name":"S3","bytesSent":192,"clientIP":"::ffff:172.18.0.5","clientPort":45066,"httpMethod":"PUT","httpURL":"/testbucket/testkey","httpCode":400,"time":1502458550491,"req_id":"7f4fac280644b5cf203c","elapsed_ms":2.607924,"level":"info","message":"responded with error XML","hostname":"faf8cb0b47d4","pid":103}

And the node backend logs the error:

{ InvalidURI: Couldn't parse the specified URI.
     at Request.extractError (/usr/src/api/node_modules/aws-sdk/lib/services/s3.js:577:35)
     at Request.callListeners (/usr/src/api/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
     at Request.emit (/usr/src/api/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
     at Request.emit (/usr/src/api/node_modules/aws-sdk/lib/request.js:683:14)
     at Request.transition (/usr/src/api/node_modules/aws-sdk/lib/request.js:22:10)
     at AcceptorStateMachine.runTo (/usr/src/api/node_modules/aws-sdk/lib/state_machine.js:14:12)
     at /usr/src/api/node_modules/aws-sdk/lib/state_machine.js:26:10
     at Request.<anonymous> (/usr/src/api/node_modules/aws-sdk/lib/request.js:38:9)
     at Request.<anonymous> (/usr/src/api/node_modules/aws-sdk/lib/request.js:685:12)
     at Request.callListeners (/usr/src/api/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
     at Request.emit (/usr/src/api/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
     at Request.emit (/usr/src/api/node_modules/aws-sdk/lib/request.js:683:14)
     at Request.transition (/usr/src/api/node_modules/aws-sdk/lib/request.js:22:10)
     at AcceptorStateMachine.runTo (/usr/src/api/node_modules/aws-sdk/lib/state_machine.js:14:12)
     at /usr/src/api/node_modules/aws-sdk/lib/state_machine.js:26:10
     at Request.<anonymous> (/usr/src/api/node_modules/aws-sdk/lib/request.js:38:9)
   message: 'Couldn\'t parse the specified URI.',
   code: 'InvalidURI',
   region: null,
   time: 2017-08-11T13:35:50.510Z,
   requestId: '7f4fac280644b5cf203c',
   extendedRequestId: '7f4fac280644b5cf203c',
   cfId: undefined,
   statusCode: 400,
   retryable: false,
   retryDelay: 57.08331622136704 }

I saw some answers about utf-8 encoding problems but it didn't work on my case :/

Does anyone has an idea about why it can't parse the URI ?

Thank you for your time !

Upvotes: 5

Views: 2045

Answers (1)

Palisand
Palisand

Reputation: 1362

You can fix this by providing your own config.json using volumes, as suggested here.

  1. Copy config.json.
  2. Add "s3server": "us-east-1", to "restEndpoints".
  3. docker run -v $(pwd)/config.json:/usr/src/app/config.json ... or if you're using docker compose, add something like the following to your docker-compose.yaml:
s3server:
    image: scality/s3server
    restart: always
    expose: [8000]
    stdin_open: true
    tty: true
    container_name: s3server
    volumes:
      - "${PWD}/s3config.json:/usr/src/app/config.json"

Upvotes: 1

Related Questions