CODEWITHSUNDEEP
amazon-web-servicesamazon-athena
shoya
shoya

Reputation: 53

How to create AWS Athena partition via AWS SDK

I tried adding athena partition via aws sdk nodejs.

const AWS = require('aws-sdk');
const athena = new AWS.Athena({apiVersion: '2017-05-18'});

let queryParams = {
    QueryString: `ALTER TABLE table_name ADD PARTITION (year='2018',month='10',day='10') location 's3://bucket-name/2018/10/10/'`,
    ResultConfiguration: {
        OutputLocation: 's3://bucket-name/result-logs'
    },
    QueryExecutionContext: {
        Database: 'database_name'
    }
}
athena.startQueryExecution(queryParams, (err, data) => {
    console.log('start query');
    console.log('err : ' ,  err);
    console.log('data : ' , data);
});

however, I got following error logs.

{ InvalidRequestException: line 1:28: missing 'COLUMN' at 'PARTITION'
    at Request.extractError (/project/node_modules/aws-sdk/lib/protocol/json.js:48:27)
    at Request.callListeners (/project/node_modules/aws-sdk/lib/sequential_executor.js:109:20)
    at Request.emit (/project/node_modules/aws-sdk/lib/sequential_executor.js:81:10)
    at Request.emit (/project/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/project/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/project/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /project/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/project/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/project/node_modules/aws-sdk/lib/request.js:685:12)
    at Request.callListeners (/project/node_modules/aws-sdk/lib/sequential_executor.js:119:18)
  message: 'line 1:28: missing \'COLUMN\' at \'PARTITION\'',
  code: 'InvalidRequestException',
  time: 2018-10-11T02:03:48.979Z,
  requestId: 'xxxxxxx',
  statusCode: 400,
  retryable: false,
  retryDelay: 81.3136245913281 }

I could not find COLUMN and PARTITION params in aws docs

how to define COLUMN and PARTITION in params json?

Upvotes: 4

Views: 1436

Answers (2)

Ofir Malka
Ofir Malka

Reputation: 384

Had the same issue, in my case i was building the query string like that:

`ALTER TABLE table ADD IF NOT EXISTS
    PARTITION (x = ${x}, y = ${y}, dt = ${dt})`

missing '' around the ${dt} x, y are integers while dt is a date string XXXX-XX-XX

Upvotes: 1

Tom
Tom

Reputation: 1730

The S3 object key path should include the partition name as well as the value.

For example:

s3://bucket-name/2018/10/10/

should be

s3://bucket-name/year=2018/month=10/day=10/

Upvotes: 0

Related Questions