CODEWITHSUNDEEP
node.jsamazon-web-servicesjdbcaws-lambdaamazon-athena
Mitchell Griest
Mitchell Griest

Reputation: 507

Trouble connecting to AWS Athena via JDBC using Node Lambda

My Goal

I am trying to use AWS's JDBC Driver to allow a Lambda function running Node 6.10 to connect to AWS Athena and create a database. (I will also want to be able to create and query against tables inside of that databse).

What I've Tried

I have tried the following code from an answer to a similar question:

var JDBC = require('jdbc');
var jinst = require('jdbc/lib/jinst');

if (!jinst.isJvmCreated()) {
  jinst.addOption("-Xrs");
  jinst.setupClasspath(['./AthenaJDBC41-*.jar']);
}

var config = {
  // Required 
  url: 'jdbc:awsathena://athena.us-east-1.amazonaws.com:443',
   // Optional 
  drivername: 'com.amazonaws.athena.jdbc.AthenaDriver',
  minpoolsize: 10,
  maxpoolsize: 100,
  properties: {
                s3_staging_dir: 's3://aws-athena-query-results-*/',
                log_path: '/logs/athenajdbc.log',
                user: 'access_key',
                password: 'secret_key'
   }
};


var hsqldb = new JDBC(config);

hsqldb.initialize(function(err) {
  if (err) {
    console.log(err);
  }
});

The Errors I'm Seeing

When I run this on my own machine (Mac OSX El Capitan 10.11.6), I see the popup pictured below with the message No Java runtime present, requesting install. printed to my console.

Popop

When I deploy my code to Lambda and run it there, it fails with the following message:

Error: /var/task/node_modules/java/build/Release/nodejavabridge_bindings.node: invalid ELF header

When run locally, I can see that things fail at the var hsqldb = new JDBC(config); line, but when running on Lambda, the error occurs immediately upon requiring JDBC (the first line of the code above).

Update

The invalid ELF header issue seems to be pointing to the idea that the node_modules/java/build/Release/nodejavabridge_bindings.node file was compiled for an architecture incompatible with the one on which AWS Lambda runs (Linux x64).

This explains the difference in behavior when running locally vs when running on Lambda.

I have tried using node-gyp to compile the resource specifically for the x64 architecture, and saw the issue change but not resolve.

The node-gyp command I ran successfully was node-gyp configure --arch=x64 (run inside the node_modules/java/ directory)

Instead of an invalid ELF header error when running on Lambda, we now see a module initialization error (See logs below)

module initialization error: Error
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at Object.<anonymous> (/var/task/node_modules/java/lib/nodeJavaBridge.js:21:16)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)

Upvotes: 1

Views: 1450

Answers (1)

jens walter
jens walter

Reputation: 14029

You are describing a couple of issues here.

First the missing JVM in MacOS. This is a documented bug within node-java. This link describes a workaround for that issue.

https://github.com/joeferner/node-java/issues/90#issuecomment-45613235

After applying that and changing the "setupClasspath"-statment, your sample should be runnable locally.

jinst.setupClasspath(['./AthenaJDBC41-1.0.1.jar']);

As for the ELF-problem, you cannot build Linux native modules for node in MacOS. And since npm does not distribute prebuild versions, you can only build you deployable on a target equivalent machine.

This means you need to install/package your modules on a Linux AMI (preferably the Lambda AMI).

Here an AWS blog post on how to do this:

https://aws.amazon.com/blogs/compute/nodejs-packages-in-lambda/

AMI versions used:

http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html

Upvotes: 2

Related Questions