CODEWITHSUNDEEP
odbcdatabricksarm64
James Palawaga
James Palawaga

Reputation: 298

('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/simba/spark/lib/64/libsparkodbc_sb64.so' : file not found (0) (SQLDriverConnect)")

I was attempting to get the Databricks ODBC driver working on Golang. I followed the directions on https://github.com/alexbrainman/odbc in order to setup the Go library and the system library.

Instead of setting up the MSSQL ODBC driver, I installed Databricks' ODBC driver and configured the system to use it. I re-used a the mssql_test.go file to create a testbed to test the connection to Databricks, using a DSN/connection string I built using the guide from their website.

However, when attempting to connect to the database, I get the error:

('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/simba/spark/lib/64/libsparkodbc_sb64.so' : file not found (0) (SQLDriverConnect)")

No matter how I change the connection string, LD_LIBRARY paths, or anything else, I cannot seem to get things to work. It repeatedly gives this error, even though if I do an os.Stat() on this file in Go, I can see it clearly exists in the execution environment.

I'm running Go/the driver inside the a Debian linux container user docker, on my M1 Macbook.

Upvotes: 2

Views: 4346

Answers (3)

Antonio
Antonio

Reputation: 61

I had same issue while was using python interpreter from python:3.11.3-slim-bullseye image. Creating Dockerfile as:

FROM python:3.11.3-slim-bullseye

resulted loading image by default for the host machine architecture - ARM in my case.

So, one have to choose the correct OS/Arch selecting it from the dropdown, and consequently pull the docker image by digest.

FROM python:3.11.3-slim-bullseye@sha256:7b866f12347fbfccbb73284d9c04bbd67b5f9cca8b46786e9fa2bd07af53f09a

This was the trick!

dockerhub

Also on a docker desktop you may see a warning sign if the AMD image is run on ARM host machine: docker desktop AMD image run

Upvotes: 0

Pawe zaicnupagadi Jarosz
Pawe zaicnupagadi Jarosz

Reputation: 11

If you have similar situation, check this:

ldd /opt/simba/spark/lib/64/libsparkodbc_sb64.so

It will tell you whether there are some depending files missing.

Upvotes: 1

James Palawaga
James Palawaga

Reputation: 298

In this case, the reason that the file cannot be found is that the driver has not been compiled for ARM. This post on Github illustrates the issue.

We can confirm by running file on the library. It yields...

$> file /opt/simba/spark/lib/64/libsparkodbc_sb64.so
/opt/simba/spark/lib/64/libsparkodbc_sb64.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped

You can see it is compiled for x86-64, but the platform is actually ARM. No good!

There are a few different options to solve this problem:

  1. if you have the source of the driver, recompile the driver to target ARM.
  2. if you're building on docker, you might be able to rebuild your docker container to use the x86 platform instead. You can do this with the --platform linux/amd64 build flag to either your Dockerfile (e.g. FROM --platform=linux/amd64 golang:1.17 or to your build command.
  3. build/test/deploy your code in an x86 environment.

In the case of Databricks, they are aware of the issue, and an ARM-based driver is forthcoming.

Upvotes: 1

Related Questions