node error 'invalid ELF header' regarding lz4 dependency

[jansepke]

Hello, we are not able to execute this library as node is having issues with the lz4 dependency of this package. Can you please assist on how to use databricks-sql-nodejs in a standard Linux or Mac + NodeJS environment.

steps to reproduce:

npm install @databricks/sql
node -e "require('@databricks/sql')"

error message:

{...}/node_modules/@databricks/sql/dist/utils/lz4.js:10
            throw err;
            ^

Error: {...}/node_modules/lz4/build/Release/xxhash.node: invalid ELF header
    at Module._extensions..node (node:internal/modules/cjs/loader:1460:18)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Module._load (node:internal/modules/cjs/loader:1019:12)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at require (node:internal/modules/helpers:177:18)
    at Object.<anonymous> (/home/dejhjs3e/projects/databricks-node/node_modules/lz4/lib/utils.js:4:11)
    at Module._compile (node:internal/modules/cjs/loader:1364:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Module._load (node:internal/modules/cjs/loader:1019:12) {
  code: 'ERR_DLOPEN_FAILED'
}

versions:

• @databricks/sql: 1.10.0
• npm: 10.8.2
• node: v18.20.8, v20.16.0, v22.6.0 (tested all of them)
• OS: Ubuntu 24.04.2 and Mac OS X 14.4.1

related issue: #270

[vadimgaidai]

Hi, I'm experiencing the same issue when using @databricks/sql (version 1.10.0)

When I run the project using Node v18.20.8, I get the following error:

/usr/src/app/node_modules/@databricks/sql/dist/utils/lz4.js:10
            throw err;
            ^
Error: {...}/node_modules/lz4/build/Release/xxhash.node: invalid ELF header
    at Module._extensions..node (node:internal/modules/cjs/loader:1460:18)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Module._load (node:internal/modules/cjs/loader:1019:12)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at require (node:internal/modules/helpers:177:18)
    at Object.<anonymous> (/usr/src/app/node_modules/lz4/lib/utils.js:4:11)
    at Module._compile (node:internal/modules/cjs/loader:1364:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Module._load (node:internal/modules/cjs/loader:1019:12) {
  code: 'ERR_DLOPEN_FAILED'
}

I've also tested with other Node versions with the same result. Any updates or workarounds would be greatly appreciated. Thanks!

[jansepke]

One possible workaround is to change the npm config

npm config set -L project omit optional
rm -rf node_modules
npm install @databricks/sql

then the problematic lz4 dependency will never get installed but also all other optional dependencies of your project

[zagazoom]

+1 on this, we are using Auth0 Actions and it is not possible for us to control the npm config. So we are forced to use an older version