NPM install node-rdkafka fails with node-gyp rebuild error

Question

I am trying to install node-rdkafka using npm and seeing the below error

• OS: macOS Catalina 10.15.7
• node version: 10.16.3 tried (8.x, 12.x and 13.x) as well via nvm

/Users/VMac/.nvm/versions/node/v10.16.3/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin/node-gyp: line 5: /usr/local/lib/node_modules/node-gyp/bin/node-gyp.js: No such file or directory
npm WARN kafka-log-parser@1.0.0 No repository field.

npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn
npm ERR! node-rdkafka@2.7.0 install: `node-gyp rebuild`
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the node-rdkafka@2.7.0 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/VMac/.npm/_logs/2020-09-30T14_35_01_271Z-debug.log

When I try running installing node-gyp using npm install -g node-gyp and run node-gyp rebuild, I see the below error

gyp info it worked if it ends with ok
gyp info using node-gyp@7.1.0
gyp info using node@10.16.3 | darwin | x64
gyp info find Python using Python version 3.8.3 found at "/usr/local/anaconda3/bin/python3"
gyp info spawn /usr/local/anaconda3/bin/python3
gyp info spawn args [ '/Users/VMac/.nvm/versions/node/v10.16.3/lib/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/VMac/Documents/VMAC/Code/GIT/kafka-log-simulator/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/VMac/.nvm/versions/node/v10.16.3/lib/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/VMac/Library/Caches/node-gyp/10.16.3/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/VMac/Library/Caches/node-gyp/10.16.3',
gyp info spawn args   '-Dnode_gyp_dir=/Users/VMac/.nvm/versions/node/v10.16.3/lib/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/VMac/Library/Caches/node-gyp/10.16.3/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/VMac/Documents/VMAC/Code/GIT/kafka-log-simulator',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.' ]
gyp: binding.gyp not found (cwd: /Users/VMac/Documents/VMAC/Code/GIT/kafka-log-simulator) while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/Users/VMac/.nvm/versions/node/v10.16.3/lib/node_modules/node-gyp/lib/configure.js:351:16)
gyp ERR! stack     at ChildProcess.emit (events.js:198:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
gyp ERR! System Darwin 19.6.0
gyp ERR! command "/Users/VMac/.nvm/versions/node/v10.16.3/bin/node" "/Users/VMac/.nvm/versions/node/v10.16.3/bin/node-gyp" "rebuild"
gyp ERR! cwd /Users/VMac/Documents/VMAC/Code/GIT/kafka-log-simulator
gyp ERR! node -v v10.16.3
gyp ERR! node-gyp -v v7.1.0
gyp ERR! not ok

I also tried xcode-select --install but no luck. Any suggestions on how to fix this and successfully install node-rdkafka using npm on my mac?

Answer 1

folks, after going through several recommendations and not solving it for me the issue turned out to be I had intel version of home-brew rather than arm version in my mac.

I had to uninstall those and reinstall arm versions. that did the trick.

Answer 2

This is what worked for me:

Download the appropriate version of the "Command Line Tools for Xcode" for your version of Catalina from https://developer.apple.com/download/more/. As of MacOS 10.15.5, that's Command_Line_Tools_for_Xcode_11.5.dmg

From https://github.com/nodejs/node-gyp/blob/master/macOS_Catalina.md#installing-node-gyp-using-the-xcode-command-line-tools-via-manual-download

Answer 3

After hitting this issue myself and trying the solutions proposed in both answers without success this is what worked for me, without the need to uninstall node.

TL:DR

sudo rm -rf /Library/Developer/CommandLineTools
xcode-select --install
npm install node-rdkafka

After this I was able to install node-rdkafka without any problems.

I had to remove CommandLineTools and reinstall it, as suggested here, because when I tried to run xcode-select --install I would get the

xcode-select: error: command line tools are already installed, use "Software Update" to install updates

Answer 4

After trying various options online, I couldn't find the right solution. So, this is what I did and worked

1. Uninstalled node following the instructions here

   If you are using Homebrew, run

   brew uninstall node 
   

   Use --force or --ignore-dependencies if required

2. Run node -v. If you still see the version info, follow the instructions here

3. I also wanted to remove nvm from my machine. So, I removed the ~/.nvm folder. But, I am still seeing the nvm version. To completely clean-up, remove references from ~/.bashrc and ~/.zshrc if any

I realised that I had two versions of node

1. Installed manually
2. Installed using Homebrew

Post this, I only installed nvm (Don't install node or npm).

brew install nvm

and then installed the required version of node and npm using

nvm install <NODE_VERSION> 

This solved my node-gyp errors