CODEWITHSUNDEEP
capache-kafkaesp32platformio
Dauggie
Dauggie

Reputation: 35

Linking error: cannot find -lrdkafka pre-compiled dynamic library in PlatformIO

I'm trying to get Kafka working on an Esp32 using PlatformIO.

I tried to link a C library to a project built in PlatformIO using build flags expressed in my platformio.ini file. I am using the librdkafka library. The location of the librdkafka.so file is in /usr/local/lib but whenever I build my project with the linked library in plaformio I get the following error:

platformio/packages/toolchain-xtensa32/bin/../lib/gcc/xtensa-esp32-elf/5.2.0
/../../../../xtensa-esp32-elf/bin/ld: cannot find -lrdkafka

platformio.ini file:

[env:featheresp32]
platform = espressif32
board = featheresp32
framework = espidf
build_flags =
   -L/usr/local/lib
   -I/usr/local/include/librdkafka
   -lrdkafka

Outside of PlatformIO, I successfully linked librdkafka to the main.c project file using the following command: gcc main.c -L/usr/local/lib -l:librdkafka.so -lz -lpthread -lrt -I/usr/local/include/librdkafka

Upvotes: 0

Views: 657

Answers (2)

Julio Ribeiro
Julio Ribeiro

Reputation: 39

If the question is about hosting a Kafka server on ESP32... no way!

But, if you're looking for a way to send data from ESP32 to Kafka, I assume that you have already checked if the Kafka cluster to which you want to send data has no REST Proxy available. If you indeed depend on librdkafka, then there is a problem, as mentioned in the previous post. However, if you are using Confluent Cloud Kafka as a remote server you want to send data to, you can try the REST API available for HTTP Post. Check this out --> https://docs.confluent.io/cloud/current/kafka-rest/kafka-rest-cc.html#produce-api.

Even if the destination Kafka is not in the cloud and you are able to manage it, there are alternative solutions available, via Confluent REST PROXY you can configure.https://docs.confluent.io/platform/current/kafka-rest/quickstart.html

Upvotes: 0

romkey
romkey

Reputation: 7109

You're trying to link the Kafka library that's built for your system with your ESP32 program.

There's absolutely no way this can work. They're two different architectures. Your system that you're running platformio is most likely an Intel processor, so anything in /usr/local/lib is built for that, not for an ESP32.

You're probably running on Linux, so that Kafka library is built for Linux. The ESP32 doesn't run Linux. So there's no way that Kafka library could possibly run on the ESP32, even if it were compiled for it, which it's not.

Kafka is not designed to run on an ESP32. If you want your ESP32 to interact with a Kafka system you'll need to find a client that's compatible with Kafka that's designed to run on an ESP32. librdkafka is not in any way ESP32-compatible.

Upvotes: 1

Related Questions