Reputation: 1
This question is not answered in any thread, it is probably a mistake on Bitnami's part because they told another programmer that the same thing happened to them.If it has happened to someone or has been able to solve it and can share the solution, it would be appreciated.
The question: I have a Kafka kraft in docker-compose inside a Ubuntu server. It works correctly in locally and I can for example, list topics:
~$ docker exec -it blopa-kafka-1 /bin/bash
I have no name!@8b166cb3a647:/$ kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --list
__consumer_offsets
boton_topic
dht_topic
gps_topic
pulso_topic
The problem is when trying to connect from another computer (on the same network) with the following code:
KAFKA_BOOTSTRAP_SERVERS = os.getenv('KAFKA_BOOTSTRAP_SERVERS', '192.168.1.211:9092')
app = FastAPI()
def ckeck_conectivity_kafka(kafka_bootstrap_servers):
kafka_host, kafka_port = kafka_bootstrap_servers.split(':')
try:
with socket.create_connection((kafka_host, int(kafka_port)), timeout=10) as sock:
except Exception as e:
error_message = f"[ERROR datos_kafka] Not conection {kafka_bootstrap_servers}: {e}"
logger.error(error_message))
check_conectivity_kafka(KAFKA_BOOTSTRAP_SERVERS)
@asynccontextmanager
async def lifespan(app: FastAPI):
global producer
try:
producer = AIOKafkaProducer(
bootstrap_servers=KAFKA_BOOTSTRAP_SERVERS
)
await producer.start()
except Exception as e:
await asyncio.sleep(5)
yield
return
yield
...
app.router.lifespan_context = lifespan
This is the trace I get when I run it the code:
INFO: Started server process [135789]
INFO: Waiting for application startup.
ERROR:aiokafka:Unable connect to node with id 0: [Errno -2] Name or service not known
KafkaConnectionError: No connection to node with id 0
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:8001 (Press CTRL+C to quit)
That is, it does not throw an error, the connection is made but there are problems with the name of service.
This is my .yaml
:
services:
kafka:
image: 'bitnami/kafka:latest'
environment:
- KAFKA_CFG_NODE_ID=0
- KAFKA_CFG_PROCESS_ROLES=controller,broker
- KAFKA_CFG_LISTENERS=PLAINTEXT://0.0.0.0:9092,CONTROLLER://:9093
- KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,PLAINTEXT>
- KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka:9093
- KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
- KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://192.168.1.211:9092
volumes:
- kafka-data:/bitnami/kafka/data
tty: true
restart: always
ports:
- "9092:9092"
- "9093:9093"
- "9094:9094"
volumes:
kafka-data:
driver: local
I have been able to connect with
telnet
to 192.168.1.211:9092 or nc
:
nc -zv 192.168.1.211 9092
192.168.1.211: inverse host lookup failed: Unknown host
(UNKNOWN) [192.168.1.211] 9092 (?) open
It does not know the service or the host but the port is open but I imagine that some problem in the configuration prevents my code from working.
It's probably not a difficult problem, but solutions such as "you'll need to advertise the external hostname/ip (e.g. 192.168.x.y) of the host as well as/in place of localhost. Requires Docker port forwarding (and firewall / security group changes)" have not worked for me. This is already done and it doesn't work. In fact, if it were not allowed to go through the firewall, the nc or telnet tests would not work.
I add nmap and kcat results:
sudo nmap -p 9092 192.168.1.211 -vv -sV --traceroute
Scanning 192.168.1.211 [1 port] Discovered open port 9092/tcp on 192.168.1.211 HOP RTT ADDRESS 1 53.84 ms 192.168.1.211
echo "Hello, Kafka!" | kcat -P -b 192.168.1.211:9092 -t boton_topic
%3|1727082503.291|FAIL|rdkafka#producer-1| [thrd:8b166cb3a647:9092/0]: 8b166cb3a647:9092/0: Failed to resolve '8b166cb3a647:9092': Name or service not known (after 102ms in state CONNECT) % ERROR: Local: Host resolution failure: 8b166cb3a647:9092/0: Failed to resolve '8b166cb3a647:9092': Name or service not known (after 102ms in state CONNECT) %3|1727082504.353|FAIL|rdkafka#producer-1| [thrd:8b166cb3a647:9092/0]: 8b166cb3a647:9092/0: Failed to resolve '8b166cb3a647:9092': Name or service not known (after 61ms in state CONNECT, 1 identical error(s) suppressed) % ERROR: Local: Host resolution failure: 8b166cb3a647:9092/0: Failed to resolve '8b166cb3a647:9092': Name or service not known (after 61ms in state CONNECT, 1 identical error(s) suppressed)
This is the docker-compose server information:
sudo lsof -i -P -n | grep LISTEN
docker-pr 1624 root 4u IPv4 9165 0t0 TCP *:9092 (LISTEN) docker-pr 1629 root 4u IPv6 9171 0t0 TCP *:9092 (LISTEN)
Upvotes: 0
Views: 48