• 카테고리

    질문 & 답변

  • 세부 분야

    데이터 엔지니어링

  • 해결 여부

    미해결

SimpleProducer 실행 시 Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected 오류

22.09.24 21:27 작성 조회수 1.6k

0

안녕하세요 데브 원영님!

강의를 수강하며 예제를 실행 중에 오류를 마주쳤는데 구글링을 해도 해결이 되지 않아 문의 남깁니다.

import java.util.Properties;

public class SimpleProducer {
    private final static Logger logger = LoggerFactory.getLogger(SimpleProducer.class);
    private final static String TOPIC_NAME = "test";
    private final static String BOOTSTRAP_SERVERS = "localhost:9092";

    public static void main(String[] args) {

        Properties configs = new Properties();
        configs.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, BOOTSTRAP_SERVERS);
        configs.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        configs.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());

        KafkaProducer<String, String> producer = new KafkaProducer<>(configs);

        String messageValue = "testMessage";
        ProducerRecord<String, String> record = new ProducerRecord<>(TOPIC_NAME, messageValue);
        producer.send(record);
        logger.info("{}", record);
        producer.flush();
        producer.close();
    }
}
[main] INFO org.apache.kafka.clients.producer.ProducerConfig - ProducerConfig values: 
	acks = 1
	batch.size = 16384
	bootstrap.servers = [localhost:9092]
	buffer.memory = 33554432
	client.dns.lookup = default
	client.id = producer-1
	compression.type = none
	connections.max.idle.ms = 540000
	delivery.timeout.ms = 120000
	enable.idempotence = false
	interceptor.classes = []
	key.serializer = class org.apache.kafka.common.serialization.StringSerializer
	linger.ms = 0
	max.block.ms = 60000
	max.in.flight.requests.per.connection = 5
	max.request.size = 1048576
	metadata.max.age.ms = 300000
	metadata.max.idle.ms = 300000
	metric.reporters = []
	metrics.num.samples = 2
	metrics.recording.level = INFO
	metrics.sample.window.ms = 30000
	partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
	receive.buffer.bytes = 32768
	reconnect.backoff.max.ms = 1000
	reconnect.backoff.ms = 50
	request.timeout.ms = 30000
	retries = 2147483647
	retry.backoff.ms = 100
	sasl.client.callback.handler.class = null
	sasl.jaas.config = null
	sasl.kerberos.kinit.cmd = /usr/bin/kinit
	sasl.kerberos.min.time.before.relogin = 60000
	sasl.kerberos.service.name = null
	sasl.kerberos.ticket.renew.jitter = 0.05
	sasl.kerberos.ticket.renew.window.factor = 0.8
	sasl.login.callback.handler.class = null
	sasl.login.class = null
	sasl.login.refresh.buffer.seconds = 300
	sasl.login.refresh.min.period.seconds = 60
	sasl.login.refresh.window.factor = 0.8
	sasl.login.refresh.window.jitter = 0.05
	sasl.mechanism = GSSAPI
	security.protocol = PLAINTEXT
	security.providers = null
	send.buffer.bytes = 131072
	ssl.cipher.suites = null
	ssl.enabled.protocols = [TLSv1.2]
	ssl.endpoint.identification.algorithm = https
	ssl.key.password = null
	ssl.keymanager.algorithm = SunX509
	ssl.keystore.location = null
	ssl.keystore.password = null
	ssl.keystore.type = JKS
	ssl.protocol = TLSv1.2
	ssl.provider = null
	ssl.secure.random.implementation = null
	ssl.trustmanager.algorithm = PKIX
	ssl.truststore.location = null
	ssl.truststore.password = null
	ssl.truststore.type = JKS
	transaction.timeout.ms = 60000
	transactional.id = null
	value.serializer = class org.apache.kafka.common.serialization.StringSerializer

[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version: 2.5.0
[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId: 66563e712b0b9f84
[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka startTimeMs: 1664022671614
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
[kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected

Process finished with exit code 130

BOOTSTRAP_SERVER를 my-kafka로 실행하면 아예 오류가 떠버려서 localhost로 바꿔서 진행하였습니다.

에러 내용은 다음과 같습니다.

server.properites 내용은 다음과 같습니다.

제가 놓치고 있는 부분이 있다면 알려주세요! 감사합니다.:)

course-thumbnail

[아파치 카프카 애플리케이션 프로그래밍] 개념부터 컨슈머, 프로듀서, 커넥트, 스트림즈까지!

답변 2

·

답변을 작성해보세요.

1

asdqweasdzxcasd님의 프로필

asdqweasdzxcasd

2023.07.19

저도 잘은 모르지만, 댓글 남깁니다...
저는 WSL2에서 Kafka를 실행했을 때, 같은 오류를 겪었어요...
wsl2 는 windows와는 별개의 가상 머신으로 실행되기 때문에 wsl2에서 localhost로 접근하면 wsl2 가상머신을 가리킨다고 하네요. 그리고 windows에서 localhost로 접근하면 windows를 가리키기 때문에 wsl2에는 접근할 수 없다고 하네요... (아닐 수도 있습니다.)

그래서 wsl2의 kafka 브로커에 연결하려면 ipv6 주소를 통해 내부 가상 머신을 카리키는 [::1] 을 사용하는 방법과, wsl2의 ip주소를 사용하는 방법이 있다고 하더라고요...

1. [::1] 사용 (IPv6)

1)
vi config/server.properties 파일에서
listeners=PLAINTEXT://[::1]:9092 로 수정
advertised.listeners=PLAINTEXT://[::1]:9092 로 수정

2)
WSL2 터미널에서 consumer 확인 시, my-kafka 대신 [::1] 입력
bin/kafka-console-consumer.sh --bootstrap-server [::1]:9092 --topic test --from-beginning

3)
Intellij Code에서 BOOTSTRAP_SERVERS 의 ip [::1] 로 수정
private final static String BOOTSTRAP_SERVERS = "[::1]:9092";

2. WSL2 IP 주소 사용

1)
WSL2 터미널에서 [ ip addr | grep "eth0" ] 명령어로 ip 확인하면,
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000

2)
inet [님 wsl2 ip 주소]/20 brd 172.24.79.255 scope global eth0
라는 메시지가 뜸니다.

3)
그러면 server.properties에 들어가서 listeners와 advertised.listeners를 수정해줍니다.
# listeners=PLAINTEXT://:9092
# advertised.listeners=PLAINTEXT://[님 wsl2 ip 주소]:9092

WSL2 터미널에서 consumer 는 --bootstrap-server my-kafka:9092 그대로 실행해도 됩니다.


4)
인텔리제이 코드에서 BOOTSTRAP_SERVERS 의 ip 를 [님 wsl2 ip 주소]로 수정해줍니다.
private final static String BOOTSTRAP_SERVERS = "[님 wsl2 ip 주소]:9092";

그런데 wsl2 ip주소는 aws처럼 컴퓨터 재부팅 할 때마다 바뀌어서 컴터 킬 때마다 수정해줘야 할 수도 있다고 하네요.

0

데브원영 DVWY님의 프로필

데브원영 DVWY

지식공유자

2022.09.27

안녕하세요 최지수님!

보내주신 내용으로는 정상적으로 통신되어야 할것으로 보입니다. kafka-topics.sh와 같은 명령어로 토픽 생성, 리스트 확인이 가능한지 확인하시거나, 브로커 로그를 확인하셔서 정상적으로 브로커가 실행되었는지를 우선적으로 체크하셔야 될것으로 보이네요.

이 글과 비슷한 Q&A