response.getBody() 뭐 이런식의 메서드가 없어서요...getHeader()는 있는데.... body의 특정 문자열을 바꿔서 리턴하고자 해서 문의 해봅니다.

게이트 웨이를 도커에만 올리면 에러가 나옵니다 500 Server Error for HTTP POST "/user-service/login"io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /127.0.0.1:8889 유레카에서 유저 서비스 클릭 시 : /127.0.0.1:8889요렇게 나오는데 http://localhost:8888/user-service/default에서는{token.expireDate: 20002666,order_service.url: "http://order-service/order-service/%s/orders",order_service.exception.user_empty: "user's ord exist3333",user_service.url: "http://user-service/user",user_service.exception.user_empty: "user_Empty",token.secret: "1234"}gateway.host: "172.18.0.6",token.expireDate: 20002666,이렇게 나옵니다 http://localhost:8000/user-service/default이건 에러가 나구요도커의 네트워크는 추가 했구요"Containers": { "11a12dd45aaa3274226b1b462e1997b50e9b7ed61b405595c0c3b89393d6e036": { "Name": "rabbitmq", "EndpointID": "3425a23e5c021d2bcc307bed1d9b4ac03c17b92d4828f8b633469d8bbe8df8ac", "MacAddress": "02:42:ac:12:00:04", "IPv4Address": "172.18.0.4/16", "IPv6Address": "" }, "28da3474585596f6a3a435e48cfc9084e7d397fafaea751537ac7c82ac81bfd1": { "Name": "user-service1", "EndpointID": "543022aef70070d571f9d193151af692e59c5bc752feb15b5a0d75d23338ff34", "MacAddress": "02:42:ac:12:00:03", "IPv4Address": "172.18.0.3/16", }, "47dbfe92f425d933ca6e72018eb11b95f0cba48434be9b28f571570787db0d02": { "Name": "gateway-service", "EndpointID": "f3cfa36a9b73a2a925d62f0d8045d51dd8bdb4afe3b2926cae6a4997bac92114", "MacAddress": "02:42:ac:12:00:06", "IPv4Address": "172.18.0.6/16", "IPv6Address": "" }, "b4910d66fee36364691710a189c79f47c2a7fac7675166bbb102b6dfba83ef80": { "Name": "eureka", "EndpointID": "23d981be13ce1a1e7f1f4b85ea61553a13d73dfcf5af6039e6befa68d0cea497", "MacAddress": "02:42:ac:12:00:02", "IPv4Address": "172.18.0.2/16", "IPv6Address": "" }, "e5326aa441bdcf8133c66384251212b14d1d0a452291a462a667de21e3f23c6c": { "Name": "config", "EndpointID": "5f8de35288b1d51927f840e050203671ff0c880b0b8f39c2dd614055c49306bc", "MacAddress": "02:42:ac:12:00:05", "IPv4Address": "172.18.0.5/16", "IPv6Address": "" } 어디가 문제일까요?

안녕하세요! 좋은 강의 제공해주셔서 감사합니다. 다름이 아니라, 수업에서 JWT 토큰 유효성 로직을 api gateway service에 구현했는데, user-service가 아닌 gateway service에 구현한 이유가 있을까요? api gateway에 토큰 유효성 로직을 구현하게 되면, 만료된 토큰으로 api gateway를 거치지 않고 바로 user-service에 요청하는 경우, 토큰 유효성 검사를 하지 않아서 문제가 발생할 수 있지 않나요?

안녕하세요 선생님 가상머신 우분투에 올려서 터미널로 프로듀싱한 레코드를스프링에서 컨슈밍해서 보려고하는데 터미널 컨슈머에서는 정상으로 레코드를 받아오는데스프링에서는 브로커에 닿지 않는 것 같은데 혹시 서버 프로퍼티 문제일까요? 어떻게 하면 브로커에 연결이 될까요 ?    package com.example; import com.fasterxml.jackson.core.JsonProcessingException; import com.fasterxml.jackson.core.type.TypeReference; import org.apache.kafka.clients.consumer.ConsumerConfig; import org.apache.kafka.clients.consumer.ConsumerRecord; import org.apache.kafka.clients.consumer.ConsumerRecords; import org.apache.kafka.clients.consumer.KafkaConsumer; import org.apache.kafka.common.serialization.StringDeserializer; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import com.fasterxml.jackson.databind.ObjectMapper; import java.time.Duration; import java.util.*; public class SimpleConsumer { private final static Logger logger = LoggerFactory.getLogger(SimpleConsumer.class); private final static String TOPIC_NAME = "test"; private final static String BOOTSTRAP_SERVERS = "localhost:9092"; private final static String GROUP_ID = "test-group"; public static void main(String[] args) { Properties configs = new Properties(); configs.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, BOOTSTRAP_SERVERS); configs.put(ConsumerConfig.GROUP_ID_CONFIG, GROUP_ID); configs.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); configs.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); KafkaConsumer<String, String> consumer = new KafkaConsumer<>(configs); consumer.subscribe(Arrays.asList(TOPIC_NAME)); while (true) { ConsumerRecords<String, String> records = consumer.poll(Duration.ofSeconds(1)); for (ConsumerRecord<String, String> record : records) { logger.info("record:{}", record); String message = record.value(); System.out.println(message); } } }  구동시 에러[main] INFO org.apache.kafka.clients.consumer.ConsumerConfig - ConsumerConfig values: allow.auto.create.topics = true auto.commit.interval.ms = 5000 auto.offset.reset = latest bootstrap.servers = [localhost:9092] check.crcs = true client.dns.lookup = default client.id = client.rack = connections.max.idle.ms = 540000 default.api.timeout.ms = 60000 enable.auto.commit = true exclude.internal.topics = true fetch.max.bytes = 52428800 fetch.max.wait.ms = 500 fetch.min.bytes = 1 group.id = test-group group.instance.id = null heartbeat.interval.ms = 3000 interceptor.classes = [] internal.leave.group.on.close = true isolation.level = read_uncommitted key.deserializer = class org.apache.kafka.common.serialization.StringDeserializer max.partition.fetch.bytes = 1048576 max.poll.interval.ms = 300000 max.poll.records = 500 metadata.max.age.ms = 300000 metric.reporters = [] metrics.num.samples = 2 metrics.recording.level = INFO metrics.sample.window.ms = 30000 partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor] receive.buffer.bytes = 65536 reconnect.backoff.max.ms = 1000 reconnect.backoff.ms = 50 request.timeout.ms = 30000 retry.backoff.ms = 100 sasl.client.callback.handler.class = null sasl.jaas.config = null sasl.kerberos.kinit.cmd = /usr/bin/kinit sasl.kerberos.min.time.before.relogin = 60000 sasl.kerberos.service.name = null sasl.kerberos.ticket.renew.jitter = 0.05 sasl.kerberos.ticket.renew.window.factor = 0.8 sasl.login.callback.handler.class = null sasl.login.class = null sasl.login.refresh.buffer.seconds = 300 sasl.login.refresh.min.period.seconds = 60 sasl.login.refresh.window.factor = 0.8 sasl.login.refresh.window.jitter = 0.05 sasl.mechanism = GSSAPI security.protocol = PLAINTEXT security.providers = null send.buffer.bytes = 131072 session.timeout.ms = 10000 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2] ssl.endpoint.identification.algorithm = https ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLSv1.2 ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version: 2.5.0 [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId: 66563e712b0b9f84 [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka startTimeMs: 1679530659001 [main] INFO org.apache.kafka.clients.consumer.KafkaConsumer - [Consumer clientId=consumer-test-group-1, groupId=test-group] Subscribed to topic(s): test [main] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-test-group-1, groupId=test-group] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. [main] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-test-group-1, groupId=test-group] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected [main] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-test-group-1, groupId=test-group] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. [main] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-test-group-1, groupId=test-group] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected [main] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-test-group-1, groupId=test-group] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. [main] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-test-group-1, groupId=test-group] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected [main] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-test-group-1, groupId=test-group] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. [main] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-test-group-1, groupId=test-group] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected [main] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-test-group-1, groupId=test-group] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. [main] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-test-group-1, groupId=test-group] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected  server properties   -----------------------------------------------------++ 로컬 터미널에서는 스프링에 정상적으로 연동이 되는데가상머신에서 띄운 터미널은 로컬터미널과 같은 토픽인데도 컨슈밍을 하지 않습니다(스프링에도 가상머신에 띄운 프로듀서 레코드 전송은 안되는데 로컬 터미널 프로듀서에서 보낸 것은 받음 ) 로컬 가상머신위와 같이 동시에 각각 터미널에 같은 토픽, 부트스트랩서버로 보냈는데 각각 터미널에서만 통신이 되는 것 같습니다   public class SimpleConsumer { private final static Logger logger = LoggerFactory.getLogger(SimpleConsumer.class); private final static String TOPIC_NAME = "study"; private final static String BOOTSTRAP_SERVERS = "localhost:9092"; private final static String GROUP_ID = "test-group"; public static void main(String[] args) { Properties configs = new Properties(); configs.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, BOOTSTRAP_SERVERS); configs.put(ConsumerConfig.GROUP_ID_CONFIG, GROUP_ID); configs.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); configs.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); KafkaConsumer<String, String> consumer = new KafkaConsumer<>(configs); consumer.subscribe(Arrays.asList(TOPIC_NAME)); while (true) { ConsumerRecords<String, String> records = consumer.poll(Duration.ofSeconds(1)); for (ConsumerRecord<String, String> record : records) { logger.info("record:{}", record); String message = record.value(); System.out.println("message : " + message); } } } }[main] INFO com.example.SimpleConsumer - record:ConsumerRecord(topic = study, partition = 0, leaderEpoch = 0, offset = 3, CreateTime = 1679534086551, serialized key size = 1, serialized value size = 3, headers = RecordHeaders(headers = [], isReadOnly = false), key = 4, value = aaa) message : aaa [main] INFO com.example.SimpleConsumer - record:ConsumerRecord(topic = study, partition = 0, leaderEpoch = 0, offset = 4, CreateTime = 1679534345840, serialized key size = 1, serialized value size = 3, headers = RecordHeaders(headers = [], isReadOnly = false), key = 5, value = 555) message : 555 위에 터미널 프로듀서로 보낸 값만 넘어옵니다  이상입니다감사합니다.

안녕하세요.빅디님의 책과 강의로 열심히 따라가고 있는데 여러 업무를 병행하다보니 강의의 반도 진행하지 못했습니다. 강의를 들으면 정말 제가 알지 못했던 분야를 너무 쉽게 알아가는 기쁨에 가슴이 뛰는데 업무에 치이면서 살다보니 어느덧 강의 만료일이 다가 왔습니다.혹시 기간 연장을 조금 더 해주실수 있다면 부탁드리겠습니다.이 강의는 꼭 완료하고 싶습니다.부탁드리겠습니다. 감사합니다.

안녕하세요~ 강의 항상 잘듣고있습니다ㅎㅎ배치사이즈를 작게하면 성능이 낮아지고배치사이즈를 크게하면 ack유실시 중복처리가 많아져서배치사이즈를 크게하고ack(commit)을 전체가 아닌 일부분만 할 수 없을까? 생각하게되었어요.예를들어서 1000개 배치로 가져오고10개까지 commit, 100개까지커밋, ....요런것도 가능할려나요?.?

안녕하세요. 강의를 수강하던 중 이해하기 힘든 현상을 발견되어 이렇게 질문드립니다.Jwt는 SigningKey로 복호화가 가능하다면, 유효한 토큰이다.(유효한 토큰이다란 기준이 여러 개 있습니다만) 라고 알고 있는데요. 아래와 같은 현상을 발견하게되서 질문하게 되었습니다.(현재 Gateway server - token.secret = A)(현재 User service - token.secret = A)1.User-service 회원생성2.User-service 로그인 (응답에서 jwt토큰 복사)로그인 시 User-service의 token.secret을 SigningKey로 jwt 토큰 발급3.Gateway의 AuthorizationHeaderFilter를 통과하는 user-service/actuator/health 호출 (response, 200)여기까지는 정상입니다만4.ecommerce.yml의 token.secret 변경 및 커밋5.Gateway의 /refresh 호출액츄에이터에 의한 token.secret 변경 (현재 Gateway server - token.secret = B)(현재 User service - token.secret = A)(현재 JWT 토큰의 SigningKey = A)6.Gateway의 AuthorizationHeaderFilter를 통과하는 user-service/actuator/health 호출 (response, 200) 여기가 질문 포인트입니다. AuthorizationHeaderFilter의 isJwtValid의 코드를 보면subject = Jwts.parser().setSigningKey(env.getProperty("token.secret")) .parseClaimsJws(jwt).getBody() .getSubject();이런 구문이 있습니다. Jwt 생성 당시의 Siginingkey랑 다른 signingkey로 복호화하면 오류가 나면서 인가에 실패하는 것을 기대했습니다. 하지만 왜 정상적으로 200을 반환하는지 이해가 잘 되지 않습니다. (Gateway의 token.secret은 B로 변경됨을 로그로 확인했습니다. ) 

안녕하세요.먼저 좋은 수업을 강의 해주셔서 감사합니다!다름이 아니라 제가 해당 수업 진행 중 오류가 있어서 문의드립니다!mysql 콘솔에서는 오류가 발생 하지 않았으나,H2 Console 에서 테이블을 조회시 SELECT * FROM ORDERS;(conn=42) Table 'mydb.ORDERS' doesn't exist 42S02/1146 이러한 에러가 발생하는데 원인이 무엇일까요??order-service 에서 조회시에는 문제가 없습니다..

마이크로 서비스 사용간 (RestTemplate 혹은 Feign) 에서 Eureka 의 서비스 명으로 USER-SERVICE 같은 마이크로 서비스명을 통하여 직접 호출하면 결론적으로는 GATEWAY-SERVICE 를 통하여 사용했을때의 장점을 잃어버리는 것 같은데, 섹션 10 에서는 게이트웨이를 통하지 않고 직접 서비스를 호출하는 이유가 있을까요?

안녕하세요 지금까지 강의 잘 따라오고 있었는데요 spark ssh 접속하여 shell 실행 후 smartcar_master_df 생성까지 잘 되었는데 smartcar_master_df 조회를 하면 아래와 같은 오류가 발생합니다. 구글링해서 찾아봐도 해결이 안되는데 도움 부탁드립니다. >> 오류메세지 >> WARN cluster.YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

안녕하세요. 업무를 병행하면서 학습을 하기 쉽지가 않네요... 혹시 연장 신청이 가능할지 궁금해서 문의 남깁니다. 가능하다면 꼭 좀 부탁드리겠습니다. 감사합니다.

안녕하세요.선생님 강의를 듣고 kafka connect 매커니즘에 대해 상세하게 알게 됐습니다.다만, 실무에 적용을 하는 도중 문제에 봉착해 도움을 구하고자 문의드립니다.현재 debezium mysql connector를 사용하여 source 데이터는 topic으로 저장하는데 성공하였지만,해당 데이터를 mongodb 에 저장하는데 저장/업데이트는 정상적으로 되지만 delete 시 반영이 안되는 문제가 있습니다.RDB와는 다르게 mongodb sink connector는 insert.mode는 지원하지 않고write model Strategy 를 활용하는 걸로 보이는데,아래와 같이 sink connector를 설정할 경우 account_id 를 key 로 해서 업데이트는 가능한데, 삭제는 안되네요? "connector.class": "com.mongodb.kafka.connect.MongoSinkConnector", "document.id.strategy":"com.mongodb.kafka.connect.sink.processor.id.strategy.PartialValueStrategy", "document.id.strategy.partial.value.projection.list":"account_id", "document.id.strategy.partial.value.projection.type":"AllowList", "writemodel.strategy":"com.mongodb.kafka.connect.sink.writemodel.strategy.ReplaceOneBusinessKeyStrategy",혹시 Source 에서 입력, 업데이트, 삭제를 mongodb에 반영하려면 어떻게 해야 되는지 알 수 있을까요?감사합니다. 

MSA에서 다른 어플리케이션의 데이터를 가져오는 방법으로 RestTemplate, FeignClient를 사용하는 방법을 알려주셨는데요.데이터를 가져오는 것은 보통 rest 통신을 사용하여 가져오나요?실무에서 카프카나 다른 라이브러리를 사용해서 가져오는지, 아니면 설명해주신 것 처럼 rest 통신을 통해 가져오는지 궁금합니다~

안녕하세요. 선생님Confluent AVRO Install 하고 나서 sink connect 등록하고 상태를 확인 해보니 아래와 같이 에러가 발생 했습니다.ubuntu@ip-172-31-0-29:~$ curl --location --request POST 'http://localhost:8083/connectors' \--header 'Content-Type: application/json' \--data-raw '{"name": "mysql-sink-connector","config": {"connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector","tasks.max": "1","topics": "source_mysqldb.hesdp_mgr.TB_ME_TERMS_BAS","connection.url": "jdbc:mysql://lg-mysql:3306/aurora2","connection.user": "sink","connection.password": "sink1234","table.name.format": "aurora2.TB_ME_TERMS_BAS","insert.mode": "upsert","pk.fields": "TERMS_VER_ID","pk.mode": "record_key","delete.enabled": "true","key.converter": "io.confluent.connect.avro.AvroConverter","value.converter": "io.confluent.connect.avro.AvroConverter","key.converter.schema.registry.url": "http://lg-schema01:8081","value.converter.schema.registry.url": "http://lg-schema01:8081"}}'{"name":"mysql-sink-connector","config":{"connector.class":"io.confluent.connect.jdbc.JdbcSinkConnector","tasks.max":"1","topics":"source_mysqldb.hesdp_mgr.TB_ME_TERMS_BAS","connection.url":"jdbc:mysql://lg-mysql:3306/aurora2","connection.user":"sink","connection.password":"sink1234","table.name.format":"aurora2.TB_ME_TERMS_BAS","insert.mode":"upsert","pk.fields":"TERMS_VER_ID","pk.mode":"record_key","delete.enabled":"true","key.converter":"io.confluent.connect.avro.AvroConverter","value.converter":"io.confluent.connect.avro.AvroConverter","key.converter.schema.registry.url":"http://lg-schema01:8081","value.converter.schema.registry.url":"http://lg-schema01:8081","name":"mysql-sink-connector"},"tasks":[],"type":"sink"} ubuntu@ip-172-31-0-29:~$ curl -X GET http://localhost:8083/connectors/mysql-sink-connector/status | jq '.'% Total % Received % Xferd Average Speed Time Time Time CurrentDload Upload Total Spent Left Speed100 1898 100 1898 0 0 31616 0 --:--:-- --:--:-- --:--:-- 32169{"name": "mysql-sink-connector","connector": {"state": "RUNNING","worker_id": "172.31.13.238:8083"},"tasks": [{"id": 0,"state": "FAILED","worker_id": "172.31.13.238:8083","trace": "java.lang.NoClassDefFoundError: com/google/common/base/Ticker\n\tat io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.<init>(CachedSchemaRegistryClient.java:175)\n\tat io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.<init>(CachedSchemaRegistryClient.java:158)\n\tat io.confluent.kafka.schemaregistry.client.SchemaRegistryClientFactory.newClient(SchemaRegistryClientFactory.java:36)\n\tat io.confluent.connect.avro.AvroConverter.configure(AvroConverter.java:68)\n\tat org.apache.kafka.connect.runtime.isolation.Plugins.newConverter(Plugins.java:297)\n\tat org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:640)\n\tat org.apache.kafka.connect.runtime.Worker.startSinkTask(Worker.java:544)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:1703)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$getTaskStartingCallable$31(DistributedHerder.java:1753)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\nCaused by: java.lang.ClassNotFoundException: com.google.common.base.Ticker\n\tat java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)\n\tat java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)\n\tat org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:103)\n\tat java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)\n\t... 13 more\n"}],"type": "sink"} 참고로 스키마 레지스토리 서버가 잘 붙나 확인 해본 결과 잘 붙습니다.ubuntu@ip-172-31-0-29:~$ curl -v lg-schema01:8081* Trying 172.31.46.33:8081...* Connected to lg-schema01 (172.31.46.33) port 8081 (#0)> GET / HTTP/1.1> Host: lg-schema01:8081> User-Agent: curl/7.81.0> Accept: />* Mark bundle as not supporting multiuse< HTTP/1.1 200 OK< Date: Mon, 13 Mar 2023 06:14:56 GMT< Content-Type: application/vnd.schemaregistry.v1+json< Vary: Accept-Encoding, User-Agent< Content-Length: 2<* Connection #0 to host lg-schema01 left intact{}ubuntu@ip-172-31-0-29:~$ 현재 제가 구성한 환경 정보 입니다.Apache Zookeeper 3.8.1 3개Apache Kafka 2.13-3.3.2 3개=> Source connect : Debezium mysql connect 2.1.2=> Sink connect : Confluent JDBC Connector 10.6.3=> Avro : Confluent Avro converter 7.3Confluent Schema registry 7.3.0 1개mysql DB 1 개 (Source 와 Sink 는 user 로 분리해서 데이터 넣는 방식으로 테스트 중입니다) 구글링 검색 결과 Guava 를 못찾아서 그런거 같다는데, 제가 설치한 환경의 디펜던시가 문제일까요?      

미러메이커2를 사용할 때 각 카프카의 버전이 달라도 사용이 가능한가요? 예를 들어서 하나의 클러스터는 2.2버전이고 하나의 클러스터는 3.1 버전이라고 했을 때2.2버전은 미러메이커2를 지원하지 않는데 3.1 버전의 클러스터에서 미러메이커2를 실행하면 2.2버전에서 3.1버전으로 데이터가 가져와지나요...?

안녕하세요 강사님 좋은 강의 감사합니다. 파티션 할당 전략 중 CooperativesStickyAssignor 전략 실습중에 에러가 발생하였습니다 에러 내용은 두번째 컨슈머를 기동하면서 첫번째 컨슈머 콘솔에서"Request joining group due to: group is already rebalancing" 라는 에러가 발생하면서 리밸런싱이 완료되지 않아 어느 컨슈머에도 파티션이 할당되지 않는 문제가 발생합니다. 해당 에러는  타임아웃이 나기전까지 반복해서 발생하며 이후에는 두 컨슈머 모두 종료됩니다  해결방법좀 부탁드립니다ㅠ.ㅠ

안녕하세요. 선생님 수업을 잘 듣고 있습니다.일단 선생님이 알려주신 부분하고 다르게 하고 있는게 있어서 환경 부터 어떻게 했는지 말씀 드릴게요.일단 kafka는 Confluent kafka 가 아닌 apache kafka 로 설치 했습니다.EC2 에서 총 8개의 인스턴스를 구축 했으며Apache Zookeeper 3.8.1 3개Apache Kafka 2.13-3.3.2 3개=> Source connect : Debezium mysql connect 2.1.2=> Sink connect : Confluent JDBC Connector 10.6.3Confluent Schema registry 7.3.0 1개mysql DB 1 개 (Source 와 Sink 는 user 로 분리해서 데이터 넣는 방식으로 테스트 중입니다)위와 같이 구성을 해서 구축을 했습니다.테스트는 json 방식으로 Source 와 sink 쪽에 각각 connect 정보 등록 해서 데이터 전달 되는것까지 통과 했습니다.Confluent Schema registry 는 설치 하고 카프카서버에서 스키마 서버 8081포트 가 뜨는것도 확인 했습니다.Json 방식으로 Schema registry 로 데이터 전달 되는것도 문제 없었습니다. 문제점. 수업에 있는 Avro 로 connect 등록 할려고 하니 바로 에러가 납니다.{"error_code":500,"message":"Error trying to forward REST request: Connector configuration is invalid and contains the following 2 error(s):\nInvalid value io.apicurio.registry.utils.converter.AvroConverter for configuration key.converter: Class io.apicurio.registry.utils.converter.AvroConverter could not be found.\nInvalid value io.apicurio.registry.utils.converter.AvroConverter for configuration value.converter: Class io.apicurio.registry.utils.converter.AvroConverter could not be found.\nYou can also find the above list of errors at the endpoint /connector-plugins/{connectorType}/config/validate"} 위와 같은 에러에 대해 일단 조치한 점은kafka 의 connect-distributed.properties 에서 설정한 "plugin.path" 에Kafka Connect Avro Converter(https://www.confluent.io/hub/confluentinc/kafka-connect-avro-converter)를 압축 풀어서 두었습니다. connect 서버 재 가동 후, 아래 명령어로 Plugin 확인 하였는데 mysql connect 나 jdbc Source connect , jdbc sink connect 는 뜨는데 "avro" 관련된 plugin 이 보이지 않았습니다."curl --location --request GET 'localhost:8083/connector-plugins' | jq '.'" 그래서 위 에러가 Avro 를 못찾는다는 이유가, kafka에 avro 를 설정을 못한건지?Confluent Schema registry 에 avro 문제 인건지? 아니면 제가 Confluent kafka 가 아닌 apache kafka 로 설치 해서 인지 알수가 있을까요?   

안녕하세요 강의를 들으면서 cdc를 익히고 있는 수강생입니다.해당 강의 내용을 바탕으로 oracle --> kafka --> oracle 로 cdc 구성을 하고 있는데요. 구성 중 질문이 있어 남기게 되었습니다.A라는 테이블에 대해 source connector 와 sink connector 를 하나만 생성 했을 때는데이터 변경 시 정상적으로 빠르게 작동을 하는데요.추가적으로 B라는 테이블 source connector 와 sink connector 를 생성하면 A,B 모두 데이터 변경이 엄청 느립니다. 대략 5-10분 정도 뒤에 변경이 되는데요 원인이 무엇일까요? (sink 쪽에서 느립니다.)참고로 debezium 버전은 2.1 입니다.1.9에서는 최초 source connector 생성시 스냅샷 시간이 별로 걸리지 않았는데요, 2.1에서는 스냅샷 시간이 상당히 늘어났습니다. 혹시 다른 옵션이 있을까요? 매뉴얼에서는 찾지 못해 질문 드립니다."auto.evolve": "true" 옵션 사용 시 source 에서 컬럼을 추가하면 sink 에서도 생성이 되는 것은 확인했는데 source 에서 다시 컬럼을 삭제하면 sink 에서는 삭제가 안되는데요. 추가 해야 될 옵션이 있을까요? 아니면 아직 컬럼 삭제에 대한 지원이 되지 않는 것일 까요??참고로 제가 작성한 json 내용은 아래와 같습니다.#########################################################################################A 테이블 sourcecurl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '{"name": "source_connector_af_claim","config" : {"connector.class" : "io.debezium.connector.oracle.OracleConnector","db_type":"oracle","tasks.max" : "1", "database.server.name" : "source_connector_01_af_claim","database.user" : "xxxxx","database.password" : "xxxxx","database.url": "jdbc:oracle:thin:@10.74.102.218:1521:aaa","database.dbname" : "aaa","database.out.server.name":"ora_source_af_claim_out", "schema.history.internal.kafka.bootstrap.servers" : "10.74.105.61:9092","schema.history.internal.kafka.topic": "ora_source_af_claim_history", "include.schema.changes": "true","database.connection.adapter": "logminer","topic.prefix": "ORA_SOURCE_01", "schema.include.list": "xxxxx","table.include.list":"xxxxx.AF_CLAIM","include.schema.changes": "true", "auto.evolve": "true","time.precision.mode": "connect", "key.converter": "org.apache.kafka.connect.json.JsonConverter","value.converter": "org.apache.kafka.connect.json.JsonConverter", "snapshot.mode" : "initial", "tombstones.on.delete": "true", "transforms": "rename_topic","transforms.rename_topic.type" : "org.apache.kafka.connect.transforms.RegexRouter","transforms.rename_topic.regex" : "ORA_SOURCE_01(.*)","transforms.rename_topic.replacement" : "source_$1", "transforms": "unwrap","transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState","transforms.unwrap.drop.tombstones": "false" }}'#########################################################################################A 테이블 sinkcurl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '{"name": "sink_connector_af-claim","config": {"connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector","tasks.max": "1","table.name.format": "AF_CLAIM_TEST","topics": "ORA_SOURCE_01.xxxxx.AF_CLAIM", "connection.url": "jdbc:oracle:thin:@10.74.102.218:1521:aaa","connection.user": "xxxxx","connection.password": "xxxxx", "auto.evolve": "true","insert.mode": "upsert","delete.enabled": "true", "key.converter": "org.apache.kafka.connect.json.JsonConverter","value.converter": "org.apache.kafka.connect.json.JsonConverter", "table.include.list" : "xxxxx.AF_CLAIM","pk.fields": "AF_CLAIM_GROUP_NO , AF_CLAIM_NO","pk.mode": "record_key" }}'#########################################################################################B 테이블 sourcecurl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '{"name": "source_connector_af_cs_mgmt","config" : {"connector.class" : "io.debezium.connector.oracle.OracleConnector","db_type":"oracle","tasks.max" : "1", "database.server.name" : "source_connector_02_af_cs_mgmt","database.user" : "xxxxx","database.password" : "xxxxx","database.url": "jdbc:oracle:thin:@10.74.102.218:1521:aaa","database.dbname" : "aaa","database.out.server.name":"ora_source_af_cs_mgmt_out", "schema.history.internal.kafka.bootstrap.servers" : "10.74.105.61:9092","schema.history.internal.kafka.topic": "ora_source_af_cs_mgmt_history", "schema.include.list": "xxxxx","include.schema.changes": "true","database.connection.adapter": "logminer","topic.prefix": "ORA_SOURCE_02","table.include.list":"xxxxx.AF_CS_MGMT","include.schema.changes": "true", "auto.evolve": "true","time.precision.mode": "connect", "key.converter": "org.apache.kafka.connect.json.JsonConverter","value.converter": "org.apache.kafka.connect.json.JsonConverter", "snapshot.mode" : "schema_only", "tombstones.on.delete": "true", "transforms": "rename_topic","transforms.rename_topic.type" : "org.apache.kafka.connect.transforms.RegexRouter","transforms.rename_topic.regex" : "ORA_SOURCE_02(.*)","transforms.rename_topic.replacement" : "source_$1", "transforms": "unwrap","transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState","transforms.unwrap.drop.tombstones": "false"  }}'#########################################################################################B 테이블 sinkcurl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '{"name": "sink-connector_af_cs_mgmt","config": {"connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector","tasks.max": "1","table.name.format": "AF_CS_MGMT_TEST","topics": "ORA_SOURCE_02.xxxxx.AF_CS_MGMT", "connection.url": "jdbc:oracle:thin:@10.74.102.218:1521:aaa","connection.user": "xxxxx","connection.password": "xxxxx", "auto.evolve": "true","insert.mode": "upsert","delete.enabled": "true", "key.converter": "org.apache.kafka.connect.json.JsonConverter","value.converter": "org.apache.kafka.connect.json.JsonConverter", "table.include.list" : "xxxxx.AF_CS_MGMT","pk.fields": "AFCR_NO","pk.mode": "record_key" }}'   

워크플로우는 받은 파일 그대로 넣어서 다음과 같습니다.  working_day car-batch-log가 다음과 같습니다.  에러메시지는 이렇고 Main Class [org.apache.oozie.action.hadoop.Hive2Main], exit code [2] 로그는 다음과 같습니다.2023-03-10 10:42:47,347 INFO org.apache.oozie.service.JPAService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[] No results found 2023-03-10 10:42:47,393 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@:start:] Start action [0000001-230310103158623-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2023-03-10 10:42:47,397 INFO org.apache.oozie.action.control.StartActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@:start:] Starting action 2023-03-10 10:42:47,638 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@:start:] [***0000001-230310103158623-oozie-oozi-W@:start:***]Action status=DONE 2023-03-10 10:42:47,639 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@:start:] [***0000001-230310103158623-oozie-oozi-W@:start:***]Action updated in DB! 2023-03-10 10:42:47,656 INFO org.apache.oozie.action.control.StartActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@:start:] Action ended with external status [OK] 2023-03-10 10:42:47,691 INFO org.apache.oozie.service.JPAService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@:start:] No results found 2023-03-10 10:42:47,718 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000001-230310103158623-oozie-oozi-W@:start: 2023-03-10 10:42:47,723 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000001-230310103158623-oozie-oozi-W 2023-03-10 10:42:47,769 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-02d7] Start action [0000001-230310103158623-oozie-oozi-W@hive-02d7] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2023-03-10 10:42:47,772 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-02d7] Starting action. Getting Action File System 2023-03-10 10:42:52,218 WARN org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-02d7] Invalid configuration value [null] defined for launcher max attempts count, using default [2]. 2023-03-10 10:42:52,218 INFO org.apache.oozie.action.hadoop.YarnACLHandler: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-02d7] Not setting ACLs because mapreduce.cluster.acls.enabled is set to false 2023-03-10 10:42:52,922 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-02d7] checking action, hadoop job ID [application_1678411853306_0005] status [RUNNING] 2023-03-10 10:42:52,933 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-02d7] [***0000001-230310103158623-oozie-oozi-W@hive-02d7***]Action status=RUNNING 2023-03-10 10:42:52,933 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-02d7] [***0000001-230310103158623-oozie-oozi-W@hive-02d7***]Action updated in DB! 2023-03-10 10:42:52,941 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-02d7] No Notification URL is defined. Therefore nothing to notify for job 0000001-230310103158623-oozie-oozi-W@hive-02d7 2023-03-10 10:43:01,536 INFO org.apache.oozie.servlet.CallbackServlet: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-02d7] callback for action [0000001-230310103158623-oozie-oozi-W@hive-02d7] 2023-03-10 10:43:01,961 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-02d7] action completed, external ID [application_1678411853306_0005] 2023-03-10 10:43:01,997 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-02d7] Action ended with external status [SUCCEEDED] 2023-03-10 10:43:02,041 INFO org.apache.oozie.service.JPAService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-02d7] No results found 2023-03-10 10:43:02,095 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-b93d] Start action [0000001-230310103158623-oozie-oozi-W@hive-b93d] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2023-03-10 10:43:02,100 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-b93d] Starting action. Getting Action File System 2023-03-10 10:43:04,515 WARN org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-b93d] Invalid configuration value [null] defined for launcher max attempts count, using default [2]. 2023-03-10 10:43:04,515 INFO org.apache.oozie.action.hadoop.YarnACLHandler: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-b93d] Not setting ACLs because mapreduce.cluster.acls.enabled is set to false 2023-03-10 10:43:05,956 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-b93d] checking action, hadoop job ID [application_1678411853306_0006] status [RUNNING] 2023-03-10 10:43:05,978 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-b93d] [***0000001-230310103158623-oozie-oozi-W@hive-b93d***]Action status=RUNNING 2023-03-10 10:43:05,978 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-b93d] [***0000001-230310103158623-oozie-oozi-W@hive-b93d***]Action updated in DB! 2023-03-10 10:43:05,999 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-b93d] No Notification URL is defined. Therefore nothing to notify for job 0000001-230310103158623-oozie-oozi-W@hive-b93d 2023-03-10 10:43:06,003 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-02d7] No Notification URL is defined. Therefore nothing to notify for job 0000001-230310103158623-oozie-oozi-W@hive-02d7 2023-03-10 10:44:00,881 INFO org.apache.oozie.servlet.CallbackServlet: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-b93d] callback for action [0000001-230310103158623-oozie-oozi-W@hive-b93d] 2023-03-10 10:44:01,141 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-b93d] External Child IDs : [job_1678411853306_0007] 2023-03-10 10:44:01,145 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-b93d] action completed, external ID [application_1678411853306_0006] 2023-03-10 10:44:01,207 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-b93d] Action ended with external status [SUCCEEDED] 2023-03-10 10:44:01,271 INFO org.apache.oozie.service.JPAService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-b93d] No results found 2023-03-10 10:44:01,589 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-d326] Start action [0000001-230310103158623-oozie-oozi-W@hive-d326] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2023-03-10 10:44:01,592 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-d326] Starting action. Getting Action File System 2023-03-10 10:44:04,228 WARN org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-d326] Invalid configuration value [null] defined for launcher max attempts count, using default [2]. 2023-03-10 10:44:04,230 INFO org.apache.oozie.action.hadoop.YarnACLHandler: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-d326] Not setting ACLs because mapreduce.cluster.acls.enabled is set to false 2023-03-10 10:44:05,032 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-d326] checking action, hadoop job ID [application_1678411853306_0008] status [RUNNING] 2023-03-10 10:44:05,036 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-d326] [***0000001-230310103158623-oozie-oozi-W@hive-d326***]Action status=RUNNING 2023-03-10 10:44:05,036 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-d326] [***0000001-230310103158623-oozie-oozi-W@hive-d326***]Action updated in DB! 2023-03-10 10:44:05,042 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-d326] No Notification URL is defined. Therefore nothing to notify for job 0000001-230310103158623-oozie-oozi-W@hive-d326 2023-03-10 10:44:05,042 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-b93d] No Notification URL is defined. Therefore nothing to notify for job 0000001-230310103158623-oozie-oozi-W@hive-b93d 2023-03-10 10:44:13,249 INFO org.apache.oozie.servlet.CallbackServlet: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-d326] callback for action [0000001-230310103158623-oozie-oozi-W@hive-d326] 2023-03-10 10:44:13,716 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-d326] action completed, external ID [application_1678411853306_0008] 2023-03-10 10:44:13,727 WARN org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-d326] Launcher ERROR, reason: Main Class [org.apache.oozie.action.hadoop.Hive2Main], exit code [2] 2023-03-10 10:44:13,751 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-d326] Action ended with external status [FAILED/KILLED] 2023-03-10 10:44:13,754 INFO org.apache.oozie.command.wf.ActionEndXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-d326] ERROR is considered as FAILED for SLA 2023-03-10 10:44:13,788 INFO org.apache.oozie.service.JPAService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-d326] No results found 2023-03-10 10:44:13,815 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@Kill] Start action [0000001-230310103158623-oozie-oozi-W@Kill] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2023-03-10 10:44:13,827 INFO org.apache.oozie.action.control.KillActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@Kill] Starting action 2023-03-10 10:44:13,831 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@Kill] [***0000001-230310103158623-oozie-oozi-W@Kill***]Action status=DONE 2023-03-10 10:44:13,831 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@Kill] [***0000001-230310103158623-oozie-oozi-W@Kill***]Action updated in DB! 2023-03-10 10:44:13,859 INFO org.apache.oozie.action.control.KillActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 2 - Workflow] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@Kill] Action ended with external status [OK] 2023-03-10 10:44:14,218 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@Kill] No Notification URL is defined. Therefore nothing to notify for job 0000001-230310103158623-oozie-oozi-W@Kill 2023-03-10 10:44:14,218 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000001-230310103158623-oozie-oozi-W 2023-03-10 10:44:14,218 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-230310103158623-oozie-oozi-W] ACTION[0000001-230310103158623-oozie-oozi-W@hive-d326] No Notification URL is defined. Therefore nothing to notify for job 0000001-230310103158623-oozie-oozi-W@hive-d326추가로 다른 질문에서 본 해결책으로 권한 설정을 하니 이렇게 뜹니다.   

안녕하세요 선생님 양질의 강의 너무 감사하게 잘듣고 있습니다.다름이아니라 강의를 듣던 중 궁금한 부분이 있어서 글을 남기게 되었습니다.Broker로 부터 통신을 한 후에 pendingCompletion(ConcurrentLinkedQueue) 에 response를 적재를 하고적재된 pendingCompletion 기준으로 Fetcher가 데이터를 가져오게 되는데 이 과정에서 Fetcher는 별도의 스레드로 동작하는지요?"Consumer Fetcher관련 주요 파라미터와 Fetcher 메커니즘의 이해: 13:22"