안녕하세요. oracle source connector 생성 후 topic 만들 시 발생하는 오류 질문입니다.

Question

안녕하세요. debezium oracle connector를 생성하고 topic 만들 시에 발생하는 오류에 관련하여 질문 있습니다. 우선 오류는 [2023-05-07 20:39:26,049] INFO WorkerSourceTask{id=oracle_connector-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:505) [2023-05-07 20:39:26,049] WARN Couldn't commit processed log positions with the source database due to a concurrent connector shutdown or restart (io.debezium.connector.common.BaseSourceTask:243) [2023-05-07 20:39:31,455] WARN Couldn't resolve server kafka:9092 from bootstrap.servers as DNS resolution failed for kafka (org.apache.kafka.clients.ClientUtils:75) [2023-05-07 20:39:31,457] INFO [Producer clientId=UK-dbhistory] Closing the Kafka producer with timeoutMillis = 0 ms. (org.apache.kafka.clients.producer.KafkaProducer:1189) [2023-05-07 20:39:31,457] INFO WorkerSourceTask{id=oracle_connector-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask:488) [2023-05-07 20:39:31,458] INFO WorkerSourceTask{id=oracle_connector-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:505) [2023-05-07 20:39:31,459] ERROR WorkerSourceTask{id=oracle_connector-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:191) org.apache.kafka.common.KafkaException: Failed to construct kafka producer at org.apache.kafka.clients.producer.KafkaProducer.( KafkaProducer.java:441 ) at org.apache.kafka.clients.producer.KafkaProducer.( KafkaProducer.java:301 ) at io.debezium.relational.history.KafkaDatabaseHistory.start( KafkaDatabaseHistory.java:235 ) at io.debezium.relational.HistorizedRelationalDatabaseSchema.( HistorizedRelationalDatabaseSchema.java:40 ) at io.debezium.connector.oracle .OracleDatabaseSchema.( OracleDatabaseSchema.java:35 ) at io.debezium.connector.oracle .OracleConnectorTask.start( OracleConnectorTask.java:55 ) at io.debezium.connector.common.BaseSourceTask.start( BaseSourceTask.java:106 ) at org.apache.kafka.connect.runtime.WorkerSourceTask.execute( WorkerSourceTask.java:232 ) at org.apache.kafka.connect.runtime.WorkerTask.doRun( WorkerTask.java:189 ) at org.apache.kafka.connect.runtime.WorkerTask.run ( WorkerTask.java:239 ) at java.util.concurrent.Executors$RunnableAdapter.call( Executors.java:511 ) at java.util.concurrent.FutureTask.run ( FutureTask.java:266 ) at java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1149 ) at java.util.concurrent.ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:624 ) at java.lang.Thread.run ( Thread.java:750 ) Caused by: org.apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers at org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses( ClientUtils.java:89 ) at org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses( ClientUtils.java:48 ) at org.apache.kafka.clients.producer.KafkaProducer.( KafkaProducer.java:415 ) ... 14 more [2023-05-07 20:39:31,461] ERROR WorkerSourceTask{id=oracle_connector-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:192) [2023-05-07 20:39:31,461] INFO Stopping down connector (io.debezium.connector.common.BaseSourceTask:192) [2023-05-07 20:39:31,462] WARN Could not stop task (org.apache.kafka.connect.runtime.WorkerSourceTask:175) java.lang.NullPointerException at io.debezium.connector.oracle .OracleConnectorTask.doStop( OracleConnectorTask.java:129 ) at io.debezium.connector.common.BaseSourceTask.stop( BaseSourceTask.java:206 ) at io.debezium.connector.common.BaseSourceTask.stop( BaseSourceTask.java:176 ) at org.apache.kafka.connect.runtime.WorkerSourceTask.close( WorkerSourceTask.java:173 ) at org.apache.kafka.connect.runtime.WorkerTask.doClose( WorkerTask.java:168 ) at org.apache.kafka.connect.runtime.WorkerTask.doRun( WorkerTask.java:195 ) at org.apache.kafka.connect.runtime.WorkerTask.run ( WorkerTask.java:239 ) at java.util.concurrent.Executors$RunnableAdapter.call( Executors.java:511 ) at java.util.concurrent.FutureTask.run ( FutureTask.java:266 ) at java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1149 ) at java.util.concurrent.ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:624 ) at java.lang.Thread.run ( Thread.java:750 ) 이렇게 발생하는데 topic은 생성되지 않고 connector만 생성되었습니다. kafka task를 삭제하지 못하고 있다는데, 어떻게 해야하나요? connector를 삭제하고 다시 생성해도 동일한 오류가 발생합니다. -oracle connector 설정 파일입니다- { "name": "oracle_connector", "config": { "connector.class": " io.debezium.connector.oracle .OracleConnector", "key.converter": "org.apache.kafka.connect.json.JsonConverter", "key.converter.schemas.registry.url": " http://0.0.0.0:8081 ", "database.hostname": "mydb", "database.port": "port", "database.user": "user", "database.password": "userpwd", "database.dbname": "dbname", "topic.prefix": "test_", "tasks.max": "1", "database.history.kafka.bootstrap.servers": "kafka:9092", "database.history.kafka.topic": "dbhistory.testdb", " database.server.name ": "servername", " database.out.server.name ": "servername" } }

권 철민 · Answer

안녕하십니까, connector 설정 파일에 db connection 정보가 제대로 되어 있는거 맞나요? 올려 주신 설정 파일에는 아래와 같이 그냥 port, user과 같이 되어 있고, 더구나 db 접속 ip address도 없습니다. "database.hostname": "mydb", "database.port": "port", "database.user": "user", "database.password": "userpwd", "database.dbname": "dbname", 그리고 아래는 key converter가 Schema registry url도 제대로 잡혀 있지 않고, JSonConverter로 되어 있습니다. "key.converter": "org.apache.kafka.connect.json.JsonConverter", "key.converter.schemas.registry.url": " http://0.0.0.0:8081 ", 강의 실습이 mysql로 되어있지만, 강의 내용을 참조하셔서 오라클도 적용해 보시려는 것은 알겠습니다만, 제 도움을 받으시려면 환경 파일을 다시 한번 강의 내용을 보시고 재 점검 해주시고, 정확한 값을 올려 주셨으면 합니다. 감사합니다.