Flume에서 배치 처리중 오류 발생
미해결
15일간의 빅데이터 파일럿 프로젝트
스마트카 상태정보(SmartCarStatusInfo_20160101.txt)를 car-batch-log/ 로 복사해서 Flume이 Hadoop에 저장하는 처리중 에러가 발생합니다. 어디를 잘못 했을까요? 에러 내용 -------------------------------------------------------------------------------------- 2021-05-04 08:51:25,195 INFO org.apache.flume.node.AbstractConfigurationProvider: Created channel DriverCarInfo_Channel 2021-05-04 08:51:25,195 INFO org.apache.flume.channel.DefaultChannelFactory: Creating instance of channel SmartCarInfo_Channel type memory 2021-05-04 08:51:25,203 INFO org.apache.flume.node.AbstractConfigurationProvider: Created channel SmartCarInfo_Channel 2021-05-04 08:51:25,205 INFO org.apache.flume.source.DefaultSourceFactory: Creating instance of source SmartCarInfo_SpoolSource, type spooldir 2021-05-04 08:51:25,234 ERROR org.apache.flume.node.PollingPropertiesFileConfigurationProvider: Unhandled error java.lang.NoSuchMethodError: org.apache.flume.Context.getSubProperties(Ljava/lang/String;)Ljava/util/Map; at org.apache.flume.source.SpoolDirectorySource.configure(SpoolDirectorySource.java:176) at org.apache.flume.conf.Configurables.configure(Configurables.java:41) at org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:325) at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:105) at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:145) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
- 플럼
- flume
- bigdata
- Hadoop
- Flume
- 데이터 엔지니어링
- redis
- Impala
- Kafka
- ZooKeeper