스쿱 에러....

안녕하세요 스쿱에서 자꾸 에러가 발생하여 문의드립니다.

 

전 과정까지는 문제없이 진행하였으나, 

 

sqoop export --connect jdbc:postgresql://127.0.0.1:7432/postgres --username cloudera-scm --password acjZt03D1Q --table smartcar_symptom_info --export-dir /user/hive/warehouse/managed_smartcar_symptom_info

 

위 명령어를 진행하는데 계속 exception이 발생합니다... 원인도 불분명하여 일단 질문 남깁니다...

sqoop export --connect jdbc:postgresql://127.0.0.1:7432/postgres --username cloudera-scm --password vsqBur5ygT --table smartcar_symptom_info --export-dir /user/                hive/warehouse/managed_smartcar_symptom_info

Warning: /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.

Please set $ACCUMULO_HOME to the root of your Accumulo installation.

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

22/04/14 21:11:43 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7-cdh6.3.2

22/04/14 21:11:43 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

22/04/14 21:11:43 INFO manager.SqlManager: Using default fetchSize of 1000

22/04/14 21:11:43 INFO tool.CodeGenTool: Beginning code generation

22/04/14 21:11:44 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "smartcar_symptom_info" AS t LIMIT 1

22/04/14 21:11:44 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce

22/04/14 21:11:49 ERROR orm.CompilationManager: Could not rename /tmp/sqoop-root/compile/6f489f0c3f57d16ddd9b99ee38d3c452/smartcar_symptom_info.java to /root/./smartcar_symptom_in                fo.java. Error: Destination '/root/./smartcar_symptom_info.java' already exists

22/04/14 21:11:49 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/6f489f0c3f57d16ddd9b99ee38d3c452/smartcar_symptom_info.jar

22/04/14 21:11:50 INFO mapreduce.ExportJobBase: Beginning export of smartcar_symptom_info

22/04/14 21:11:50 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar

22/04/14 21:11:52 WARN mapreduce.ExportJobBase: IOException checking input file header: java.io.EOFException

22/04/14 21:11:52 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative

22/04/14 21:11:52 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative

22/04/14 21:11:52 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps

22/04/14 21:11:52 INFO client.RMProxy: Connecting to ResourceManager at server01.hadoop.com/192.168.56.101:8032

22/04/14 21:11:53 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /user/root/.staging/job_1649933706082_0003

22/04/14 21:11:54 WARN hdfs.DataStreamer: Caught exception

java.lang.InterruptedException

        at java.lang.Object.wait(Native Method)

        at java.lang.Thread.join(Thread.java:1252)

        at java.lang.Thread.join(Thread.java:1326)

        at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:986)

        at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:640)

        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:810)

22/04/14 21:11:54 WARN hdfs.DataStreamer: Caught exception

java.lang.InterruptedException

        at java.lang.Object.wait(Native Method)

        at java.lang.Thread.join(Thread.java:1252)

        at java.lang.Thread.join(Thread.java:1326)

        at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:986)

        at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:640)

        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:810)

22/04/14 21:11:54 WARN hdfs.DataStreamer: Caught exception

java.lang.InterruptedException

        at java.lang.Object.wait(Native Method)

        at java.lang.Thread.join(Thread.java:1252)

        at java.lang.Thread.join(Thread.java:1326)

        at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:986)

        at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:640)

        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:810)

22/04/14 21:11:54 WARN hdfs.DataStreamer: Caught exception

java.lang.InterruptedException

        at java.lang.Object.wait(Native Method)

        at java.lang.Thread.join(Thread.java:1252)

        at java.lang.Thread.join(Thread.java:1326)

        at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:986)

        at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:640)

        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:810)

22/04/14 21:11:55 WARN hdfs.DataStreamer: Caught exception

java.lang.InterruptedException

        at java.lang.Object.wait(Native Method)

        at java.lang.Thread.join(Thread.java:1252)

        at java.lang.Thread.join(Thread.java:1326)

        at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:986)

        at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:640)

        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:810)

22/04/14 21:11:55 WARN hdfs.DataStreamer: Caught exception

java.lang.InterruptedException

        at java.lang.Object.wait(Native Method)

        at java.lang.Thread.join(Thread.java:1252)

        at java.lang.Thread.join(Thread.java:1326)

        at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:986)

        at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:640)

        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:810)

22/04/14 21:11:55 WARN hdfs.DataStreamer: Caught exception

java.lang.InterruptedException

        at java.lang.Object.wait(Native Method)

        at java.lang.Thread.join(Thread.java:1252)

        at java.lang.Thread.join(Thread.java:1326)

        at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:986)

        at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:640)

        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:810)

22/04/14 21:11:55 WARN hdfs.DataStreamer: Caught exception

java.lang.InterruptedException

        at java.lang.Object.wait(Native Method)

        at java.lang.Thread.join(Thread.java:1252)

        at java.lang.Thread.join(Thread.java:1326)

        at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:986)

        at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:640)

        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:810)

22/04/14 21:11:55 WARN hdfs.DataStreamer: Caught exception

java.lang.InterruptedException

        at java.lang.Object.wait(Native Method)

        at java.lang.Thread.join(Thread.java:1252)

        at java.lang.Thread.join(Thread.java:1326)

        at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:986)

        at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:640)

        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:810)

22/04/14 21:11:56 WARN hdfs.DataStreamer: Caught exception

java.lang.InterruptedException

        at java.lang.Object.wait(Native Method)

        at java.lang.Thread.join(Thread.java:1252)

        at java.lang.Thread.join(Thread.java:1326)

        at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:986)

        at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:640)

        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:810)

22/04/14 21:11:57 WARN hdfs.DataStreamer: Caught exception

java.lang.InterruptedException

        at java.lang.Object.wait(Native Method)

        at java.lang.Thread.join(Thread.java:1252)

        at java.lang.Thread.join(Thread.java:1326)

        at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:986)

        at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:640)

        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:810)

22/04/14 21:11:57 WARN hdfs.DataStreamer: Caught exception

java.lang.InterruptedException

        at java.lang.Object.wait(Native Method)

        at java.lang.Thread.join(Thread.java:1252)

        at java.lang.Thread.join(Thread.java:1326)

        at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:986)

        at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:640)

        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:810)

22/04/14 21:11:58 WARN hdfs.DataStreamer: Caught exception

java.lang.InterruptedException

        at java.lang.Object.wait(Native Method)

        at java.lang.Thread.join(Thread.java:1252)

        at java.lang.Thread.join(Thread.java:1326)

        at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:986)

        at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:640)

        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:810)

22/04/14 21:11:58 WARN hdfs.DataStreamer: Caught exception

java.lang.InterruptedException

        at java.lang.Object.wait(Native Method)

        at java.lang.Thread.join(Thread.java:1252)

        at java.lang.Thread.join(Thread.java:1326)

        at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:986)

        at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:640)

        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:810)

22/04/14 21:11:58 INFO input.FileInputFormat: Total input files to process : 2

22/04/14 21:11:58 INFO input.FileInputFormat: Total input files to process : 2

22/04/14 21:11:58 INFO mapreduce.JobSubmitter: number of splits:5

22/04/14 21:11:58 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative

22/04/14 21:11:58 INFO Configuration.deprecation: yarn.resourcemanager.system-metrics-publisher.enabled is deprecated. Instead, use yarn.system-metrics-publisher.enabled

22/04/14 21:11:58 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1649933706082_0003

22/04/14 21:11:58 INFO mapreduce.JobSubmitter: Executing with tokens: []

22/04/14 21:11:59 INFO conf.Configuration: resource-types.xml not found

22/04/14 21:11:59 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.

22/04/14 21:11:59 INFO impl.YarnClientImpl: Submitted application application_1649933706082_0003

22/04/14 21:11:59 INFO mapreduce.Job: The url to track the job: http://server01.hadoop.com:8088/proxy/application_1649933706082_0003/

22/04/14 21:11:59 INFO mapreduce.Job: Running job: job_1649933706082_0003

22/04/14 21:12:10 INFO mapreduce.Job: Job job_1649933706082_0003 running in uber mode : false

22/04/14 21:12:10 INFO mapreduce.Job:  map 0% reduce 0%

22/04/14 21:12:31 INFO mapreduce.Job:  map 100% reduce 0%

22/04/14 21:12:32 INFO mapreduce.Job: Job job_1649933706082_0003 failed with state FAILED due to: Task failed task_1649933706082_0003_m_000000

Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0

 

22/04/14 21:12:32 INFO mapreduce.Job: Counters: 12

        Job Counters

                Failed map tasks=2

                Killed map tasks=3

                Launched map tasks=4

                Data-local map tasks=4

                Total time spent by all maps in occupied slots (ms)=74538

                Total time spent by all reduces in occupied slots (ms)=0

                Total time spent by all map tasks (ms)=74538

                Total vcore-milliseconds taken by all map tasks=74538

                Total megabyte-milliseconds taken by all map tasks=76326912

        Map-Reduce Framework

                CPU time spent (ms)=0

                Physical memory (bytes) snapshot=0

                Virtual memory (bytes) snapshot=0

22/04/14 21:12:32 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead

22/04/14 21:12:32 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 40.4452 seconds (0 bytes/sec)

22/04/14 21:12:32 INFO mapreduce.ExportJobBase: Exported 0 records.

22/04/14 21:12:32 ERROR mapreduce.ExportJobBase: Export job failed!

22/04/14 21:12:32 ERROR tool.ExportTool: Error during export:

Export job failed!

        at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:444)

        at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:930)

        at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:93)

        at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:112)

        at org.apache.sqoop.Sqoop.run(Sqoop.java:146)

        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)

        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:182)

        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:233)

        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:242)

        at org.apache.sqoop.Sqoop.main(Sqoop.java:251)