• 카테고리

    질문 & 답변

  • 세부 분야

    데이터 엔지니어링

  • 해결 여부

    미해결

주제5 워크플로우 실행 에러

21.11.17 17:22 작성 조회수 567

0

Subject 5 - Workflow 실행시 3번째 하이브 쿼리문에서 에러가 발생합니다.

 

로그는 

2021-11-17 17:08:05,939 INFO org.apache.oozie.service.JPAService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[] No results found
2021-11-17 17:08:06,056 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@:start:] Start action [0000000-211117134747764-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2021-11-17 17:08:06,089 INFO org.apache.oozie.action.control.StartActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@:start:] Starting action
2021-11-17 17:08:06,122 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@:start:] [***0000000-211117134747764-oozie-oozi-W@:start:***]Action status=DONE
2021-11-17 17:08:06,132 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@:start:] [***0000000-211117134747764-oozie-oozi-W@:start:***]Action updated in DB!
2021-11-17 17:08:07,005 INFO org.apache.oozie.action.control.StartActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@:start:] Action ended with external status [OK]
2021-11-17 17:08:07,619 INFO org.apache.oozie.service.JPAService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@:start:] No results found
2021-11-17 17:08:07,684 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@:start:
2021-11-17 17:08:07,688 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W
2021-11-17 17:08:07,828 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] Start action [0000000-211117134747764-oozie-oozi-W@hive-537b] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2021-11-17 17:08:07,848 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] Starting action. Getting Action File System
2021-11-17 17:08:07,931 INFO org.apache.oozie.service.HadoopAccessorService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] Processing configuration file [/var/run/cloudera-scm-agent/process/274-oozie-OOZIE_SERVER/action-conf/default.xml] for action [default] and hostPort [*]
2021-11-17 17:08:07,966 INFO org.apache.oozie.service.HadoopAccessorService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] Processing configuration file [/var/run/cloudera-scm-agent/process/274-oozie-OOZIE_SERVER/action-conf/hive2.xml] for action [hive2] and hostPort [*]
2021-11-17 17:08:16,679 WARN org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] Invalid configuration value [null] defined for launcher max attempts count, using default [2].
2021-11-17 17:08:16,703 INFO org.apache.oozie.action.hadoop.YarnACLHandler: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] Not setting ACLs because mapreduce.cluster.acls.enabled is set to false
2021-11-17 17:08:20,280 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] checking action, hadoop job ID [application_1637124309648_0001] status [RUNNING]
2021-11-17 17:08:20,294 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] [***0000000-211117134747764-oozie-oozi-W@hive-537b***]Action status=RUNNING
2021-11-17 17:08:20,298 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] [***0000000-211117134747764-oozie-oozi-W@hive-537b***]Action updated in DB!
2021-11-17 17:08:20,312 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@hive-537b
2021-11-17 17:08:41,461 INFO org.apache.oozie.servlet.CallbackServlet: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] callback for action [0000000-211117134747764-oozie-oozi-W@hive-537b]
2021-11-17 17:08:41,990 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] action completed, external ID [application_1637124309648_0001]
2021-11-17 17:08:42,046 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] Action ended with external status [SUCCEEDED]
2021-11-17 17:08:42,469 INFO org.apache.oozie.service.JPAService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] No results found
2021-11-17 17:08:42,566 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] Start action [0000000-211117134747764-oozie-oozi-W@hive-6e14] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2021-11-17 17:08:42,592 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] Starting action. Getting Action File System
2021-11-17 17:08:47,110 WARN org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] Invalid configuration value [null] defined for launcher max attempts count, using default [2].
2021-11-17 17:08:47,113 INFO org.apache.oozie.action.hadoop.YarnACLHandler: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] Not setting ACLs because mapreduce.cluster.acls.enabled is set to false
2021-11-17 17:08:48,576 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] checking action, hadoop job ID [application_1637124309648_0002] status [RUNNING]
2021-11-17 17:08:48,585 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] [***0000000-211117134747764-oozie-oozi-W@hive-6e14***]Action status=RUNNING
2021-11-17 17:08:48,587 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] [***0000000-211117134747764-oozie-oozi-W@hive-6e14***]Action updated in DB!
2021-11-17 17:08:48,601 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@hive-6e14
2021-11-17 17:08:48,603 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@hive-537b
2021-11-17 17:10:36,799 INFO org.apache.oozie.servlet.CallbackServlet: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] callback for action [0000000-211117134747764-oozie-oozi-W@hive-6e14]
2021-11-17 17:10:37,091 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] External Child IDs  : [job_1637124309648_0003]
2021-11-17 17:10:37,099 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] action completed, external ID [application_1637124309648_0002]
2021-11-17 17:10:37,175 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] Action ended with external status [SUCCEEDED]
2021-11-17 17:10:37,296 INFO org.apache.oozie.service.JPAService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] No results found
2021-11-17 17:10:37,408 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] Start action [0000000-211117134747764-oozie-oozi-W@hive-6e91] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2021-11-17 17:10:37,425 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] Starting action. Getting Action File System
2021-11-17 17:10:42,219 WARN org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] Invalid configuration value [null] defined for launcher max attempts count, using default [2].
2021-11-17 17:10:42,222 INFO org.apache.oozie.action.hadoop.YarnACLHandler: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] Not setting ACLs because mapreduce.cluster.acls.enabled is set to false
2021-11-17 17:10:44,054 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] checking action, hadoop job ID [application_1637124309648_0004] status [RUNNING]
2021-11-17 17:10:44,060 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] [***0000000-211117134747764-oozie-oozi-W@hive-6e91***]Action status=RUNNING
2021-11-17 17:10:44,060 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] [***0000000-211117134747764-oozie-oozi-W@hive-6e91***]Action updated in DB!
2021-11-17 17:10:44,071 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@hive-6e91
2021-11-17 17:10:44,073 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@hive-6e14
2021-11-17 17:10:58,958 INFO org.apache.oozie.servlet.CallbackServlet: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] callback for action [0000000-211117134747764-oozie-oozi-W@hive-6e91]
2021-11-17 17:10:59,128 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] action completed, external ID [application_1637124309648_0004]
2021-11-17 17:10:59,132 WARN org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] Launcher ERROR, reason: Main Class [org.apache.oozie.action.hadoop.Hive2Main], exit code [2]
2021-11-17 17:10:59,174 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] Action ended with external status [FAILED/KILLED]
2021-11-17 17:10:59,186 INFO org.apache.oozie.command.wf.ActionEndXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] ERROR is considered as FAILED for SLA
2021-11-17 17:10:59,279 INFO org.apache.oozie.service.JPAService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] No results found
2021-11-17 17:10:59,329 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@Kill] Start action [0000000-211117134747764-oozie-oozi-W@Kill] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2021-11-17 17:10:59,342 INFO org.apache.oozie.action.control.KillActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@Kill] Starting action
2021-11-17 17:10:59,752 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@Kill] [***0000000-211117134747764-oozie-oozi-W@Kill***]Action status=DONE
2021-11-17 17:10:59,754 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@Kill] [***0000000-211117134747764-oozie-oozi-W@Kill***]Action updated in DB!
2021-11-17 17:10:59,780 INFO org.apache.oozie.action.control.KillActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@Kill] Action ended with external status [OK]
2021-11-17 17:10:59,925 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@Kill] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@Kill
2021-11-17 17:10:59,927 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W
2021-11-17 17:10:59,927 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@hive-6e91

Log Type: prelaunch.err

Log Upload Time: Wed Nov 17 17:11:00 +0900 2021

Log Length: 0

 
 

Log Type: prelaunch.out

Log Upload Time: Wed Nov 17 17:11:00 +0900 2021

Log Length: 70

Setting up env variables
Setting up job resources
Launching container

 

Log Type: stderr

Log Upload Time: Wed Nov 17 17:11:00 +0900 2021

Log Length: 4561

Showing 4096 bytes of 4561 total. Click here for the full log.

aticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show Log4j2 internal initialization logging.
Connecting to jdbc:hive2://server02.hadoop.com:10000/default
Connected to: Apache Hive (version 2.1.1-cdh6.3.2)
Driver: Hive JDBC (version 2.1.1-cdh6.3.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://server02.hadoop.com:10000/def> USE default;
INFO  : Compiling command(queryId=hive_20211117171057_9409a0f0-f0c7-4196-a475-a9290b9eccdb): USE default
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO  : Completed compiling command(queryId=hive_20211117171057_9409a0f0-f0c7-4196-a475-a9290b9eccdb); Time taken: 0.338 seconds
INFO  : Executing command(queryId=hive_20211117171057_9409a0f0-f0c7-4196-a475-a9290b9eccdb): USE default
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=hive_20211117171057_9409a0f0-f0c7-4196-a475-a9290b9eccdb); Time taken: 0.016 seconds
INFO  : OK
No rows affected (0.542 seconds)
0: jdbc:hive2://server02.hadoop.com:10000/def> 
0: jdbc:hive2://server02.hadoop.com:10000/def> insert overwrite local directory  
'/home/pilot-pjt/item-buy-list'
. . . . . . . . . . . . . . . . . . . . . . .> ROW FORMAT DELIMITED
. . . . . . . . . . . . . . . . . . . . . . .> FIELDS TERMINATED BY ','
. . . . . . . . . . . . . . . . . . . . . . .> select car_number, concat_ws(","  
, collect_set(item))
. . . . . . . . . . . . . . . . . . . . . . .> from managed_smartcar_item_buylis 
t_info
. . . . . . . . . . . . . . . . . . . . . . .> group by car_number
. . . . . . . . . . . . . . . . . . . . . . .> Error: Error while compiling statement: FAILED: RuntimeException Cannot create staging directory 'hdfs://server01.hadoop.com:8020/home/pilot-pjt/item-buy-list/.hive-staging_hive_2021-11-17_17-10-57_746_3560942864087896700-3': Permission denied: user=admin, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:400)
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:256)
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:194)
	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1855)
	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1839)
	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1798)
	at org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:61)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3101)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1123)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:696)
	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675) (state=42000,code=40000)
Closing: 0: jdbc:hive2://server02.hadoop.com:10000/default

 

Log Type: stdout

Log Upload Time: Wed Nov 17 17:11:00 +0900 2021

Log Length: 198282

Showing 4096 bytes of 198282 total. Click here for the full log.

 launch_container.sh
  jetty-jndi-9.3.25.v20180904.jar
  jersey-container-servlet-core-2.25.1.jar
  datanucleus-core-4.1.6.jar
  asm-tree-6.0.jar
------------------------


Script [hive-6e91.sql] content: 
------------------------
USE default;

insert overwrite local directory '/home/pilot-pjt/item-buy-list'
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
select car_number, concat_ws("," , collect_set(item))
from managed_smartcar_item_buylist_info
group by car_number

------------------------

Beeline command arguments :
             -u
             jdbc:hive2://server02.hadoop.com:10000/default
             -n
             admin
             -p
             DUMMY
             -d
             org.apache.hive.jdbc.HiveDriver
             -f
             hive-6e91.sql
             -a
             delegationToken
             --hiveconf
             mapreduce.job.tags=oozie-418ffadb1764d24a19ec7f0c02056574
             --hiveconf
             oozie.action.id=0000000-211117134747764-oozie-oozi-W@hive-6e91
             --hiveconf
             oozie.child.mapreduce.job.tags=oozie-418ffadb1764d24a19ec7f0c02056574
             --hiveconf
             oozie.action.rootlogger.log.level=INFO
             --hiveconf
             oozie.job.id=0000000-211117134747764-oozie-oozi-W
             --hiveconf
             oozie.HadoopAccessorService.created=true

Fetching child yarn jobs
tag id : oozie-418ffadb1764d24a19ec7f0c02056574
No child applications found
=================================================================

>>> Invoking Beeline command line now >>>


<<< Invocation of Beeline command completed <<<

No child hadoop job is executed.
java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:410)
	at org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:55)
	at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
	at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
	at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
	at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:141)
Caused by: java.lang.SecurityException: Intercepted System.exit(2)
	at org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
	at java.lang.Runtime.exit(Runtime.java:107)
	at java.lang.System.exit(System.java:971)
	at org.apache.oozie.action.hadoop.Hive2Main.runBeeline(Hive2Main.java:273)
	at org.apache.oozie.action.hadoop.Hive2Main.run(Hive2Main.java:250)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:104)
	at org.apache.oozie.action.hadoop.Hive2Main.main(Hive2Main.java:65)
	... 16 more
Intercepting System.exit(2)
Failing Oozie Launcher, Main Class [org.apache.oozie.action.hadoop.Hive2Main], exit code [2]
Oozie Launcher, uploading action data to HDFS sequence file: hdfs://server01.hadoop.com:8020/user/admin/oozie-oozi/0000000-211117134747764-oozie-oozi-W/hive-6e91--hive2/action-data.seq
Stopping AM
Callback notification attempts left 0
Callback notification trying http://server02.hadoop.com:11000/oozie/callback?id=0000000-211117134747764-oozie-oozi-W@hive-6e91&status=FAILED
Callback notification to http://server02.hadoop.com:11000/oozie/callback?id=0000000-211117134747764-oozie-oozi-W@hive-6e91&status=FAILED succeeded
Callback notification succeeded

이렇습니다.

 

무슨 문제일까요 ㅜㅜ

다른분 게시물에 나와있는 디렉토리 만들고 권한 변경은 이미 진행해보았지만 변화가 없었습니다.

course-thumbnail

15일간의 빅데이터 파일럿 프로젝트

6.탐색 파일럿 실행 4단계 - 07.주제5-운전자의 차량용품 구매이력 정보

강의실 바로가기

답변 1

답변을 작성해보세요.

0

Big.D님의 프로필

Big.D

지식공유자

2021.11.18

안녕하세요! 빅디 입니다.

로그 내용상으로 HDFS의 권한 문제로 보입니다.

우선 아래 2가지 확인 조치해 보세요~

1. "CM홈 > HDFS > 구성"에서 검색란에 "HDFS 권한 검사" 입력후, 해당기능 해제 되었는지 확인

2. hdfs inode = "/"의 supergroup 권한을 위해 Local(리눅스) 해당 group/계정 추가

  - Putty로 Server02 접속 

  - $ groupadd supergroup

  - $ usermod -a G supergroup admin

 

마지막으로 전체 클러스터 리스타트 하시면 됩니다.   - 빅디 드림

 

이 글과 비슷한 Q&A