강의

N
챌린지

멘토링

N
클립

로드맵

인프런 커뮤니티 질문&답변

유기욱님의 프로필 이미지
유기욱

작성한 질문수

15일간의 빅데이터 파일럿 프로젝트

6.탐색 파일럿 실행 4단계 - 07.주제5-운전자의 차량용품 구매이력 정보

주제5 워크플로우 실행 에러

작성

·

783

0

Subject 5 - Workflow 실행시 3번째 하이브 쿼리문에서 에러가 발생합니다.

 

로그는 

2021-11-17 17:08:05,939 INFO org.apache.oozie.service.JPAService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[] No results found
2021-11-17 17:08:06,056 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@:start:] Start action [0000000-211117134747764-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2021-11-17 17:08:06,089 INFO org.apache.oozie.action.control.StartActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@:start:] Starting action
2021-11-17 17:08:06,122 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@:start:] [***0000000-211117134747764-oozie-oozi-W@:start:***]Action status=DONE
2021-11-17 17:08:06,132 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@:start:] [***0000000-211117134747764-oozie-oozi-W@:start:***]Action updated in DB!
2021-11-17 17:08:07,005 INFO org.apache.oozie.action.control.StartActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@:start:] Action ended with external status [OK]
2021-11-17 17:08:07,619 INFO org.apache.oozie.service.JPAService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@:start:] No results found
2021-11-17 17:08:07,684 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@:start:
2021-11-17 17:08:07,688 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W
2021-11-17 17:08:07,828 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] Start action [0000000-211117134747764-oozie-oozi-W@hive-537b] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2021-11-17 17:08:07,848 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] Starting action. Getting Action File System
2021-11-17 17:08:07,931 INFO org.apache.oozie.service.HadoopAccessorService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] Processing configuration file [/var/run/cloudera-scm-agent/process/274-oozie-OOZIE_SERVER/action-conf/default.xml] for action [default] and hostPort [*]
2021-11-17 17:08:07,966 INFO org.apache.oozie.service.HadoopAccessorService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] Processing configuration file [/var/run/cloudera-scm-agent/process/274-oozie-OOZIE_SERVER/action-conf/hive2.xml] for action [hive2] and hostPort [*]
2021-11-17 17:08:16,679 WARN org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] Invalid configuration value [null] defined for launcher max attempts count, using default [2].
2021-11-17 17:08:16,703 INFO org.apache.oozie.action.hadoop.YarnACLHandler: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] Not setting ACLs because mapreduce.cluster.acls.enabled is set to false
2021-11-17 17:08:20,280 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] checking action, hadoop job ID [application_1637124309648_0001] status [RUNNING]
2021-11-17 17:08:20,294 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] [***0000000-211117134747764-oozie-oozi-W@hive-537b***]Action status=RUNNING
2021-11-17 17:08:20,298 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] [***0000000-211117134747764-oozie-oozi-W@hive-537b***]Action updated in DB!
2021-11-17 17:08:20,312 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@hive-537b
2021-11-17 17:08:41,461 INFO org.apache.oozie.servlet.CallbackServlet: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] callback for action [0000000-211117134747764-oozie-oozi-W@hive-537b]
2021-11-17 17:08:41,990 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] action completed, external ID [application_1637124309648_0001]
2021-11-17 17:08:42,046 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] Action ended with external status [SUCCEEDED]
2021-11-17 17:08:42,469 INFO org.apache.oozie.service.JPAService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] No results found
2021-11-17 17:08:42,566 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] Start action [0000000-211117134747764-oozie-oozi-W@hive-6e14] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2021-11-17 17:08:42,592 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] Starting action. Getting Action File System
2021-11-17 17:08:47,110 WARN org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] Invalid configuration value [null] defined for launcher max attempts count, using default [2].
2021-11-17 17:08:47,113 INFO org.apache.oozie.action.hadoop.YarnACLHandler: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] Not setting ACLs because mapreduce.cluster.acls.enabled is set to false
2021-11-17 17:08:48,576 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] checking action, hadoop job ID [application_1637124309648_0002] status [RUNNING]
2021-11-17 17:08:48,585 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] [***0000000-211117134747764-oozie-oozi-W@hive-6e14***]Action status=RUNNING
2021-11-17 17:08:48,587 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] [***0000000-211117134747764-oozie-oozi-W@hive-6e14***]Action updated in DB!
2021-11-17 17:08:48,601 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@hive-6e14
2021-11-17 17:08:48,603 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-537b] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@hive-537b
2021-11-17 17:10:36,799 INFO org.apache.oozie.servlet.CallbackServlet: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] callback for action [0000000-211117134747764-oozie-oozi-W@hive-6e14]
2021-11-17 17:10:37,091 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] External Child IDs  : [job_1637124309648_0003]
2021-11-17 17:10:37,099 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] action completed, external ID [application_1637124309648_0002]
2021-11-17 17:10:37,175 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] Action ended with external status [SUCCEEDED]
2021-11-17 17:10:37,296 INFO org.apache.oozie.service.JPAService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] No results found
2021-11-17 17:10:37,408 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] Start action [0000000-211117134747764-oozie-oozi-W@hive-6e91] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2021-11-17 17:10:37,425 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] Starting action. Getting Action File System
2021-11-17 17:10:42,219 WARN org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] Invalid configuration value [null] defined for launcher max attempts count, using default [2].
2021-11-17 17:10:42,222 INFO org.apache.oozie.action.hadoop.YarnACLHandler: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] Not setting ACLs because mapreduce.cluster.acls.enabled is set to false
2021-11-17 17:10:44,054 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] checking action, hadoop job ID [application_1637124309648_0004] status [RUNNING]
2021-11-17 17:10:44,060 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] [***0000000-211117134747764-oozie-oozi-W@hive-6e91***]Action status=RUNNING
2021-11-17 17:10:44,060 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] [***0000000-211117134747764-oozie-oozi-W@hive-6e91***]Action updated in DB!
2021-11-17 17:10:44,071 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@hive-6e91
2021-11-17 17:10:44,073 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e14] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@hive-6e14
2021-11-17 17:10:58,958 INFO org.apache.oozie.servlet.CallbackServlet: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] callback for action [0000000-211117134747764-oozie-oozi-W@hive-6e91]
2021-11-17 17:10:59,128 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] action completed, external ID [application_1637124309648_0004]
2021-11-17 17:10:59,132 WARN org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] Launcher ERROR, reason: Main Class [org.apache.oozie.action.hadoop.Hive2Main], exit code [2]
2021-11-17 17:10:59,174 INFO org.apache.oozie.action.hadoop.Hive2ActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] Action ended with external status [FAILED/KILLED]
2021-11-17 17:10:59,186 INFO org.apache.oozie.command.wf.ActionEndXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] ERROR is considered as FAILED for SLA
2021-11-17 17:10:59,279 INFO org.apache.oozie.service.JPAService: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] No results found
2021-11-17 17:10:59,329 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@Kill] Start action [0000000-211117134747764-oozie-oozi-W@Kill] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2021-11-17 17:10:59,342 INFO org.apache.oozie.action.control.KillActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@Kill] Starting action
2021-11-17 17:10:59,752 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@Kill] [***0000000-211117134747764-oozie-oozi-W@Kill***]Action status=DONE
2021-11-17 17:10:59,754 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@Kill] [***0000000-211117134747764-oozie-oozi-W@Kill***]Action updated in DB!
2021-11-17 17:10:59,780 INFO org.apache.oozie.action.control.KillActionExecutor: SERVER[server02.hadoop.com] USER[admin] GROUP[-] TOKEN[] APP[Subject 5 - Workflow] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@Kill] Action ended with external status [OK]
2021-11-17 17:10:59,925 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@Kill] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@Kill
2021-11-17 17:10:59,927 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W
2021-11-17 17:10:59,927 INFO org.apache.oozie.command.wf.WorkflowNotificationXCommand: SERVER[server02.hadoop.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000000-211117134747764-oozie-oozi-W] ACTION[0000000-211117134747764-oozie-oozi-W@hive-6e91] No Notification URL is defined. Therefore nothing to notify for job 0000000-211117134747764-oozie-oozi-W@hive-6e91

Log Type: prelaunch.err

Log Upload Time: Wed Nov 17 17:11:00 +0900 2021

Log Length: 0

 
 

Log Type: prelaunch.out

Log Upload Time: Wed Nov 17 17:11:00 +0900 2021

Log Length: 70

Setting up env variables
Setting up job resources
Launching container

 

Log Type: stderr

Log Upload Time: Wed Nov 17 17:11:00 +0900 2021

Log Length: 4561

Showing 4096 bytes of 4561 total. Click here for the full log.

aticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show Log4j2 internal initialization logging.
Connecting to jdbc:hive2://server02.hadoop.com:10000/default
Connected to: Apache Hive (version 2.1.1-cdh6.3.2)
Driver: Hive JDBC (version 2.1.1-cdh6.3.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://server02.hadoop.com:10000/def> USE default;
INFO  : Compiling command(queryId=hive_20211117171057_9409a0f0-f0c7-4196-a475-a9290b9eccdb): USE default
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO  : Completed compiling command(queryId=hive_20211117171057_9409a0f0-f0c7-4196-a475-a9290b9eccdb); Time taken: 0.338 seconds
INFO  : Executing command(queryId=hive_20211117171057_9409a0f0-f0c7-4196-a475-a9290b9eccdb): USE default
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=hive_20211117171057_9409a0f0-f0c7-4196-a475-a9290b9eccdb); Time taken: 0.016 seconds
INFO  : OK
No rows affected (0.542 seconds)
0: jdbc:hive2://server02.hadoop.com:10000/def> 
0: jdbc:hive2://server02.hadoop.com:10000/def> insert overwrite local directory  
'/home/pilot-pjt/item-buy-list'
. . . . . . . . . . . . . . . . . . . . . . .> ROW FORMAT DELIMITED
. . . . . . . . . . . . . . . . . . . . . . .> FIELDS TERMINATED BY ','
. . . . . . . . . . . . . . . . . . . . . . .> select car_number, concat_ws(","  
, collect_set(item))
. . . . . . . . . . . . . . . . . . . . . . .> from managed_smartcar_item_buylis 
t_info
. . . . . . . . . . . . . . . . . . . . . . .> group by car_number
. . . . . . . . . . . . . . . . . . . . . . .> Error: Error while compiling statement: FAILED: RuntimeException Cannot create staging directory 'hdfs://server01.hadoop.com:8020/home/pilot-pjt/item-buy-list/.hive-staging_hive_2021-11-17_17-10-57_746_3560942864087896700-3': Permission denied: user=admin, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:400)
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:256)
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:194)
	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1855)
	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1839)
	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1798)
	at org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:61)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3101)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1123)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:696)
	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675) (state=42000,code=40000)
Closing: 0: jdbc:hive2://server02.hadoop.com:10000/default

 

Log Type: stdout

Log Upload Time: Wed Nov 17 17:11:00 +0900 2021

Log Length: 198282

Showing 4096 bytes of 198282 total. Click here for the full log.

 launch_container.sh
  jetty-jndi-9.3.25.v20180904.jar
  jersey-container-servlet-core-2.25.1.jar
  datanucleus-core-4.1.6.jar
  asm-tree-6.0.jar
------------------------


Script [hive-6e91.sql] content: 
------------------------
USE default;

insert overwrite local directory '/home/pilot-pjt/item-buy-list'
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
select car_number, concat_ws("," , collect_set(item))
from managed_smartcar_item_buylist_info
group by car_number

------------------------

Beeline command arguments :
             -u
             jdbc:hive2://server02.hadoop.com:10000/default
             -n
             admin
             -p
             DUMMY
             -d
             org.apache.hive.jdbc.HiveDriver
             -f
             hive-6e91.sql
             -a
             delegationToken
             --hiveconf
             mapreduce.job.tags=oozie-418ffadb1764d24a19ec7f0c02056574
             --hiveconf
             oozie.action.id=0000000-211117134747764-oozie-oozi-W@hive-6e91
             --hiveconf
             oozie.child.mapreduce.job.tags=oozie-418ffadb1764d24a19ec7f0c02056574
             --hiveconf
             oozie.action.rootlogger.log.level=INFO
             --hiveconf
             oozie.job.id=0000000-211117134747764-oozie-oozi-W
             --hiveconf
             oozie.HadoopAccessorService.created=true

Fetching child yarn jobs
tag id : oozie-418ffadb1764d24a19ec7f0c02056574
No child applications found
=================================================================

>>> Invoking Beeline command line now >>>


<<< Invocation of Beeline command completed <<<

No child hadoop job is executed.
java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:410)
	at org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:55)
	at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
	at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
	at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
	at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:141)
Caused by: java.lang.SecurityException: Intercepted System.exit(2)
	at org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
	at java.lang.Runtime.exit(Runtime.java:107)
	at java.lang.System.exit(System.java:971)
	at org.apache.oozie.action.hadoop.Hive2Main.runBeeline(Hive2Main.java:273)
	at org.apache.oozie.action.hadoop.Hive2Main.run(Hive2Main.java:250)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:104)
	at org.apache.oozie.action.hadoop.Hive2Main.main(Hive2Main.java:65)
	... 16 more
Intercepting System.exit(2)
Failing Oozie Launcher, Main Class [org.apache.oozie.action.hadoop.Hive2Main], exit code [2]
Oozie Launcher, uploading action data to HDFS sequence file: hdfs://server01.hadoop.com:8020/user/admin/oozie-oozi/0000000-211117134747764-oozie-oozi-W/hive-6e91--hive2/action-data.seq
Stopping AM
Callback notification attempts left 0
Callback notification trying http://server02.hadoop.com:11000/oozie/callback?id=0000000-211117134747764-oozie-oozi-W@hive-6e91&status=FAILED
Callback notification to http://server02.hadoop.com:11000/oozie/callback?id=0000000-211117134747764-oozie-oozi-W@hive-6e91&status=FAILED succeeded
Callback notification succeeded

이렇습니다.

 

무슨 문제일까요 ㅜㅜ

다른분 게시물에 나와있는 디렉토리 만들고 권한 변경은 이미 진행해보았지만 변화가 없었습니다.

bigdataHadoopredis데이터 엔지니어링FlumeZooKeeperImpalaKafka

퀴즈

빅데이터 탐색(Exploratory Data Analysis, EDA)의 주요 목적은 무엇일까요?

원시 데이터의 장기 보관

데이터 내 숨겨진 패턴, 관계, 추세 파악

실시간 데이터 스트리밍 처리

데이터 보안 강화

답변 1

0

Big.D님의 프로필 이미지
Big.D
지식공유자

안녕하세요! 빅디 입니다.

로그 내용상으로 HDFS의 권한 문제로 보입니다.

우선 아래 2가지 확인 조치해 보세요~

1. "CM홈 > HDFS > 구성"에서 검색란에 "HDFS 권한 검사" 입력후, 해당기능 해제 되었는지 확인

2. hdfs inode = "/"의 supergroup 권한을 위해 Local(리눅스) 해당 group/계정 추가

  - Putty로 Server02 접속 

  - $ groupadd supergroup

  - $ usermod -a G supergroup admin

 

마지막으로 전체 클러스터 리스타트 하시면 됩니다.   - 빅디 드림

 
유기욱님의 프로필 이미지
유기욱

작성한 질문수

전체 Q&A
질문하기