Azkaban——hadoop/spark/hive等大数据任务调度利器 | AiTi修炼|重剑无锋,拈花微笑
Header
Header

Azkaban——hadoop/spark/hive等大数据任务调度利器

初识Azkaban

Azkaban是由Linkedin公司开源的一个批量工作流任务调度器,用于在一个工作流内以一个特定的顺序运行一组工作和流程。在Azkaban中使用job配置文件建立任务之间的依赖关系,并提供一个易于使用的web用户界面维护和跟踪你的工作流,可以用于调度Hadoop、hive、spark等大数据处理业务场景常见的相互依赖的Job。

  • 提供功能清晰,简单易用的Web UI界面
  • 提供job配置文件快速建立任务和任务之间的依赖关系
  • 提供模块化和可插拔的插件机制,原生支持command、Java、Hive、Pig、Hadoop
  • 基于Java开发,代码结构清晰,易于二次开发

Azkaban 部署模式与体系组件

Azkaban有两种部署方式:solo server mode 和 cluster server mode。

其中solo server mode(单机模式)是指webServer和executorServer运行在同一个进程中,该进程的名字是AzkabanSingleServer。可以使用自带的H2数据库或者配置mysql数据。该模式适用于小规模的使用。

而cluster server mode(集群模式):该模式使用MySQL数据库,webServer和executorServer运行在不同进程中,该模式适用于大规模应用。

在安装部署之前需要先熟悉Azkaban的组件体系:

  • 关系型数据库(MySQL)
  • AzkabanWebServer
  • AzkabanExecutorServer

azkaban 安装,体系组件

根据Azkaban的组件体系,我们在安装部署Azkaban的时候需要下载一下内容:

  • 1、MySQL脚本
    azkaban-sql-script-2.5.0.tar.gz

  • 2、Azkaban Web Server
    azkaban-web-server-2.5.0.tar.gz

  • 3、Azkaban Executor Server
    azkaban-executor-server-2.5.0.tar.gz

Azkaban的安装

  • 1、导入Azkaban所需的mysql相关sql脚本
    将azkaban-sql-script-2.5.0.tar.gz解压(tar -xvf azkaban-sql-script-2.5.0.tar.gz)执行其中的create-all-sql-2.5.0.sql文件,建好Azkaban需要用到的表。(ps:在mysql创建数据库、用户账号,并将相关的sql文件导入方法参考http://rscala.com/index.php/427.html)

  • 2、Azkaban Web Server配置
    将azkaban-web-server-2.5.0.tar.gz解压(tar -xvf azkaban-web-server-2.5.0.tar.gz),解压后的文件结构如下:

azkaban-access.log  
azkaban.version  
azkaban-webserver.log  
bin  
conf  
currentpid  
extlib  
lib  
plugins  
temp 
web

进入conf文件夹下,有两个文件需要根据实际情况进行配置修改,分别是:azkaban.properties(Azkaban系统参数 )和
azkaban-users.xml(XmlUserManager模式下的用户管理文件 )。

对于azkaban.properties的修改配置如下:

#Azkaban Personalization Settings#zkaban-web-server-2.5.0.tar.gzAzkaban Personalization Settings
azkaban.name=RScala Azkaban Test    #主标题
azkaban.label=My Local Azkaban  #副标题
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
default.timezone.id=Asia/Shanghai   #时区#Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=conf/azkaban-users.xml#Loader for projects
executor.global.properties=conf/global.properties
azkaban.project.dir=projects

# 数据库配置
database.type=mysql
mysql.port=3306
mysql.host=rscala
mysql.database=azkaban  # 数据库名与上一步新建的database保持一致
mysql.user=root
mysql.password=root
mysql.numconnections=100# Velocity dev mode
velocity.dev.mode=false

# Azkaban Jetty server properties.
jetty.maxThreads=25
jetty.ssl.port=8888 #https 访问端口
jetty.port=8081
jetty.keystore=web/keystore   #web目录下
jetty.password=azkaban
jetty.keypassword=azkaban
jetty.truststore=web/truststore #web目录下
jetty.trustpassword=azkaban

# Azkaban Executor settings
executor.port=12321 #指定executor的port
executor.host=rscala  #指定executor所在机器# mail settings
mail.sender=
mail.host=
job.failure.email=
job.success.email=

lockdown.create.projects=false

其中jetty.keystore、jetty.truststore是配置azkaban的ssl加密安全,即在web访问azkaban是使用https协议。关于keystore文件以及truststore文件使用以下方法进行生成(配置jetty的SSL的KeyStore):
ps:在需要输入密码的地方,输入上面配置文件中的password参数(本文安装实践的密码是azkaban),其他信息随便填写。

keytool -genkey -keystore keystore -alias jetty-azkaban -keyalg RSA -validity 3560

keytool -export -alias jetty-azkaban -keystore keystore -rfc -file selfsignedcert.cer

keytool -import -alias certificatekey -file selfsignedcert.cer -keystore truststore

生成Jetty的ssl安全证书后,将生成的keystore和truststore文件移动到web路径下,如果需要删除可以使用以下命令操作:

keytool -delete -alias jetty-azkaban -keystore keystore -storepass azkaban

对于azkaban-users.xml的配置如下:

<azkaban-users>
                <!-- 用户为azkaban 密码为azkaban,角色为admin 所在组azkaban,在web页面登录时候需要账号,可以根据实际修改配置-->
        <user username="azkaban" password="azkaban" roles="admin" groups="azkaban" />
        <user username="metrics" password="metrics" roles="metrics"/>
        <role name="admin" permissions="ADMIN" />
        <role name="metrics" permissions="METRICS"/>
</azkaban-users>
  • 启动 Azkaban 的Web Server
    在解压后的目录执行bin/azkaban-web-start.sh这样的命令就可以启动 Azkaban 的Web Server,启动后会有日志输出
2017/05/06 20:12:01.393 +0800 ERROR [AzkabanWebServer] [Azkaban] Starting Jetty Azkaban Executor...
2017/05/06 20:12:01.425 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban settings file from bin/../conf
2017/05/06 20:12:01.425 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban properties file
2017/05/06 20:12:01.442 +0800 INFO [AzkabanWebServer] [Azkaban] Setting up Jetty Https Server with port:8443 and numThreads:25
2017/05/06 20:12:01.485 +0800 INFO [log] [Azkaban] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2017/05/06 20:12:01.549 +0800 INFO [AzkabanWebServer] [Azkaban] Loading user manager class azkaban.user.XmlUserManager
2017/05/06 20:12:01.591 +0800 INFO [XmlUserManager] [Azkaban] Loading user azkaban
2017/05/06 20:12:01.593 +0800 INFO [XmlUserManager] [Azkaban] Loading user metrics
2017/05/06 20:12:02.365 +0800 INFO [AzkabanWebServer] [Azkaban] Loading JDBC for project management
2017/05/06 20:12:02.368 +0800 INFO [ExecutorManager] [Azkaban] Cleaning old logs from execution_logs
2017/05/06 20:12:02.370 +0800 INFO [ProjectManager] [Azkaban] Project version retention is set to 3
2017/05/06 20:12:02.764 +0800 INFO [ExecutorManager] [Azkaban] Cleaning old log files before 2017-02-11T20:12:02.443+08:00
2017/05/06 20:12:02.791 +0800 INFO [ExecutorManager] [Azkaban] Cleaned up 0 log entries.
2017/05/06 20:12:02.885 +0800 INFO [TriggerManager] [Azkaban] TriggerManager loaded.
2017/05/06 20:12:02.885 +0800 INFO [AzkabanWebServer] [Azkaban] Loading built-in checker and action types
2017/05/06 20:12:02.892 +0800 INFO [CheckerTypeLoader] [Azkaban] Registering checker BasicTimeChecker
2017/05/06 20:12:02.892 +0800 INFO [CheckerTypeLoader] [Azkaban] Registering checker SlaChecker
2017/05/06 20:12:02.892 +0800 INFO [CheckerTypeLoader] [Azkaban] Registering checker ExecutionChecker
2017/05/06 20:12:02.892 +0800 INFO [ActionTypeLoader] [Azkaban] Registering action ExecuteFlowAction
2017/05/06 20:12:02.892 +0800 INFO [ActionTypeLoader] [Azkaban] Registering action KillExecutionAction
2017/05/06 20:12:02.892 +0800 INFO [ActionTypeLoader] [Azkaban] Registering action AlertAction
2017/05/06 20:12:02.892 +0800 INFO [ActionTypeLoader] [Azkaban] Registering action CreateTriggerAction
2017/05/06 20:12:02.892 +0800 INFO [AzkabanWebServer] [Azkaban] Loading trigger based scheduler
2017/05/06 20:12:02.895 +0800 INFO [AzkabanWebServer] [Azkaban] Loading plug-in checker and action types
2017/05/06 20:12:02.896 +0800 ERROR [AzkabanWebServer] [Azkaban] plugin path plugins/triggers doesn't exist!
2017/05/06 20:12:02.896 +0800 INFO [AzkabanWebServer] [Azkaban] Setting timezone to Asia/Shanghai
2017/05/06 20:12:02.896 +0800 INFO [AzkabanWebServer] [Azkaban] Registering MBeans...
2017/05/06 20:12:02.939 +0800 INFO [AzkabanWebServer] [Azkaban] Bean azkaban.jmx.JmxJettyServer registered.
2017/05/06 20:12:02.945 +0800 INFO [AzkabanWebServer] [Azkaban] Bean azkaban.jmx.JmxTriggerManager registered.
2017/05/06 20:12:02.948 +0800 INFO [AzkabanWebServer] [Azkaban] Bean azkaban.jmx.JmxExecutorManager registered.
2017/05/06 20:12:02.950 +0800 INFO [AzkabanWebServer] [Azkaban] Setting up web resource dir web/
2017/05/06 20:12:03.058 +0800 INFO [JdbcTriggerLoader] [Azkaban] Loading all triggers from db.
2017/05/06 20:12:03.126 +0800 INFO [JdbcTriggerLoader] [Azkaban] Loaded 0 triggers.
2017/05/06 20:12:03.127 +0800 INFO [log] [Azkaban] jetty-6.1.26
2017/05/06 20:12:04.098 +0800 INFO [log] [Azkaban] Started SslSocketConnector@0.0.0.0:8443
2017/05/06 20:12:04.100 +0800 INFO [AzkabanWebServer] [Azkaban] Server running on ssl port 8443.
2017/05/06 20:16:08.213 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Session id c2bae8ea-4985-45d9-bf14-0b2bf62959e9
2017/05/06 20:16:08.214 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Found session azkaban: [azkaban,]
2017/05/06 20:16:08.226 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Session id c2bae8ea-4985-45d9-bf14-0b2bf62959e9
2017/05/06 20:16:08.227 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Found session azkaban: [azkaban,]

在启动完全之后就可以可以在Web页面中访问,输入https://rscala:8888, 具体界面如下
azkaban_login

azkaban_login_index
* 安装Azkaban Executor Server
对下载的azkaban-executor-server-2.5.0.tar.gz 进行解压(tar -xvf azkaban-executor-server-2.5.0.tar.gz ),可以看到 azkaban-executor的结构如下:

azkaban.version  
bin  
conf  
extlib  
lib  
plugins

需要对conf目录下的文件进行配置,而在该目录有这些文件:azkaban.private.properties azkaban.properties global.properties,默认情况只需要对 azkaban.properties 进行配置修改,具体如下:

#Azkaban
default.timezone.id=America/Los_Angeles

# Azkaban JobTypes Plugins
azkaban.jobtype.plugin.dir=plugins/jobtypes

#Loader for projects
executor.global.properties=conf/global.properties
azkaban.project.dir=projects

database.type=mysql
mysql.port=3306
mysql.host=localhost
mysql.database=azkaban2 #需要保持与web的数据库一致 
mysql.user=root                  #需要保持与web的数据库一致
mysql.password=root         #需要保持与web的数据库一致
mysql.numconnections=100

# Azkaban Executor settings
executor.maxThreads=50
executor.port=12321
executor.flow.threads=30

配置好后,就可以在解压目录使用命令/bin/azkaban-executor-start.sh启动Azkaban Executor Server服务,启动成功之后会有以下的日志打印。

[Azkaban] Starting Jetty Azkaban Executor...
2017/05/06 23:12:43.398 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban settings file from bin/../conf
2017/05/06 23:12:43.399 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban private properties file
2017/05/06 23:12:43.403 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban properties file
2017/05/06 23:12:43.456 +0800 INFO [AzkabanExecutorServer] [Azkaban] Setting timezone to America/Los_Angeles
2017/05/06 23:12:43.503 +0800 INFO [log] [Azkaban] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2017/05/06 23:12:43.657 +0800 INFO [FlowRunnerManager] [Azkaban] Execution dir retention set to 86400000 ms
2017/05/06 23:12:43.677 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning recently finished
2017/05/06 23:12:43.677 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning old projects
2017/05/06 23:12:43.677 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning old execution dirs
2017/05/06 23:12:43.702 +0800 INFO [AzkabanExecutorServer] [Azkaban] Registering MBeans...
2017/05/06 23:12:43.833 +0800 INFO [AzkabanExecutorServer] [Azkaban] Bean azkaban.jmx.JmxJettyServer registered.
2017/05/06 23:12:43.853 +0800 INFO [AzkabanExecutorServer] [Azkaban] Bean azkaban.jmx.JmxFlowRunnerManager registered.
2017/05/06 23:12:43.853 +0800 INFO [log] [Azkaban] jetty-6.1.26
2017/05/06 23:12:44.023 +0800 INFO [log] [Azkaban] Started SocketConnector@0.0.0.0:12321
2017/05/06 23:12:44.023 +0800 INFO [AzkabanExecutorServer] [Azkaban] Azkaban Executor Server started on port 12321

Azkaban的使用案例

在azkaban中,一个Flow包含有多个job,每个独立的Flows都对应一个Projects,登录Azkaban主界面后,可以看到有四个标签页,分别对应了它的四大功能模块:

azkaban_intf

  • Projects:每个独立的Flows都对应一个Projects,Flows将在Projects中运行
  • Scheduling:显示定时执行任务
  • Executing:显示正在执行的任务
  • History:显示历史执行任务

在实际应用的时候,flow以及job需要我们配置打包好,之后再上传到Azkaban的web管理界面,存储到对应的project中,然后配置相关project的运行调度策略。下面以一个hello azkaban例子来说明使用azkaban的过程。

  • 在本地创建一个文件夹,并在该文件夹下面创建两个文件,分别为:hello_azkaban_1.job和hello_azkban_2.job。
    hello_azkaban1.job 执行完之后再执行hello_azkban_2.job其中hello_azkaban_1.job的内容如下:
type=command
command=echo “this is hello_azkaban_1.job start ”

而hello_azkban2.job的内容如下:

type=command
command=echo “this is hello_azkaban_2.job start”
dependencies=hello_azkaban_1
  • 打包把两个文件打包成hello_azkaban.zip文件

  • 在azkaban web 创建project,命名为test_project ,并将打包好的zip文件——hello_azkaban.zip,上传到该project

test_project

azkaban_gp

上传project1

上传project

  • 在web界面进行单次调度执行,具体操作以及结果如下:

azkaban_excut

azkaban_es1

azkaban_ex2
web后台日志如下:

2017/05/07 00:01:12.444 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Found session azkaban: [azkaban,]
2017/05/07 00:01:12.445 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Session id e2e7586e-4e58-42e6-8ef7-c188069f989b
2017/05/07 00:01:12.449 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Found session azkaban: [azkaban,]
2017/05/07 00:01:12.439 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Found session azkaban: [azkaban,]
2017/05/07 00:05:03.715 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Session id e2e7586e-4e58-42e6-8ef7-c188069f989b
2017/05/07 00:05:03.715 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Found session azkaban: [azkaban,]
2017/05/07 00:05:03.720 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Session id e2e7586e-4e58-42e6-8ef7-c188069f989b
2017/05/07 00:05:03.722 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Found session azkaban: [azkaban,]
2017/05/07 00:05:18.949 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Session id e2e7586e-4e58-42e6-8ef7-c188069f989b
2017/05/07 00:05:18.950 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Found session azkaban: [azkaban,]
2017/05/07 00:05:18.951 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Session id e2e7586e-4e58-42e6-8ef7-c188069f989b
2017/05/07 00:05:18.952 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Found session azkaban: [azkaban,]
2017/05/07 00:06:18.767 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Session id e2e7586e-4e58-42e6-8ef7-c188069f989b
2017/05/07 00:06:18.776 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Found session azkaban: [azkaban,]
2017/05/07 00:06:18.785 +0800 INFO [ExecutorManager] [Azkaban] Submitting execution flow hello_azkaban_2 by azkaban
2017/05/07 00:06:18.925 +0800 INFO [JdbcExecutorLoader] [Azkaban] Flow given hello_azkaban_2 given id 1
2017/05/07 00:06:22.764 +0800 INFO [TriggerManager] [Azkaban] Flow finish event received. 1
2017/05/07 00:06:22.770 +0800 INFO [TriggerManager] [Azkaban] Flow finish event received. 1
2017/05/07 00:06:45.425 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Session id e2e7586e-4e58-42e6-8ef7-c188069f989b
2017/05/07 00:06:45.425 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Found session azkaban: [azkaban,]
2017/05/07 00:06:45.738 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Session id e2e7586e-4e58-42e6-8ef7-c188069f989b
2017/05/07 00:06:45.739 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Found session azkaban: [azkaban,]
2017/05/07 00:06:45.838 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Session id e2e7586e-4e58-42e6-8ef7-c188069f989b
2017/05/07 00:06:45.839 +0800 INFO [LoginAbstractAzkabanServlet] [Azkaban] Found session azkaban: [azkaban,]

excutor日志如下:

 2017/05/07 00:06:19.906 +0800 INFO [ExecutorServlet] [Azkaban] User null has called action execute on 1
2017/05/07 00:06:20.409 +0800 INFO [FlowRunnerManager] [Azkaban] Flow 1 submitted with path executions/1
2017/05/07 00:06:20.432 +0800 INFO [FlowRunnerManager] [Azkaban] First time executing new project. Setting up in directory projects/1.1
2017/05/07 00:06:20.433 +0800 INFO [JdbcProjectLoader] [Azkaban] Retrieving to 1 version:1
2017/05/07 00:06:20.531 +0800 INFO [JdbcProjectLoader] [Azkaban] Md5 Hash is valid
2017/05/07 00:06:20.532 +0800 INFO [FlowRunnerManager] [Azkaban] Downloading zip file.
2017/05/07 00:06:20.890 +0800 INFO [hello_azkaban_2] [Azkaban] Running execid:1 flow:hello_azkaban_2 project:1 version:1
2017/05/07 00:06:20.904 +0800 INFO [hello_azkaban_2] [Azkaban] Update active reference
2017/05/07 00:06:20.953 +0800 INFO [hello_azkaban_2] [Azkaban] Updating initial flow directory.
2017/05/07 00:06:21.069 +0800 INFO [hello_azkaban_2] [Azkaban] Fetching job and shared properties.
2017/05/07 00:06:21.076 +0800 INFO [hello_azkaban_2] [Azkaban] Starting flows
2017/05/07 00:06:21.079 +0800 INFO [hello_azkaban_2] [Azkaban] Running flow 'hello_azkaban_2'.
2017/05/07 00:06:21.176 +0800 INFO [hello_azkaban_2] [Azkaban] Submitting job 'hello_azkaban_1' to run.
Uploading flowId hello_azkaban_2
cwd=/home/hadoop/hadoop/azkaban/azkaban-executor-2.5.0/executions/1
output properties file=/home/hadoop/hadoop/azkaban/azkaban-executor-2.5.0/executions/1/hello_azkaban_1_output_8087442804541439013_tmp
2017/05/07 00:06:21.449 +0800 INFO [hello_azkaban_2] [Azkaban] Job hello_azkaban_1 finished with status SUCCEEDED in 0 seconds
2017/05/07 00:06:21.457 +0800 INFO [hello_azkaban_2] [Azkaban] Submitting job 'hello_azkaban_2' to run.
Uploading flowId hello_azkaban_2
2017/05/07 00:06:21.511 +0800 INFO [hello_azkaban_2] [Azkaban] No attachment file for job hello_azkaban_1 written.
cwd=/home/hadoop/hadoop/azkaban/azkaban-executor-2.5.0/executions/1
output properties file=/home/hadoop/hadoop/azkaban/azkaban-executor-2.5.0/executions/1/hello_azkaban_2_output_7619771042459610521_tmp
2017/05/07 00:06:22.374 +0800 INFO [hello_azkaban_2] [Azkaban] Job hello_azkaban_2 finished with status SUCCEEDED in 0 seconds
2017/05/07 00:06:22.376 +0800 INFO [hello_azkaban_2] [Azkaban] Flow '' is set to SUCCEEDED in 1 seconds
2017/05/07 00:06:22.391 +0800 INFO [hello_azkaban_2] [Azkaban] No attachment file for job hello_azkaban_2 written.
2017/05/07 00:06:22.463 +0800 INFO [hello_azkaban_2] [Azkaban] Finishing up flow. Awaiting Termination
2017/05/07 00:06:22.507 +0800 INFO [hello_azkaban_2] [Azkaban] Finished Flow
2017/05/07 00:06:22.508 +0800 INFO [hello_azkaban_2] [Azkaban] Setting end time for flow 1 to 1494086782508
2017/05/07 00:06:22.666 +0800 INFO [FlowRunnerManager] [Azkaban] Flow 1 is finished. Adding it to recently finished flows list.
2017/05/07 00:06:43.798 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning recently finished
2017/05/07 00:07:43.799 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning old projects
2017/05/07 00:08:43.802 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning recently finished
2017/05/07 00:08:43.803 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning execution 1 from recently finished flows list.
2017/05/07 00:10:43.804 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning recently finished
2017/05/07 00:12:43.809 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning recently finished
2017/05/07 00:12:43.809 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning old projects
2017/05/07 00:12:43.809 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning old execution dirs

rscala.com版权所有,本文Azkaban——hadoop/spark/hive等大数据任务调度利器转载请注明出处:http://rscala.com/index.php/429.html

该文章归档分类于 azkaban, 大数据基础, 大数据进阶

One Response

  • 我看了,你启动web服务器的时候有报错信息:
    2017/05/06 20:12:02.896 +0800 ERROR [AzkabanWebServer] [Azkaban] plugin path plugins/triggers doesn’t exist!

    我在安装过程中也遇到了相同的报错信息,然后我按照提示,添加了目录triggers,但是添加后,页面访问不了了,我测试了一下,不加的话,页面可以访问,任务也能执行,但是定时任务是不会执行的,加了的话,定时任务可以执行了,但是页面访问不了,报错: java.lang.RuntimeException: Velocity could not be initialized!

    回复


Leave a Reply to 凉凉 Cancel reply

电子邮件地址不会被公开。 必填项已用*标注

*

code