定时任务管理系统

Related tags

golang cron crontab scheduler
Overview

gocron - 定时任务管理系统

Downloads license Release

项目简介

使用Go语言开发的轻量级定时任务集中调度和管理系统, 用于替代Linux-crontab 查看文档

原有的延时任务拆分为独立项目延迟队列

功能特性

  • Web界面管理定时任务
  • crontab时间表达式, 精确到秒
  • 任务执行失败可重试
  • 任务执行超时, 强制结束
  • 任务依赖配置, A任务完成后再执行B任务
  • 账户权限控制
  • 任务类型
    • shell任务

    在任务节点上执行shell命令, 支持任务同时在多个节点上运行

    • HTTP任务

    访问指定的URL地址, 由调度器直接执行, 不依赖任务节点

  • 查看任务执行结果日志
  • 任务执行结果通知, 支持邮件、Slack、Webhook

截图

流程图 任务 Slack

支持平台

Windows、Linux、Mac OS

环境要求

MySQL

下载

releases

版本升级

安装

二进制安装

  1. 解压压缩包  
  2. cd 解压目录
  3. 启动
  • 调度器启动
    • Windows: gocron.exe web
    • Linux、Mac OS: ./gocron web
  • 任务节点启动, 默认监听0.0.0.0:5921
    • Windows: gocron-node.exe
    • Linux、Mac OS: ./gocron-node
  1. 浏览器访问 http://localhost:5920

源码安装

  • 安装Go 1.11+
  • go get -d github.com/ouqiang/gocron
  • export GO111MODULE=on
  • 编译 make
  • 启动
    • gocron ./bin/gocron web
    • gocron-node ./bin/gocron-node

docker

docker run --name gocron --link mysql:db -p 5920:5920 -d ouqg/gocron

配置: /app/conf/app.ini

日志: /app/log/cron.log

镜像不包含gocron-node, gocron-node需要和具体业务一起构建

开发

  1. 安装Go1.9+, Node.js, Yarn
  2. 安装前端依赖 make install-vue
  3. 启动gocron, gocron-node make run
  4. 启动node server make run-vue, 访问地址 http://localhost:8080

访问http://localhost:8080, API请求会转发给gocron

make 编译

make run 编译并运行

make package 打包

生成当前系统的压缩包 gocron-v1.5-darwin-amd64.tar.gz gocron-node-v1.5-darwin-amd64.tar.gz

make package-all 生成Windows、Linux、Mac的压缩包

命令

  • gocron

    • -v 查看版本
  • gocron web

    • --host 默认0.0.0.0
    • -p 端口, 指定端口, 默认5920
    • -e 指定运行环境, dev|test|prod, dev模式下可查看更多日志信息, 默认prod
    • -h 查看帮助
  • gocron-node

    • -allow-root *nix平台允许以root用户运行
    • -s ip:port 监听地址
    • -enable-tls 开启TLS
    • -ca-file   CA证书文件  
    • -cert-file 证书文件
    • -key-file 私钥文件
    • -h 查看帮助
    • -v 查看版本

To Do List

  • 版本升级
  • 批量开启、关闭、删除任务
  • 调度器与任务节点通信支持https
  • 任务分组
  • 多用户
  • 权限控制

程序使用的组件

反馈

提交issue

ChangeLog

v1.5

  • 前端使用Vue+ElementUI重构
  • 任务通知
    • 新增WebHook通知
    • 自定义通知模板
    • 匹配任务执行结果关键字发送通知
  • 任务列表页显示任务下次执行时间

v1.4

  • HTTP任务支持POST请求
  • 后台手动停止运行中的shell任务
  • 任务执行失败重试间隔时间支持用户自定义
  • 修复API接口调用报403错误

v1.3

  • 支持多用户登录
  • 增加用户权限控制

v1.2.2

  • 用户登录页增加图形验证码
  • 支持从旧版本升级
  • 任务批量开启、关闭、删除
  • 调度器与任务节点支持HTTPS双向认证
  • 修复任务列表页总记录数显示错误

v1.1

  • 任务可同时在多个节点上运行
  • *nix平台默认禁止以root用户运行任务节点
  • 子任务命令中增加预定义占位符, 子任务可根据主任务运行结果执行相应操作
  • 删除守护进程模块
  • Web访问日志输出到终端
Issues
  • 任务显示超长时间运行,且无法停止

    任务显示超长时间运行,且无法停止

    image

    如图 服务器上查看该任务实际上已经没有在运行了,点击停止任务,接口显示操作成功,但状态仍然没改变;

    gocron :直接下载的 release 1.5 服务器 :centos 6.9 64位

    opened by qiuqiuX 13
  • 服务器是UTC时区

    服务器是UTC时区

    服务器时间是UTC时间怎么处理。设置的时候要减8小时,但是每月的1号的1点就没法确定。

    enhancement 
    opened by LanceYuan 11
  • -bash: ./gocron-node: cannot execute binary file

    -bash: ./gocron-node: cannot execute binary file

    -bash: ./gocron-node: cannot execute binary file

    请问下,这个是什么错误呀

    opened by WanSeLiangWei 10
  • postgresql支持

    postgresql支持

    增加数据库postgresql的支持

    opened by zhmin 9
  • windows下读取Root_CA.crt失败

    windows下读取Root_CA.crt失败

    failed to read ca cert file: F:/gowork/gocron-windows-amd64/Root_CA.crt

    opened by xiucaiwu 8
  • gocron1.5无法编译

    gocron1.5无法编译

    qq 20180530133027

    opened by zaite 7
  •  命令输出转换编码失败(gbk to utf8)

    命令输出转换编码失败(gbk to utf8)

    我认为这里处理不太好 我用windows,go编译的二进制输出的应该不是gbk 转换失败 转换失败之后是不是应该保持原样,不然不知道输出的是什么 或者不转

    opened by smirkcat 7
  • 手工执行UPDATE `task_log` SET可以成功,但是gocron自己执行报错

    手工执行UPDATE `task_log` SET可以成功,但是gocron自己执行报错

    [[email protected] log]# tail cron.log 2020-05-26/11:45:35 [ERR] 编辑任务#获取任务详情失败#任务ID-0 2020-05-26/11:46:59 [INF] 开始执行任务#bj-his-test-expdp.sh#命令-su - oracle -lc "/usr/local/shell/expdp.sh" 2020-05-26/11:56:16 [INF] 任务完成#bj-his-test-expdp.sh#命令-su - oracle -lc "/usr/local/shell/expdp.sh" 2020-05-26/11:56:16 [ERR] [[任务结束#更新任务日志失败- Error 1366: Incorrect string value: '\xD5\xEF\xB6\xCF" ...' for column 'result' at row 1]] 2020-05-26/12:31:25 [INF] 开始执行任务#bj-his-test-expdp.sh#命令-su - oracle -lc "/usr/local/shell/expdp.sh" 2020-05-26/12:34:07 [INF] 任务完成#bj-his-test-expdp.sh#命令-su - oracle -lc "/usr/local/shell/expdp.sh" 2020-05-26/12:34:07 [ERR] [[任务结束#更新任务日志失败- Error 1366: Incorrect string value: '\xD5\xEF\xB6\xCF" ...' for column 'result' at row 1]] 2020-05-26/12:50:26 [INF] 开始执行任务#bj-his-test-expdp.sh#命令-su - oracle -lc "/usr/local/shell/expdp.sh" 2020-05-26/12:52:51 [INF] 任务完成#bj-his-test-expdp.sh#命令-su - oracle -lc "/usr/local/shell/expdp.sh" 2020-05-26/12:52:51 [ERR] [[任务结束#更新任务日志失败- Error 1366: Incorrect string value: '\xD5\xEF\xB6\xCF" ...' for column 'result' at row 1]]

    手工执行下面这个sql是可以成功的,为什么gocron自己执行报错:

    UPDATE task_log SET retry_times = 0, status = 2, result = '主机: [bj-his-test-172.xxxxx:5921]\n\n\nExport: Release 11.2.0.4.0 - Production on Tue May 26 12:50:26 2020\n\nCopyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.\n\nConnected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production\nWith the Partitioning, OLAP, Data Mining and Real Application Testing options\nFLASHBACK automatically enabled to preserve database integrity.\nStarting "SYS"."SYS_EXPORT_SCHEMA_01": "/******** AS SYSDBA" directory=DATA_DIR Dumpfile=appuser_local_20200526_125026.dmp LOGFILE=appuser_local_20200526_125026.log schemas=appuser compression=all \nEstimate in progress using BLOCKS method...\nProcessing object type SCHEMA_EXPORT/TABLE/TABLE_DATA\nTotal estimation using BLOCKS method: 7.367 GB\nProcessing object type SCHEMA_EXPORT/USER\nProcessing object type SCHEMA_EXPORT/SYSTEM_GRANT\nProcessing object type SCHEMA_EXPORT/ROLE_GRANT\nProcessing object type SCHEMA_EXPORT/DEFAULT_ROLE\nProcessing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA\nProcessing object type SCHEMA_EXPORT/SYNONYM/SYNONYM\nProcessing object type SCHEMA_EXPORT/TYPE/TYPE_SPEC\nProcessing object type SCHEMA_EXPORT/SEQUENCE/SEQUENCE\nProcessing object type SCHEMA_EXPORT/TABLE/TABLE\nProcessing object type SCHEMA_EXPORT/TABLE/COMMENT\nProcessing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_SPEC\nProcessing object type SCHEMA_EXPORT/FUNCTION/FUNCTION\nProcessing object type SCHEMA_EXPORT/PROCEDURE/PROCEDURE\nProcessing object type SCHEMA_EXPORT/PACKAGE/COMPILE_PACKAGE/PACKAGE_SPEC/ALTER_PACKAGE_SPEC\nProcessing object type SCHEMA_EXPORT/FUNCTION/ALTER_FUNCTION\nProcessing object type SCHEMA_EXPORT/PROCEDURE/ALTER_PROCEDURE\nProcessing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX\nProcessing object type SCHEMA_EXPORT/TABLE/INDEX/FUNCTIONAL_INDEX/INDEX\nProcessing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT\nProcessing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS\nProcessing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/FUNCTIONAL_INDEX/INDEX_STATISTICS\nProcessing object type SCHEMA_EXPORT/VIEW/VIEW\nProcessing object type SCHEMA_EXPORT/VIEW/COMMENT\nProcessing object type SCHEMA_EXPORT/PACKAGE/PACKAGE_BODY\nProcessing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/REF_CONSTRAINT\nProcessing object type SCHEMA_EXPORT/TABLE/TRIGGER\nProcessing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS\n. . exported "appuser"."EXTEND_MONITOR_ACCESS_RECORDS" 817.3 MB 359438 rows\n. . exported "appuser"."BAS_TASK_LOGS" 137.8 MB 5311296 rows\n. . exported "appuser"."INS_MEDLIST_YB1023" 29.30 MB 765537 rows\n. . exported "appuser"."PAY_TRANSACTION_DETAIL20191028" 7.634 MB 69943 rows\n. . exported "SYS"."SYS_EXPORT_SCHEMA_01" successfully loaded/unloaded\n******************************************************************************\nDump file set for SYS.SYS_EXPORT_SCHEMA_01 is:\n /data/backup/appuser_local_20200526_125026.dmp\nJob "SYS"."SYS_EXPORT_SCHEMA_01" successfully completed at Tue May 26 12:52:50 2020 elapsed 0 00:02:24\n\n\n\n', end_time = '2020-05-26 12:52:51' WHERE id=86

    opened by czxin788 7
  • update gocrontask web页面没有显示下次执行时间

    update gocrontask web页面没有显示下次执行时间

    在update gocrontask 表中status 开启关闭的时候,还需要在web页面重新开启关闭一次才会有下次执行时间。 想问下是否在哪里可以改这个下次执行时间

    opened by yy1117 7
  • 高可用问题

    高可用问题

    请问调度器挂了之后,节点的任务能正常执行吗?

    opened by yewenhai99999 0
  • 这个项目还有人维护吗

    这个项目还有人维护吗

    这个项目还有人维护吗

    opened by lqszsd 0
  • solve error upath@1.0.4: The engine

    solve error [email protected]: The engine "node" is incompatible with this …

    yarn install yarn install v1.22.10 warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json. [1/5] 🔍 Validating package.json... [2/5] 🔍 Resolving packages... warning webpack > watchpack > chokidar > [email protected]: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2. [3/5] 🚚 Fetching packages... error [email protected]: The engine "node" is incompatible with this module. Expected version ">=4 <=9". Got "16.2.0" error Found incompatible module. info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

    opened by YazaiHu 0
  • Multi-node execution failed

    Multi-node execution failed

    Selecting more than three nodes in a task will not execute any commands and will not report errors why?

    opened by Soberabsurd 0
  • WebHook通知功能无法使用

    WebHook通知功能无法使用

    环境:centos7.6 版本:v1.5.3 操作如下:

    1. 将钉钉机器人webhook地址加入到webhook
    2. 创建定时任务,使用shell方式调用脚本,并开启webhook方式通知
    3. 任务执行正常,但并未发送钉钉通知
    4. 同一台服务器,部署Jenkins,创建Jenkins job,使用shell方式调用脚本,并开通webhook方式,可以正常发送钉钉通知

    排除网络问题,重新更换服务器部署,webhook通知功能依旧无法使用,更换v1.5.2问题依旧。

    opened by maicos520 1
  • gocron 报 Error number 24 means 'Too many open files'的错误

    gocron 报 Error number 24 means 'Too many open files'的错误

    手工执行innobackupex 脚本可以备份不报错,但是一用gocron执行innobackupex 脚本就报错 /usr/local/percona-xtrabackup/bin/innobackupex --defaults-file=/data/mysql/mysql56/mysql3306/my3306.cnf --user=${USER} --password=${PAWD} --socket=/data/mysql/mysql56/mysql3306/tmp/mysql3306.sock --stream=tar ${BASEDIR} 2>> ${LOGFILE}|gzip >${BASEDIR}/${Time}.tar.gz

    gocron执行的日志报错内容如下:

    Start-Time :20210618_122057
    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    xtrabackup: recognized server arguments: --datadir=/data/mysql/mysql56/mysql3306/data --tmpdir=/data/mysql/mysql56/mysql3306/tmp --innodb_buffer_pool_size=500M --innodb_data_home_dir=/data/mysql/mysql56/mysql3306/innodb --innodb_data_file_path=ibdata1:1G:autoextend --innodb_log_group_home_dir=/data/mysql/mysql56/mysql3306/innodb --innodb_log_file_size=1G --innodb_log_files_in_group=2 --innodb_log_buffer_size=32M --innodb_file_per_table=1 --innodb_flush_method=O_DIRECT --innodb_flush_log_at_trx_commit=2 --innodb_max_dirty_pages_pct=80 --innodb_read_io_threads=16 --innodb_write_io_threads=16 --innodb_open_files=4096000 --server-id=793306 --log_bin=/data/mysql/mysql56/mysql3306/binlog/mysql-bin 
    xtrabackup: recognized client arguments: 
    210618 12:20:57 innobackupex: Starting the backup operation
    
    IMPORTANT: Please check that the backup run completes successfully.
               At the end of a successful backup run innobackupex
               prints "completed OK!".
    
    210618 12:20:57  version_check Connecting to MySQL server with DSN 'dbi:mysql:;mysql_read_default_group=xtrabackup;port=3306;mysql_socket=/data/mysql/mysql56/mysql3306/tmp/mysql3306.sock' as 'root'  (using password: YES).
    210618 12:20:57  version_check Connected to MySQL server
    210618 12:20:57  version_check Executing a version check against the server...
    210618 12:20:57  version_check Done.
    210618 12:20:57 Connecting to MySQL server host: localhost, user: root, password: set, port: 3306, socket: /data/mysql/mysql56/mysql3306/tmp/mysql3306.sock
    Using server version 5.6.35-log
    /usr/local/percona-xtrabackup/bin/innobackupex version 2.4.19 based on MySQL server 5.7.26 Linux (x86_64) (revision id: c2d69da)
    xtrabackup: uses posix_fadvise().
    xtrabackup: cd to /data/mysql/mysql56/mysql3306/data
    xtrabackup: open files limit requested 0, set to 1024
    xtrabackup: using the following InnoDB configuration:
    xtrabackup:   innodb_data_home_dir = /data/mysql/mysql56/mysql3306/innodb
    xtrabackup:   innodb_data_file_path = ibdata1:1G:autoextend
    xtrabackup:   innodb_log_group_home_dir = /data/mysql/mysql56/mysql3306/innodb
    xtrabackup:   innodb_log_files_in_group = 2
    xtrabackup:   innodb_log_file_size = 1073741824
    xtrabackup: using O_DIRECT
    InnoDB: Number of pools: 1
    210618 12:20:57 >> log scanned up to (124042100446)
    xtrabackup: Generating a list of tablespaces
    InnoDB: Allocated tablespace ID 1011 for mysql/slave_gtid_info, old maximum was 0
    InnoDB: Operating system error number 24 in a file operation.
    InnoDB: Error number 24 means 'Too many open files'
    InnoDB: Some operating system error numbers are described at http://dev.mysql.com/doc/refman/5.7/en/operating-system-error-codes.html
    InnoDB: File ./netplat/sys_area_manage.ibd: 'open' returned OS error 124. Cannot continue operation
    InnoDB: Cannot continue operation.
    
    opened by czxin788 0
  • 日志文件清理

    日志文件清理

    log下的cron.log有办法定时清理或者保留几天配置吗

    opened by sunyike 0
  • build(deps): bump dns-packet from 1.3.1 to 1.3.4 in /web/vue

    build(deps): bump dns-packet from 1.3.1 to 1.3.4 in /web/vue

    Bumps dns-packet from 1.3.1 to 1.3.4.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • k8s 多副本

    k8s 多副本

    node agent 、gocron 是否支持多副本? gocron 支持的最大任务量大概有多大?

    opened by legend-goat 0
  • Feature: add response handle from timeout

    Feature: add response handle from timeout

    ChangeLog

    • 增加对http请求超时后resp的处理

    Please review it when u are free, thanks so much @ouqiang

    opened by WYGwyg 0
Releases(v1.5.3)
Owner
qiang.ou
to be a better man
qiang.ou