django+celery+beat 多个同一时间定时任务调度,但是只有一个任务可执行。

新手上路,请多包涵

Django+celery+beat 开启定时调度,其中有三个定时任务,是定时执行,频率是1分钟1次,但是发现有些任务如下所示:

celery 调度日志

[2020-03-20 17:25:00,005: INFO/Beat] Scheduler: Sending due task 定时任务测试1 (db.tasks.CrontabCeleryTest1)
[2020-03-20 17:26:00,004: INFO/Beat] Scheduler: Sending due task 定时任务测试1 (db.tasks.CrontabCeleryTest1)
[2020-03-20 17:27:00,004: INFO/Beat] Scheduler: Sending due task 定时任务测试1 (db.tasks.CrontabCeleryTest1)
[2020-03-20 17:28:00,005: INFO/Beat] Scheduler: Sending due task 定时任务测试1 (db.tasks.CrontabCeleryTest1)

celery接受任务日志

[2020-03-20 17:25:00,011: INFO/MainProcess] Received task: db.tasks.CrontabCeleryTest1[57d04272-2e7b-47b8-8d2e-067959a96f40]
[2020-03-20 17:27:00,009: INFO/MainProcess] Received task: db.tasks.CrontabCeleryTest1[df859846-a058-4696-976f-4005a68c2c28]
[2020-03-20 17:28:00,010: INFO/MainProcess] Received task: db.tasks.CrontabCeleryTest1[6795a647-3d8c-42e4-9fe9-8ad4727aa683]

celery执行返回结果

[2020-03-20 17:25:00,024: INFO/MainProcess] Task db.tasks.CrontabCeleryTest1[57d04272-2e7b-47b8-8d2e-067959a96f40] succeeded in 0.011741816997528076s: '2020-03-20 17:25:00'
[2020-03-20 17:27:00,022: INFO/MainProcess] Task db.tasks.CrontabCeleryTest1[df859846-a058-4696-976f-4005a68c2c28] succeeded in 0.011453032493591309s: '2020-03-20 17:27:00'
[2020-03-20 17:28:00,021: INFO/MainProcess] Task db.tasks.CrontabCeleryTest1[6795a647-3d8c-42e4-9fe9-8ad4727aa683] succeeded in 0.010070644319057465s: '2020-03-20 17:28:00'

可以比较明显的发现scheduler调度器是有sending是每分钟发送了任务CrontabCeleryTest1,但是实际Received和执行成功却缺失了17:26时刻的任务。

Celery的配置信息

#消息中间件
BROKER_URL = 'redis://:{REDIS_PASSWORD}@{REDIS_HOST}:{REDIS_PORT}/{REDIS_DB}'.format(REDIS_PASSWORD=REDIS_PASSWORD,REDIS_HOST=REDIS_HOST,REDIS_PORT=REDIS_PORT,REDIS_DB=REDIS_DB)


#定时任务
CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'  ###

#结果存储
CELERY_RESULT_BACKEND = 'djcelery.backends.database:DatabaseBackend' # 指定 Backend
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'

CELERY_TIMEZONE = 'Asia/Shanghai'
CELERYD_FORCE_EXECV = True #防止死锁

CELERYD_TASK_TIME_LIMIT = 86400 #任务时长

CELERY_DISABLE_RATE_LIMITS = True


# CELERY_RESULT_BACKEND = 'django-cache'
CELERYD_MAX_TASKS_PER_CHILD = 5
CELERY_ENABLE_UTC = False

Celery的启动
nohup python manage.py celery worker --app=${app} --loglevel=info --concurrency=6 -n ${HOSTNAME} --beat &>logs/celery.log &

请问一下 这个主要是什么为什么?有好的办法或者方案吗?

阅读 3.8k
撰写回答
你尚未登录,登录后可以
  • 和开发者交流问题的细节
  • 关注并接收问题和回答的更新提醒
  • 参与内容的编辑和改进,让解决方法与时俱进
推荐问题