聊聊jesque在redis中的数据结构

codecraft

本文主要介绍一下jesque在redis的存储结构

示例配置

namespace=demo
listen-queue=demoqueue

启动时的reids对象

127.0.0.1:6379> keys *
1) "demo:worker:Mars.local:3820-3:JAVA_DYNAMIC_QUEUES,demoqueue:started"
2) "demo:worker:Mars.local:3820-1:JAVA_DYNAMIC_QUEUES,demoqueue:started"
3) "demo:worker:Mars.local:3820-0:JAVA_DYNAMIC_QUEUES,demoqueue:started"
4) "demo:workers"
5) "demo:worker:Mars.local:3820-2:JAVA_DYNAMIC_QUEUES,demoqueue:started"
6) "demo:worker:Mars.local:3820-4:JAVA_DYNAMIC_QUEUES,demoqueue:started"
127.0.0.1:6379> type demo:workers
set
127.0.0.1:6379> smembers demo:workers
1) "Mars.local:3820-4:JAVA_DYNAMIC_QUEUES,demoqueue"
2) "Mars.local:3820-2:JAVA_DYNAMIC_QUEUES,demoqueue"
3) "Mars.local:3820-1:JAVA_DYNAMIC_QUEUES,demoqueue"
4) "Mars.local:3820-0:JAVA_DYNAMIC_QUEUES,demoqueue"
5) "Mars.local:3820-3:JAVA_DYNAMIC_QUEUES,demoqueue"

namespace:workers

set类型的数据接口,存储该namespace下的workers的名称

namespace:worker:host:queueType,listen-queue:state

这个是标记每个namespace的worker的状态,是个string类型,存储事件产生时间
127.0.0.1:6379> type demo:worker:Mars.local:3820-3:JAVA_DYNAMIC_QUEUES,demoqueue:started
string
127.0.0.1:6379> get demo:worker:Mars.local:3820-3:JAVA_DYNAMIC_QUEUES,demoqueue:started
"2017-11-18T15:35:24.913+0800"

新增job

127.0.0.1:6379> keys *
1) "demo:worker:Mars.local:3820-3:JAVA_DYNAMIC_QUEUES,demoqueue:started"
2) "demo:queue:demoqueue"
3) "demo:worker:Mars.local:3820-1:JAVA_DYNAMIC_QUEUES,demoqueue:started"
4) "demo:worker:Mars.local:3820-0:JAVA_DYNAMIC_QUEUES,demoqueue:started"
5) "demo:queues"
6) "demo:workers"
7) "demo:worker:Mars.local:3820-2:JAVA_DYNAMIC_QUEUES,demoqueue:started"
8) "demo:worker:Mars.local:3820-4:JAVA_DYNAMIC_QUEUES,demoqueue:started"
可以发现新增了几个queue相关的对象
demo:queue:demoqueue这个是个锁信息

namespace:queues

是一个set结构,存储队列信息
127.0.0.1:6379> smembers demo:queues
1) "demoqueue"

处理完job

127.0.0.1:6379> keys *
1) "demo:worker:Mars.local:3820-1:JAVA_DYNAMIC_QUEUES,demoqueue:started"
2) "demo:worker:Mars.local:3820-0:JAVA_DYNAMIC_QUEUES,demoqueue:started"
3) "demo:queues"
4) "demo:stat:processed"
5) "demo:stat:processed:Mars.local:3820-0:JAVA_DYNAMIC_QUEUES,demoqueue"
6) "demo:worker:Mars.local:3820-3:JAVA_DYNAMIC_QUEUES,demoqueue:started"
7) "demo:workers"
8) "demo:worker:Mars.local:3820-4:JAVA_DYNAMIC_QUEUES,demoqueue:started"
9) "demo:worker:Mars.local:3820-2:JAVA_DYNAMIC_QUEUES,demoqueue:started"
可以发现新增了两个stat相关的对象

namespace:stat:processed

是string类型,统计处理过的job的数量
127.0.0.1:6379> type demo:stat:processed
string
127.0.0.1:6379> get demo:stat:processed
"1"

namespace:stat:processed:host:queueType,listen-queue

存储每个host处理每个queue的数量
127.0.0.1:6379> type demo:stat:processed:Mars.local:3820-0:JAVA_DYNAMIC_QUEUES,demoqueue
string
127.0.0.1:6379> get demo:stat:processed:Mars.local:3820-0:JAVA_DYNAMIC_QUEUES,demoqueue
"1"

关闭worker的时候

127.0.0.1:6379> keys *
1) "demo:queues"
2) "demo:stat:processed"
就只剩下两个信息

job执行失败的时候

127.0.0.1:6379> keys *
 1) "demo:stat:failed"
 2) "demo:failed"
 3) "demo:queues"
 4) "demo:stat:processed"
 5) "demo:worker:Mars.local:3947-2:JAVA_DYNAMIC_QUEUES,demoqueue:started"
 6) "demo:worker:Mars.local:3947-3:JAVA_DYNAMIC_QUEUES,demoqueue:started"
 7) "demo:worker:Mars.local:3947-1:JAVA_DYNAMIC_QUEUES,demoqueue:started"
 8) "demo:worker:Mars.local:3947-4:JAVA_DYNAMIC_QUEUES,demoqueue:started"
 9) "demo:worker:Mars.local:3947-0:JAVA_DYNAMIC_QUEUES,demoqueue:started"
10) "demo:workers"
11) "demo:stat:failed:Mars.local:3947-0:JAVA_DYNAMIC_QUEUES,demoqueue"
可以发现多了三个failed相关的对象

namespace:failed

是一个list类型
127.0.0.1:6379> type demo:failed
list
127.0.0.1:6379> lrange demo:failed 0 -1
1) "{\"worker\":\"Mars.local:3947-0:JAVA_DYNAMIC_QUEUES,demoqueue\",\"queue\":\"demoqueue\",\"payload\":{\"class\":\"com.example.demo.job.DemoJob\",\"args\":[\"111\",1510991413785],\"vars\":null,\"jobId\":\"c4c0289e-5d13-4bc3-8499-f3e35c5956a6\"},\"backtrace\":[\"\\tat com.example.demo.job.DemoJob.run(DemoJob.java:30)\",\"\\tat cn.codecraft.jesque.component.RedisPooledWorkerImpl.execute(RedisPooledWorkerImpl.java:656)\",\"\\tat cn.codecraft.jesque.component.RedisPooledWorkerImpl.process(RedisPooledWorkerImpl.java:611)\",\"\\tat cn.codecraft.jesque.component.RedisPooledWorkerImpl.poll(RedisPooledWorkerImpl.java:420)\",\"\\tat cn.codecraft.jesque.component.RedisPooledWorkerImpl.run(RedisPooledWorkerImpl.java:171)\",\"\\tat java.lang.Thread.run(Thread.java:745)\"],\"error\":\"demo ex\",\"exception\":\"java.lang.RuntimeException\",\"failed_at\":\"2017-11-18T07:50:33.921+0000\",\"retried_at\":null}"
存储了具体的job信息及异常

namespace:stat:failed

存储失败job的计数
127.0.0.1:6379> type demo:stat:failed
string
127.0.0.1:6379> get demo:stat:failed
"1"

namespace:stat:failed:host:queueType,listen-queue

存储每个host每个queue处理失败的数量
127.0.0.1:6379> type demo:stat:failed:Mars.local:3947-0:JAVA_DYNAMIC_QUEUES,demoqueue
string
127.0.0.1:6379> get demo:stat:failed:Mars.local:3947-0:JAVA_DYNAMIC_QUEUES,demoqueue
"1"
阅读 1.8k

code-craft
spring boot , docker and so on 欢迎关注微信公众号: geek_luandun

当一个代码的工匠回首往事时,不因虚度年华而悔恨,也不因碌碌无为而羞愧,这样,当他老的时候,可以很自豪告诉世人,我曾经将代码注入生命去打造互联网的浪潮之巅,那是个很疯狂的时代,我在一波波的浪潮上留下了或重如泰山或轻如鸿毛的几笔。

11.5k 声望
1.9k 粉丝
0 条评论
你知道吗?

当一个代码的工匠回首往事时,不因虚度年华而悔恨,也不因碌碌无为而羞愧,这样,当他老的时候,可以很自豪告诉世人,我曾经将代码注入生命去打造互联网的浪潮之巅,那是个很疯狂的时代,我在一波波的浪潮上留下了或重如泰山或轻如鸿毛的几笔。

11.5k 声望
1.9k 粉丝
宣传栏