[case46]聊聊storm trident spout的_maxTransactionActive

2018-11-18
阅读 19 分钟
1.9k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/topology/MasterBatchCoordinator.java

聊聊storm trident batch的分流与聚合

2018-11-17
阅读 18 分钟
2.5k
b-1主要是处理CombinerAggregator的init,它的parallelismHint为1,由于它的上游bolt有3个task,因而它的TridentBoltExecutor的tracked.condition.expectedTaskReports为3,它要等到这三个task的聚合数据都到了之后,才能finishBatch

聊聊storm TridentBoltExecutor的finishBatch方法

2018-11-16
阅读 14 分钟
2.6k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/topology/MasterBatchCoordinator.java

聊聊storm的AggregateProcessor的execute及finishBatch方法

2018-11-15
阅读 15 分钟
2.8k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/topology/TridentBoltExecutor.java

聊聊storm WindowTridentProcessor的FreshCollector

2018-11-14
阅读 15 分钟
2.1k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/windowing/WindowTridentProcessor.java

聊聊storm TridentWindowManager的pendingTriggers

2018-11-13
阅读 13 分钟
2k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/topology/TridentBoltExecutor.java

聊聊storm的window trigger

2018-11-12
阅读 8 分钟
2.2k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/windowing/WindowTridentProcessor.java

[case45]聊聊storm-kafka-client的ProcessingGuarantee

2018-11-11
阅读 30 分钟
3.2k
storm-kafka-client-1.2.2-sources.jar!/org/apache/storm/kafka/spout/KafkaSpoutConfig.java

聊聊storm trident的coordinator

2018-11-10
阅读 30 分钟
3.3k
storm-1.2.2/storm-core/src/jvm/org/apache/storm/trident/topology/MasterBatchCoordinator.java

聊聊storm TridentTopology的构建

2018-11-09
阅读 23 分钟
3.5k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/TridentTopology.java

聊聊storm tuple的序列化

2018-11-08
阅读 23 分钟
4k
storm-2.0.0/storm-client/src/jvm/org/apache/storm/executor/ExecutorTransfer.java

聊聊storm的LoggingClusterMetricsConsumer

2018-11-07
阅读 24 分钟
2.4k
storm-2.0.0/storm-server/src/main/java/org/apache/storm/metric/LoggingClusterMetricsConsumer.java

聊聊storm的LoggingMetricsConsumer

2018-11-06
阅读 23 分钟
2.7k
storm-2.0.0/storm-client/src/jvm/org/apache/storm/metric/LoggingMetricsConsumer.java

聊聊storm的IEventLogger

2018-11-05
阅读 12 分钟
1.8k
storm-2.0.0/storm-client/src/jvm/org/apache/storm/metric/IEventLogger.java

[case44]聊聊storm trident的operations

2018-11-04
阅读 18 分钟
2.3k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/operation/Function.java

聊聊storm trident的state

2018-11-03
阅读 11 分钟
2.1k
storm-2.0.0/storm-client/src/jvm/org/apache/storm/trident/state/StateType.java

聊聊storm的CheckpointSpout

2018-11-02
阅读 18 分钟
2.2k
storm-2.0.0/storm-client/src/jvm/org/apache/storm/topology/TopologyBuilder.java

聊聊storm client的netty buffer watermark

2018-11-01
阅读 13 分钟
3k
这里有两个相关的参数,分别是storm.messaging.netty.buffer.high.watermark以及storm.messaging.netty.buffer.low.watermark

聊聊storm的messageTimeout

2018-10-31
阅读 10 分钟
2.6k
序 本文主要研究一下storm的messageTimeout TOPOLOGY_MESSAGE_TIMEOUT_SECS storm-2.0.0/storm-client/src/jvm/org/apache/storm/Config.java {代码...} defaults.yaml中topology.enable.message.timeouts默认为true defaults.yaml中topology.message.timeout.secs默认为30 defaults.yaml中topology.tick.tuple.freq.sec...

聊聊storm的maxSpoutPending

2018-10-30
阅读 13 分钟
3.4k
TOPOLOGY_MAX_SPOUT_PENDING设置的是一个spout task已经emit等待ack的tuple的最大数量,该配置仅仅对于发射可靠tuple(设置msgId)的spout起作用

聊聊storm的IWaitStrategy

2018-10-29
阅读 18 分钟
2.8k
storm-2.0.0/storm-client/src/jvm/org/apache/storm/policy/IWaitStrategy.java

[case43]聊聊storm的LinearDRPCTopologyBuilder

2018-10-28
阅读 21 分钟
2k
DRPCSpout的outputFields为Fields("args", "return-info"),ReturnResults接收的fields为Fields("result", "return-info")

聊聊storm的stream的分流与合并

2018-10-27
阅读 4 分钟
3.2k
这里在SplitStreamBolt里头将stream分为两个,之后有两个CountStreamBolt分别处理两个stream的数据,最后归到同一个stream由ReportBolt消费tuple

聊聊storm的JoinBolt

2018-10-26
阅读 17 分钟
2.2k
JoinBolt继承了BaseWindowedBolt,定义了Selector selectorType、LinkedHashMap<String, JoinInfo> joinCriteria、FieldSelector[] outputFields等属性,用于记录关联类型及关联关系

聊聊storm的WindowedBoltExecutor

2018-10-25
阅读 26 分钟
1.9k
storm-2.0.0/storm-client/src/jvm/org/apache/storm/topology/WindowedBoltExecutor.java

聊聊storm的WindowedBolt

2018-10-24
阅读 14 分钟
2.6k
序 本文主要研究一下storm的WindowedBolt 实例 {代码...} 这里主要设置了withWindow、withTimestampField、withWatermarkInterval、withLag SlidingWindowSumBolt {代码...} TupleWindow可以获取三类值,一类是当前窗口里头的所有数据,一类是上次窗口后新到达的数据,一类是过期的数据 WindowedBolt IWindowedBolt stor...

聊聊storm的reportError

2018-10-23
阅读 25 分钟
2.4k
storm-2.0.0/storm-client/src/jvm/org/apache/storm/task/IErrorReporter.java

storm drpc实例

2018-10-22
阅读 5 分钟
2.9k
这里对supervisor配置drpc.servers及drpc.port、drpc.invocations.port,好让worker通过drpc.invocations.port去访问drpc节点

[case42]聊聊storm的ack机制

2018-10-21
阅读 23 分钟
4k
对于bolt来说,要做两件事情,一是要anchor,在emit的时候把输入及输出tuple连接起来,构建tuple tree;而要对处理完的tuple进行ack,失败进行fail操作

聊聊storm的tickTuple

2018-10-20
阅读 14 分钟
3.7k
这里实例是重写getComponentConfiguration方法,直接new了一个conf,设置了Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS参数