聊聊flink的CsvReader

2018-11-26
阅读 21 分钟
3k
flink-java-1.6.2-sources.jar!/org/apache/flink/api/java/ExecutionEnvironment.java

[case47]聊聊flink的BoltWrapper

2018-11-25
阅读 18 分钟
2.3k
flink-storm_2.11-1.6.2-sources.jar!/org/apache/flink/storm/wrappers/BoltWrapper.java

聊聊flink的SpoutWrapper

2018-11-24
阅读 20 分钟
2.4k
flink-storm_2.11-1.6.2-sources.jar!/org/apache/flink/storm/wrappers/SpoutWrapper.java

聊聊flink如何兼容StormTopology

2018-11-23
阅读 16 分钟
3.2k
这里使用FlinkLocalCluster.getLocalCluster()来创建或获取FlinkLocalCluster,之后调用FlinkLocalCluster.submitTopology来提交topology,结束时通过FlinkLocalCluster.shutdown来关闭cluster

聊聊flink的log.file配置

2018-11-22
阅读 33 分钟
7.9k
flink-release-1.6.2/flink-dist/src/main/flink-bin/conf/log4j.properties

聊聊flink LocalEnvironment的execute方法

2018-11-21
阅读 15 分钟
5.2k
序 本文主要研究一下flink LocalEnvironment的execute方法 实例 {代码...} 这里使用DataSet从csv读取数据,然后进行flatMap、groupBy、sum操作,最后调用print输出 DataSet.print flink-java-1.6.2-sources.jar!/org/apache/flink/api/java/DataSet.java {代码...} print方法这里主要是调用collect方法,获取结果,然后...

聊聊storm的OpaquePartitionedTridentSpoutExecutor

2018-11-20
阅读 23 分钟
2.1k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/TridentTopology.java

聊聊storm的ICommitterTridentSpout

2018-11-19
阅读 24 分钟
2.2k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/spout/ICommitterTridentSpout.java

[case46]聊聊storm trident spout的_maxTransactionActive

2018-11-18
阅读 19 分钟
2k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/topology/MasterBatchCoordinator.java

聊聊storm trident batch的分流与聚合

2018-11-17
阅读 18 分钟
2.7k
b-1主要是处理CombinerAggregator的init,它的parallelismHint为1,由于它的上游bolt有3个task,因而它的TridentBoltExecutor的tracked.condition.expectedTaskReports为3,它要等到这三个task的聚合数据都到了之后,才能finishBatch

聊聊storm TridentBoltExecutor的finishBatch方法

2018-11-16
阅读 14 分钟
2.7k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/topology/MasterBatchCoordinator.java

聊聊storm的AggregateProcessor的execute及finishBatch方法

2018-11-15
阅读 15 分钟
2.9k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/topology/TridentBoltExecutor.java

聊聊storm WindowTridentProcessor的FreshCollector

2018-11-14
阅读 15 分钟
2.3k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/windowing/WindowTridentProcessor.java

聊聊storm TridentWindowManager的pendingTriggers

2018-11-13
阅读 13 分钟
2.2k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/topology/TridentBoltExecutor.java

聊聊storm的window trigger

2018-11-12
阅读 8 分钟
2.3k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/windowing/WindowTridentProcessor.java

[case45]聊聊storm-kafka-client的ProcessingGuarantee

2018-11-11
阅读 30 分钟
3.4k
storm-kafka-client-1.2.2-sources.jar!/org/apache/storm/kafka/spout/KafkaSpoutConfig.java

聊聊storm trident的coordinator

2018-11-10
阅读 30 分钟
3.4k
storm-1.2.2/storm-core/src/jvm/org/apache/storm/trident/topology/MasterBatchCoordinator.java

聊聊storm TridentTopology的构建

2018-11-09
阅读 23 分钟
3.6k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/TridentTopology.java

聊聊storm tuple的序列化

2018-11-08
阅读 23 分钟
4.2k
storm-2.0.0/storm-client/src/jvm/org/apache/storm/executor/ExecutorTransfer.java

聊聊storm的LoggingClusterMetricsConsumer

2018-11-07
阅读 24 分钟
2.5k
storm-2.0.0/storm-server/src/main/java/org/apache/storm/metric/LoggingClusterMetricsConsumer.java

聊聊storm的LoggingMetricsConsumer

2018-11-06
阅读 23 分钟
2.9k
storm-2.0.0/storm-client/src/jvm/org/apache/storm/metric/LoggingMetricsConsumer.java

聊聊storm的IEventLogger

2018-11-05
阅读 12 分钟
2k
storm-2.0.0/storm-client/src/jvm/org/apache/storm/metric/IEventLogger.java

[case44]聊聊storm trident的operations

2018-11-04
阅读 18 分钟
2.5k
storm-core-1.2.2-sources.jar!/org/apache/storm/trident/operation/Function.java

聊聊storm trident的state

2018-11-03
阅读 11 分钟
2.3k
storm-2.0.0/storm-client/src/jvm/org/apache/storm/trident/state/StateType.java

聊聊storm的CheckpointSpout

2018-11-02
阅读 18 分钟
2.4k
storm-2.0.0/storm-client/src/jvm/org/apache/storm/topology/TopologyBuilder.java

聊聊storm client的netty buffer watermark

2018-11-01
阅读 13 分钟
3.2k
这里有两个相关的参数,分别是storm.messaging.netty.buffer.high.watermark以及storm.messaging.netty.buffer.low.watermark

聊聊storm的messageTimeout

2018-10-31
阅读 10 分钟
2.8k
序 本文主要研究一下storm的messageTimeout TOPOLOGY_MESSAGE_TIMEOUT_SECS storm-2.0.0/storm-client/src/jvm/org/apache/storm/Config.java {代码...} defaults.yaml中topology.enable.message.timeouts默认为true defaults.yaml中topology.message.timeout.secs默认为30 defaults.yaml中topology.tick.tuple.freq.sec...

聊聊storm的maxSpoutPending

2018-10-30
阅读 13 分钟
3.6k
TOPOLOGY_MAX_SPOUT_PENDING设置的是一个spout task已经emit等待ack的tuple的最大数量,该配置仅仅对于发射可靠tuple(设置msgId)的spout起作用

聊聊storm的IWaitStrategy

2018-10-29
阅读 18 分钟
3k
storm-2.0.0/storm-client/src/jvm/org/apache/storm/policy/IWaitStrategy.java

[case43]聊聊storm的LinearDRPCTopologyBuilder

2018-10-28
阅读 21 分钟
2.2k
DRPCSpout的outputFields为Fields("args", "return-info"),ReturnResults接收的fields为Fields("result", "return-info")