聊聊micrometer的HistogramGauges

序

本文主要研究一下micrometer的HistogramGauges

AutoConfiguration

针对springboot应用，配备有各种export的AutoConfiguration，详见org.springframework.boot.actuate.autoconfigure.metrics.export包，2.0.1版本目前支持了如下类型的export：

atlas、datadog、ganglia、graphite、influx、jmx、newrelic、prometheus、properties、signalfx、simple、statsd、wavefront

这里看下statsd及prometheus的AutoConfiguration

StatsdMetricsExportAutoConfiguration

spring-boot-actuator-autoconfigure-2.0.1.RELEASE-sources.jar!/org/springframework/boot/actuate/autoconfigure/metrics/export/statsd/StatsdMetricsExportAutoConfiguration.java

@Configuration
@AutoConfigureBefore({ CompositeMeterRegistryAutoConfiguration.class,
        SimpleMetricsExportAutoConfiguration.class })
@AutoConfigureAfter(MetricsAutoConfiguration.class)
@ConditionalOnBean(Clock.class)
@ConditionalOnClass(StatsdMeterRegistry.class)
@ConditionalOnProperty(prefix = "management.metrics.export.statsd", name = "enabled", havingValue = "true", matchIfMissing = true)
@EnableConfigurationProperties(StatsdProperties.class)
public class StatsdMetricsExportAutoConfiguration {

    @Bean
    @ConditionalOnMissingBean
    public StatsdConfig statsdConfig(StatsdProperties statsdProperties) {
        return new StatsdPropertiesConfigAdapter(statsdProperties);
    }

    @Bean
    @ConditionalOnMissingBean
    public StatsdMeterRegistry statsdMeterRegistry(StatsdConfig statsdConfig,
            Clock clock) {
        return new StatsdMeterRegistry(statsdConfig, clock);
    }

    @Bean
    public StatsdMetrics statsdMetrics() {
        return new StatsdMetrics();
    }

}

可以看到，创建了StatsdMeterRegistry

PrometheusMetricsExportAutoConfiguration

spring-boot-actuator-autoconfigure-2.0.1.RELEASE-sources.jar!/org/springframework/boot/actuate/autoconfigure/metrics/export/prometheus/PrometheusMetricsExportAutoConfiguration.java

@Configuration
@AutoConfigureBefore({ CompositeMeterRegistryAutoConfiguration.class,
        SimpleMetricsExportAutoConfiguration.class })
@AutoConfigureAfter(MetricsAutoConfiguration.class)
@ConditionalOnBean(Clock.class)
@ConditionalOnClass(PrometheusMeterRegistry.class)
@ConditionalOnProperty(prefix = "management.metrics.export.prometheus", name = "enabled", havingValue = "true", matchIfMissing = true)
@EnableConfigurationProperties(PrometheusProperties.class)
public class PrometheusMetricsExportAutoConfiguration {

    @Bean
    @ConditionalOnMissingBean
    public PrometheusConfig prometheusConfig(PrometheusProperties prometheusProperties) {
        return new PrometheusPropertiesConfigAdapter(prometheusProperties);
    }

    @Bean
    @ConditionalOnMissingBean
    public PrometheusMeterRegistry prometheusMeterRegistry(
            PrometheusConfig prometheusConfig, CollectorRegistry collectorRegistry,
            Clock clock) {
        return new PrometheusMeterRegistry(prometheusConfig, collectorRegistry, clock);
    }

    @Bean
    @ConditionalOnMissingBean
    public CollectorRegistry collectorRegistry() {
        return new CollectorRegistry(true);
    }

    @ManagementContextConfiguration
    public static class PrometheusScrapeEndpointConfiguration {

        @Bean
        @ConditionalOnEnabledEndpoint
        @ConditionalOnMissingBean
        public PrometheusScrapeEndpoint prometheusEndpoint(
                CollectorRegistry collectorRegistry) {
            return new PrometheusScrapeEndpoint(collectorRegistry);
        }

    }

}

可以看到创建了PrometheusMeterRegistry

Timer.register

micrometer-core-1.0.3-sources.jar!/io/micrometer/core/instrument/Timer.java

        /**
         * Add the timer to a single registry, or return an existing timer in that registry. The returned
         * timer will be unique for each registry, but each registry is guaranteed to only create one timer
         * for the same combination of name and tags.
         *
         * @param registry A registry to add the timer to, if it doesn't already exist.
         * @return A new or existing timer.
         */
        public Timer register(MeterRegistry registry) {
            // the base unit for a timer will be determined by the monitoring system implementation
            return registry.timer(new Meter.Id(name, tags, null, description, Type.TIMER), distributionConfigBuilder.build(),
                    pauseDetector == null ? registry.config().pauseDetector() : pauseDetector);
        }

可以看到该register委托给了registry.timer方法

MeterRegistry

micrometer-core-1.0.3-sources.jar!/io/micrometer/core/instrument/MeterRegistry.java

    /**
     * Only used by {@link Timer#builder(String)}.
     *
     * @param id                          The identifier for this timer.
     * @param distributionStatisticConfig Configuration that governs how distribution statistics are computed.
     * @return A new or existing timer.
     */
    Timer timer(Meter.Id id, DistributionStatisticConfig distributionStatisticConfig, PauseDetector pauseDetectorOverride) {
        return registerMeterIfNecessary(Timer.class, id, distributionStatisticConfig, (id2, filteredConfig) -> {
            Meter.Id withUnit = id2.withBaseUnit(getBaseTimeUnitStr());
            return newTimer(withUnit, filteredConfig.merge(defaultHistogramConfig()), pauseDetectorOverride);
        }, NoopTimer::new);
    }

    /**
     * Build a new timer to be added to the registry. This is guaranteed to only be called if the timer doesn't already exist.
     *
     * @param id                          The id that uniquely identifies the timer.
     * @param distributionStatisticConfig Configuration for published distribution statistics.
     * @param pauseDetector               The pause detector to use for coordinated omission compensation.
     * @return A new timer.
     */
    protected abstract Timer newTimer(Meter.Id id, DistributionStatisticConfig distributionStatisticConfig, PauseDetector pauseDetector);

这里有调用了newTimer抽象方法

StatsdMeterRegistry.newTimer

micrometer-registry-statsd-1.0.3-sources.jar!/io/micrometer/statsd/StatsdMeterRegistry.java

    @SuppressWarnings("ConstantConditions")
    @Override
    protected Timer newTimer(Meter.Id id, DistributionStatisticConfig distributionStatisticConfig, PauseDetector
            pauseDetector) {
        Timer timer = new StatsdTimer(id, lineBuilder(id), publisher, clock, distributionStatisticConfig, pauseDetector, getBaseTimeUnit(),
                statsdConfig.step().toMillis());
        HistogramGauges.registerWithCommonFormat(timer, this);
        return timer;
    }

可以看到newTimer操作里头调用了HistogramGauges.registerWithCommonFormat(timer, this);

HistogramGauges.registerWithCommonFormat

micrometer-core-1.0.3-sources.jar!/io/micrometer/core/instrument/distribution/HistogramGauges.java

    /**
     * Register a set of gauges for percentiles and histogram buckets that follow a common format when
     * the monitoring system doesn't have an opinion about the structure of this data.
     */
    public static HistogramGauges registerWithCommonFormat(Timer timer, MeterRegistry registry) {
        Meter.Id id = timer.getId();
        return HistogramGauges.register(timer, registry,
                percentile -> id.getName() + ".percentile",
                percentile -> Tags.concat(id.getTags(), "phi", DoubleFormat.decimalOrNan(percentile.percentile())),
                percentile -> percentile.value(timer.baseTimeUnit()),
                bucket -> id.getName() + ".histogram",
                bucket -> Tags.concat(id.getTags(), "le", DoubleFormat.decimalOrWhole(bucket.bucket(timer.baseTimeUnit()))));
    }

可以看到这里使用HistogramGauges进行注册，percentileName的名称为id.getName() + ".percentile"，bucketName的名称为id.getName() + ".histogram"

HistogramGauges

micrometer-core-1.0.3-sources.jar!/io/micrometer/core/instrument/distribution/HistogramGauges.java

    private HistogramGauges(HistogramSupport meter, MeterRegistry registry,
                            Function<ValueAtPercentile, String> percentileName,
                            Function<ValueAtPercentile, Iterable<Tag>> percentileTags,
                            Function<ValueAtPercentile, Double> percentileValue,
                            Function<CountAtBucket, String> bucketName,
                            Function<CountAtBucket, Iterable<Tag>> bucketTags) {
        this.meter = meter;

        HistogramSnapshot initialSnapshot = meter.takeSnapshot();
        this.snapshot = initialSnapshot;

        ValueAtPercentile[] valueAtPercentiles = initialSnapshot.percentileValues();
        CountAtBucket[] countAtBuckets = initialSnapshot.histogramCounts();

        this.totalGauges = valueAtPercentiles.length + countAtBuckets.length;

        // set to zero initially, so the first polling of one of the gauges on each publish cycle results in a
        // new snapshot
        this.polledGaugesLatch = new CountDownLatch(0);

        for (int i = 0; i < valueAtPercentiles.length; i++) {
            final int index = i;

            ToDoubleFunction<HistogramSupport> percentileValueFunction = m -> {
                snapshotIfNecessary();
                polledGaugesLatch.countDown();
                return percentileValue.apply(snapshot.percentileValues()[index]);
            };

            Gauge.builder(percentileName.apply(valueAtPercentiles[i]), meter, percentileValueFunction)
                    .tags(percentileTags.apply(valueAtPercentiles[i]))
                    .register(registry);
        }

        for (int i = 0; i < countAtBuckets.length; i++) {
            final int index = i;

            ToDoubleFunction<HistogramSupport> bucketCountFunction = m -> {
                snapshotIfNecessary();
                polledGaugesLatch.countDown();
                return snapshot.histogramCounts()[index].count();
            };

            Gauge.builder(bucketName.apply(countAtBuckets[i]), meter, bucketCountFunction)
                    .tags(bucketTags.apply(countAtBuckets[i]))
                    .register(registry);
        }
    }

可以看到这里针对HistogramSnapshot取了percentileValues注册了Gauge，然后针对HistogramSnapshot的CountAtBucket[]注册了对应的Gauge

实例

SimpleMeterRegistry simpleMeterRegistry = new SimpleMeterRegistry();
    @Test
    public void testHistogramGauges() throws InterruptedException {
        Timer timer = Timer.builder("api-cost")
                .publishPercentileHistogram()
                .publishPercentiles(0.95,0.99)
                .register(simpleMeterRegistry);

        IntStream.rangeClosed(1,1000)
                .forEach(i -> {
                    timer.record(Duration.ofMillis(ThreadLocalRandom.current().nextInt(200)));
                    simpleMeterRegistry.getMeters()
                            .stream()
                            .forEach(m -> {
                                System.out.println(m.getId() + "-->" + m.measure());
                            });
                });
        TimeUnit.MINUTES.sleep(5);
    }

输出实例

MeterId{name='api-cost.percentile', tags=[ImmutableTag{key='phi', value='0.95'}]}-->[Measurement{statistic='VALUE', value=0.192905216}]
MeterId{name='api-cost.percentile', tags=[ImmutableTag{key='phi', value='0.99'}]}-->[Measurement{statistic='VALUE', value=0.201293824}]
MeterId{name='api-cost', tags=[]}-->[Measurement{statistic='COUNT', value=999.0}, Measurement{statistic='TOTAL_TIME', value=97.158}, Measurement{statistic='MAX', value=0.199}]
MeterId{name='api-cost.percentile', tags=[ImmutableTag{key='phi', value='0.95'}]}-->[Measurement{statistic='VALUE', value=0.192905216}]
MeterId{name='api-cost.percentile', tags=[ImmutableTag{key='phi', value='0.99'}]}-->[Measurement{statistic='VALUE', value=0.201293824}]
MeterId{name='api-cost', tags=[]}-->[Measurement{statistic='COUNT', value=1000.0}, Measurement{statistic='TOTAL_TIME', value=97.348}, Measurement{statistic='MAX', value=0.199}]

小结

目前只有Prometheus和Atlas支持Percentile histograms，不过micrometer在client端简单支持了下percentile，不过不像server端支持那么灵活，不能跨tag进行聚合，目前是把tag作为meter id的一部分，一起上报。针对qps的计算，可以使用Timer类型来计量，然后通过percentile指标，根据时间间隔进行group来统计。

doc

• 13. Histograms and percentiles