聊聊httpclient的监控

序

本文主要研究一下如何监控httpclient

MicrometerHttpRequestExecutor

micrometer-core-1.3.0-sources.jar!/io/micrometer/core/instrument/binder/httpcomponents/MicrometerHttpRequestExecutor.java

@Incubating(since = "1.2.0")
public class MicrometerHttpRequestExecutor extends HttpRequestExecutor {

    /**
     * Default header name for URI pattern.
     */
    public static final String DEFAULT_URI_PATTERN_HEADER = "URI_PATTERN";

    private static final String METER_NAME = "httpcomponents.httpclient.request";
    private static final String UNKNOWN = "UNKNOWN";

    private static final Tag STATUS_UNKNOWN = Tag.of("status", UNKNOWN);
    private static final Tag STATUS_CLIENT_ERROR = Tag.of("status", "CLIENT_ERROR");
    private static final Tag STATUS_IO_ERROR = Tag.of("status", "IO_ERROR");

    private final MeterRegistry registry;
    private final Function<HttpRequest, String> uriMapper;
    private final Iterable<Tag> extraTags;
    private final boolean exportTagsForRoute;

    /**
     * Use {@link #builder(MeterRegistry)} to create an instance of this class.
     */
    private MicrometerHttpRequestExecutor(int waitForContinue,
                                          MeterRegistry registry,
                                          Function<HttpRequest, String> uriMapper,
                                          Iterable<Tag> extraTags,
                                          boolean exportTagsForRoute) {
        super(waitForContinue);
        this.registry = Optional.ofNullable(registry).orElseThrow(() -> new IllegalArgumentException("registry is required but has been initialized with null"));
        this.uriMapper = Optional.ofNullable(uriMapper).orElseThrow(() -> new IllegalArgumentException("uriMapper is required but has been initialized with null"));
        this.extraTags = Optional.ofNullable(extraTags).orElse(Collections.emptyList());
        this.exportTagsForRoute = exportTagsForRoute;
    }

    //......
}    

MicrometerHttpRequestExecutor继承了HttpRequestExecutor，它覆盖了execute方法，使用timer来监控，定义了method、uri、status、routeTags(target.scheme、target.host、target.port)、extraTags这几个tag，metric名为httpcomponents.httpclient.request。

      HttpClientBuilder.create()
          .setRequestExecutor(MicrometerHttpRequestExecutor
                  .builder(meterRegistry)
                  .build())
          .build();

具体使用可以通过setRequestExecutor来设置MicrometerHttpRequestExecutor

Builder

    public static Builder builder(MeterRegistry registry) {
        return new Builder(registry);
    }

    public static class Builder {
        private final MeterRegistry registry;
        private int waitForContinue = HttpRequestExecutor.DEFAULT_WAIT_FOR_CONTINUE;
        private Iterable<Tag> tags = Collections.emptyList();
        private Function<HttpRequest, String> uriMapper = new DefaultUriMapper();
        private boolean exportTagsForRoute = false;

        Builder(MeterRegistry registry) {
            this.registry = registry;
        }

        /**
         * @param waitForContinue Overrides the wait for continue time for this
         *                        request executor. See {@link HttpRequestExecutor}
         *                        for details.
         * @return This builder instance.
         */
        public Builder waitForContinue(int waitForContinue) {
            this.waitForContinue = waitForContinue;
            return this;
        }

        /**
         * @param tags Additional tags which should be exposed with every value.
         * @return This builder instance.
         */
        public Builder tags(Iterable<Tag> tags) {
            this.tags = tags;
            return this;
        }

        /**
         * Allows to register a mapping function for exposing request URIs. Be
         * careful, exposing request URIs could result in a huge number of tag
         * values, which could cause problems in your meter registry.
         *
         * By default, this feature is almost disabled. It only exposes values
         * of the {@link #DEFAULT_URI_PATTERN_HEADER} HTTP header.
         *
         * @param uriMapper A mapper that allows mapping and exposing request
         *                  paths.
         * @return This builder instance.
         * @see DefaultUriMapper
         */
        public Builder uriMapper(Function<HttpRequest, String> uriMapper) {
            this.uriMapper = uriMapper;
            return this;
        }

        /**
         * Allows to expose the target scheme, host and port with every metric.
         * Be careful with enabling this feature: If your client accesses a huge
         * number of remote servers, this would result in a huge number of tag
         * values, which could cause cardinality problems.
         *
         * By default, this feature is disabled.
         *
         * @param exportTagsForRoute Set this to true, if the metrics should be
         *                           tagged with the target route.
         * @return This builder instance.
         */
        public Builder exportTagsForRoute(boolean exportTagsForRoute) {
            this.exportTagsForRoute = exportTagsForRoute;
            return this;
        }

        /**
         * @return Creates an instance of {@link MicrometerHttpRequestExecutor}
         * with all the configured properties.
         */
        public MicrometerHttpRequestExecutor build() {
            return new MicrometerHttpRequestExecutor(waitForContinue, registry, uriMapper, tags, exportTagsForRoute);
        }
    }

MicrometerHttpRequestExecutor提供了builder方法，可以设置waitForContinue、tags、uriMapper(默认是DefaultUriMapper)、exportTagsForRoute

DefaultUriMapper

    /**
     * Extracts the pattern from the request header of the request if available.
     */
    private static class DefaultUriMapper implements Function<HttpRequest, String> {
        @Override
        public String apply(HttpRequest httpRequest) {
            Header uriPattern = httpRequest.getLastHeader(DEFAULT_URI_PATTERN_HEADER);
            if (uriPattern != null && uriPattern.getValue() != null) {
                return uriPattern.getValue();
            }
            return UNKNOWN;
        }
    }

DefaultUriMapper用于获取uriPattern

MicrometerHttpClientInterceptor

micrometer-core-1.5.9.jar!/io/micrometer/core/instrument/binder/httpcomponents/MicrometerHttpClientInterceptor.java

@Incubating(
    since = "1.4.0"
)
public class MicrometerHttpClientInterceptor {
    private static final String METER_NAME = "httpcomponents.httpclient.request";
    private final Map<HttpContext, Timer.Sample> timerByHttpContext;
    private final HttpRequestInterceptor requestInterceptor;
    private final HttpResponseInterceptor responseInterceptor;

    public MicrometerHttpClientInterceptor(MeterRegistry meterRegistry, Function<HttpRequest, String> uriMapper, Iterable<Tag> extraTags, boolean exportTagsForRoute) {
        this.timerByHttpContext = new ConcurrentHashMap();
        this.requestInterceptor = (request, context) -> {
            this.timerByHttpContext.put(context, Timer.start(meterRegistry).tags(new String[]{"method", request.getRequestLine().getMethod(), "uri", (String)uriMapper.apply(request)}));
        };
        this.responseInterceptor = (response, context) -> {
            Timer.Sample sample = (Timer.Sample)this.timerByHttpContext.remove(context);
            sample.stop(meterRegistry, Timer.builder("httpcomponents.httpclient.request").tag("status", Integer.toString(response.getStatusLine().getStatusCode())).tags(exportTagsForRoute ? HttpContextUtils.generateTagsForRoute(context) : Tags.empty()).tags(extraTags));
        };
    }

    public MicrometerHttpClientInterceptor(MeterRegistry meterRegistry, Iterable<Tag> extraTags, boolean exportTagsForRoute) {
        this(meterRegistry, new DefaultUriMapper(), extraTags, exportTagsForRoute);
    }

    public HttpRequestInterceptor getRequestInterceptor() {
        return this.requestInterceptor;
    }

    public HttpResponseInterceptor getResponseInterceptor() {
        return this.responseInterceptor;
    }
}

micrometer1.4.0版本开始提供了MicrometerHttpClientInterceptor，它定义了requestInterceptor、responseInterceptor，通过timer来上报名为httpcomponents.httpclient.request,tag为method、uri、status、exportTagsForRoute、extraTags的指标

PoolingHttpClientConnectionManagerMetricsBinder

io/micrometer/core/instrument/binder/httpcomponents/PoolingHttpClientConnectionManagerMetricsBinder.java

public class PoolingHttpClientConnectionManagerMetricsBinder implements MeterBinder {

    private final PoolingHttpClientConnectionManager connectionManager;
    private final Iterable<Tag> tags;

    /**
     * Creates a metrics binder for the given pooling connection manager.
     *
     * @param connectionManager The connection manager to monitor.
     * @param name Name of the connection manager. Will be added as tag with the
     *             key "httpclient".
     * @param tags Tags to apply to all recorded metrics. Must be an even number
     *             of arguments representing key/value pairs of tags.
     */
    @SuppressWarnings("WeakerAccess")
    public PoolingHttpClientConnectionManagerMetricsBinder(PoolingHttpClientConnectionManager connectionManager, String name, String... tags) {
        this(connectionManager, name, Tags.of(tags));
    }

    /**
     * Creates a metrics binder for the given pooling connection manager.
     *
     * @param connectionManager The connection manager to monitor.
     * @param name Name of the connection manager. Will be added as tag with the
     *             key "httpclient".
     * @param tags Tags to apply to all recorded metrics.
     */
    @SuppressWarnings("WeakerAccess")
    public PoolingHttpClientConnectionManagerMetricsBinder(PoolingHttpClientConnectionManager connectionManager, String name, Iterable<Tag> tags) {
        this.connectionManager = connectionManager;
        this.tags = Tags.concat(tags, "httpclient", name);
    }

    @Override
    public void bindTo(@NonNull MeterRegistry registry) {
        registerTotalMetrics(registry);
    }

    private void registerTotalMetrics(MeterRegistry registry) {
        Gauge.builder("httpcomponents.httpclient.pool.total.max",
            connectionManager,
            (connectionManager) -> connectionManager.getTotalStats().getMax())
            .description("The configured maximum number of allowed persistent connections for all routes.")
            .tags(tags)
            .register(registry);
        Gauge.builder("httpcomponents.httpclient.pool.total.connections",
            connectionManager,
            (connectionManager) -> connectionManager.getTotalStats().getAvailable())
            .description("The number of persistent and leased connections for all routes.")
            .tags(tags).tag("state", "available")
            .register(registry);
        Gauge.builder("httpcomponents.httpclient.pool.total.connections",
            connectionManager,
            (connectionManager) -> connectionManager.getTotalStats().getLeased())
            .description("The number of persistent and leased connections for all routes.")
            .tags(tags).tag("state", "leased")
            .register(registry);
        Gauge.builder("httpcomponents.httpclient.pool.total.pending",
            connectionManager,
            (connectionManager) -> connectionManager.getTotalStats().getPending())
            .description("The number of connection requests being blocked awaiting a free connection for all routes.")
            .tags(tags)
            .register(registry);
        Gauge.builder("httpcomponents.httpclient.pool.route.max.default",
            connectionManager,
            PoolingHttpClientConnectionManager::getDefaultMaxPerRoute)
            .description("The configured default maximum number of allowed persistent connections per route.")
            .tags(tags)
            .register(registry);
    }

}

PoolingHttpClientConnectionManagerMetricsBinder实现了MeterBinder接口，它构造器接收PoolingHttpClientConnectionManager、name、tags，其bindTo方法定义了如下几个metrics

• httpcomponents.httpclient.pool.total.max
• httpcomponents.httpclient.pool.total.connections, tag:state=available
• httpcomponents.httpclient.pool.total.connections, tag:state=leased
• httpcomponents.httpclient.pool.total.pending
• httpcomponents.httpclient.pool.route.max.default

小结

micrometer为apache httpclient提供了对应的binder，用于上报相关metrics，其中1.2.0版本提供了MicrometerHttpRequestExecutor(httpcomponents.httpclient.request)，1.4.0版本提供了MicrometerHttpClientInterceptor(httpcomponents.httpclient.request)，另外对于连接池提供了PoolingHttpClientConnectionManagerMetricsBinder(httpcomponents.httpclient.pool)进行监控上报。