Android Webview + HttpDns最佳实践

博客主页

1. 说明

Android WebView场景下接入HttpDns的参考方案，提供的相关代码也为参考代码，非线上生产环境正式代码。由于Android生态碎片化严重，各厂商也进行了不同程度的定制，建议您灰度接入，并监控线上异常。

2. 背景

阿里云HTTPDNS是避免dns劫持的一种有效手段，在许多特殊场景如HTTPS/SNI、okhttp等都有最佳实践，但在webview场景下却一直没完美的解决方案。

但这并不代表在WebView场景下我们完全无法使用HTTPDNS,事实上很多场景依然可以通过HTTPDNS进行IP直连，本文旨在给出Android端HTTPDNS+WebView最佳实践供用户参考。

3. 接口

void setWebViewClient(WebViewClient client)

WebView提供了 setWebViewClient 接口对网络请求进行拦截，通过重载WebViewClient中的shouldInterceptRequest方法，我们可以拦截到所有的网络请求：

public class WebViewClient {
     // API < 21
    @Deprecated
    public WebResourceResponse shouldInterceptRequest(WebView view,
            String url) {
        return null;
    }

    // API >= 21
    public WebResourceResponse shouldInterceptRequest(WebView view,
            WebResourceRequest request) {
        return shouldInterceptRequest(view, request.getUrl().toString());
    }
}

shouldInterceptRequest有两个版本：

• API < 21: public WebResourceResponse shouldInterceptRequest(WebView view, String url);
• API >= 21 public WebResourceResponse shouldInterceptRequest(WebView view, WebResourceRequest request);

4. 实践

4.1 API < 21

当API < 21时，shouldInterceptRequest方法的版本为：

public WebResourceResponse shouldInterceptRequest(WebView view, String url) 

此时仅能获取到请求URL，请求方法、头部信息以及body等均无法获取，强行拦截该请求可能无法能到正确响应。所以当API < 21时，不对请求进行拦截：

public WebResourceResponse shouldInterceptRequest(WebView view, String url) {
    return super.shouldInterceptRequest(view, url);
}

4.2 API >= 21

当API >= 21时，shouldInterceptRequest提供了新版：

public WebResourceResponse shouldInterceptRequest(WebView view, WebResourceRequest request)

其中WebResourceRequest结构为：

public interface WebResourceRequest {
    Uri getUrl(); // 请求URL

    boolean isForMainFrame(); // 是否由主MainFrame发出的请求

    boolean isRedirect();

    boolean hasGesture(); // 是否是由某种行为(如点击)触发

    String getMethod(); // 请求方法

    Map<String, String> getRequestHeaders(); // 头部信息
}

可以看到，在API >= 21时，在拦截请求时，可以获取到如下信息：

• 请求URL
• 请求方法：POST, GET…
• 请求头

4.2.1 仅拦截GET请求

由于WebResourceRequest并没有提供请求body信息，所以只能拦截GET请求，不能拦截POST:

public WebResourceResponse shouldInterceptRequest(WebView view, WebResourceRequest request) {
    String scheme = request.getUrl().getScheme().trim();
    String method = request.getMethod();
    Map<String, String> headerFields = request.getRequestHeaders();
    String url = request.getUrl().toString();
    Log.e(TAG, "url:" + url);

    // 无法拦截body，拦截方案只能正常处理不带body的请求；
    if ((scheme.equalsIgnoreCase("http") || scheme.equalsIgnoreCase("https"))
            && method.equalsIgnoreCase("get")) {
        // ...
    } else {
         return super.shouldInterceptRequest(view, request);
    }  
}

4.2.2 设置头部信息

public WebResourceResponse shouldInterceptRequest(WebView view, WebResourceRequest request) {
    // ...
    URL url = new URL(request.getUrl().toString());
    conn = (HttpURLConnection) url.openConnection();
    // 接口获取IP
    String ip = httpdns.getIpByHostAsync(url.getHost());

    if (ip != null) {
        // 通过HTTPDNS获取IP成功，进行URL替换和HOST头设置
        Log.d(TAG, "Get IP: " + ip + " for host: " + url.getHost() + " from HTTPDNS successfully!");
        String newUrl = path.replaceFirst(url.getHost(), ip);
        conn = (HttpURLConnection) new URL(newUrl).openConnection();

        if (headers != null) {
             for (Map.Entry<String, String> field : headers.entrySet()) {
                 conn.setRequestProperty(field.getKey(), field.getValue());
             }
        }

       // 设置HTTP请求头Host域
       conn.setRequestProperty("Host", url.getHost());
    } 
}

4.2.3 HTTPS请求证书校验

如果拦截到的请求是HTTPS请求，需要进行证书校验：

if (conn instanceof HttpsURLConnection) {
    final HttpsURLConnection httpsURLConnection = (HttpsURLConnection)conn;
   
    // https场景，证书校验
    httpsURLConnection.setHostnameVerifier(new HostnameVerifier() {
        @Override
        public boolean verify(String hostname, SSLSession session) {
            String host = httpsURLConnection.getRequestProperty("Host");
            if (null == host) {
                host = httpsURLConnection.getURL().getHost();
            }
            return HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session);
        }
    });
}

4.2.4 SNI场景

如果请求涉及到SNI场景，需要自定义SSLSocket，对SNI场景不熟悉的用户可以参考SNI:

WebviewTlsSniSocketFactory sslSocketFactory = new WebviewTlsSniSocketFactory((HttpsURLConnection) conn);

// sni场景，创建SSLScocket
httpsURLConnection.setSSLSocketFactory(sslSocketFactory);

class WebviewTlsSniSocketFactory extends SSLSocketFactory {
    private final String TAG = WebviewTlsSniSocketFactory.class.getSimpleName();
    HostnameVerifier hostnameVerifier = HttpsURLConnection.getDefaultHostnameVerifier();
    private HttpsURLConnection conn;

    public WebviewTlsSniSocketFactory(HttpsURLConnection conn) {
        this.conn = conn;
    }

    @Override
    public Socket createSocket() throws IOException {
        return null;
    }

    @Override
    public Socket createSocket(String host, int port) throws IOException, UnknownHostException {
        return null;
    }

    @Override
    public Socket createSocket(String host, int port, InetAddress localHost, int localPort) throws IOException, UnknownHostException {
        return null;
    }

    @Override
    public Socket createSocket(InetAddress host, int port) throws IOException {
        return null;
    }

    @Override
    public Socket createSocket(InetAddress address, int port, InetAddress localAddress, int localPort) throws IOException {
        return null;
    }

    // TLS layer

    @Override
    public String[] getDefaultCipherSuites() {
        return new String[0];
    }

    @Override
    public String[] getSupportedCipherSuites() {
        return new String[0];
    }

    @Override
    public Socket createSocket(Socket plainSocket, String host, int port, boolean autoClose) throws IOException {
        String peerHost = this.conn.getRequestProperty("Host");
        if (peerHost == null)
            peerHost = host;
        Log.i(TAG, "customized createSocket. host: " + peerHost);
        InetAddress address = plainSocket.getInetAddress();
        if (autoClose) {
            // we don't need the plainSocket
            plainSocket.close();
        }
        // create and connect SSL socket, but don't do hostname/certificate verification yet
        SSLCertificateSocketFactory sslSocketFactory = (SSLCertificateSocketFactory) SSLCertificateSocketFactory.getDefault(0);
        SSLSocket ssl = (SSLSocket) sslSocketFactory.createSocket(address, port);

        // enable TLSv1.1/1.2 if available
        ssl.setEnabledProtocols(ssl.getSupportedProtocols());

        // set up SNI before the handshake
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
            Log.i(TAG, "Setting SNI hostname");
            sslSocketFactory.setHostname(ssl, peerHost);
        } else {
            Log.d(TAG, "No documented SNI support on Android <4.2, trying with reflection");
            try {
                java.lang.reflect.Method setHostnameMethod = ssl.getClass().getMethod("setHostname", String.class);
                setHostnameMethod.invoke(ssl, peerHost);
            } catch (Exception e) {
                Log.w(TAG, "SNI not useable", e);
            }
        }

        // verify hostname and certificate
        SSLSession session = ssl.getSession();

        if (!hostnameVerifier.verify(peerHost, session))
            throw new SSLPeerUnverifiedException("Cannot verify hostname: " + peerHost);

        Log.i(TAG, "Established " + session.getProtocol() + " connection with " + session.getPeerHost() +
                " using " + session.getCipherSuite());

        return ssl;
    }
}

4.2.5 重定向

如果服务端返回重定向，此时需要判断原有请求中是否含有cookie：

• 如果原有请求报头含有cookie，因为cookie是以域名为粒度进行存储的，重定向后cookie会改变，且无法获取到新请求URL下的cookie，所以放弃拦截
• 如果不含cookie，重新发起二次请求

private boolean needRedirect(int code) {
    return code >= 300 && code < 400;
}
/**
 * header中是否含有cookie
 * @param headers
 */
private boolean containCookie(Map<String, String> headers) {
    for (Map.Entry<String, String> headerField : headers.entrySet()) {
        if (headerField.getKey().contains("Cookie")) {
            return true;
        }
    }
    return false;
}

int code = conn.getResponseCode();// Network block
if (needRedirect(code)) {
    // 原有报头中含有cookie，放弃拦截
    if (containCookie(headers)) {
        return null;
    }

    // 临时重定向和永久重定向location的大小写有区分
    String location = conn.getHeaderField("Location");
    if (location == null) {
        location = conn.getHeaderField("location");
    }

    if (location != null) {
        if (!(location.startsWith("http://") || location
                .startsWith("https://"))) {
            //某些时候会省略host，只返回后面的path，所以需要补全url
            URL originalUrl = new URL(path);
            location = originalUrl.getProtocol() + "://"
                    + originalUrl.getHost() + location;
        }
        Log.e(TAG, "code:" + code + "; location:" + location + "; path" + path);
        // 重新发起二次请求
        return recursiveRequest(location, headers, path);
    } else {
        // 无法获取location信息，让浏览器获取
        return null;
    }
} else {
    // redirect finish.
    Log.e(TAG, "redirect finish");
    return conn;
}

4.2.6 MIME&Encoding

如果拦截网络请求，需要返回一个WebResourceResponse：

public WebResourceResponse(String mimeType, String encoding, InputStream data)

创建WebResourceResponse对象需要提供：

• 请求的MIME类型
• 请求的编码
• 请求的输入流

其中请求输入流可以通过URLConnection.getInputStream()获取到，而MIME类型和encoding可以通过请求的ContentType获取到，即通过URLConnection.getContentType(),如：

text/html;charset=utf-8

但并不是所有的请求都能得到完整的contentType信息，此时可以参考如下策略

// 注*：对于POST请求的Body数据，WebResourceRequest接口中并没有提供，这里无法处理
String contentType = connection.getContentType();
String mime = getMime(contentType);
String charset = getCharset(contentType);
HttpURLConnection httpURLConnection = (HttpURLConnection)connection;
int statusCode = httpURLConnection.getResponseCode();
String response = httpURLConnection.getResponseMessage();
Map<String, List<String>> headers = httpURLConnection.getHeaderFields();
Set<String> headerKeySet = headers.keySet();
Log.e(TAG, "code:" + httpURLConnection.getResponseCode());
Log.e(TAG, "mime:" + mime + "; charset:" + charset);

// 无mime类型的请求不拦截
if (TextUtils.isEmpty(mime)) {
    Log.e(TAG, "no MIME");
    return super.shouldInterceptRequest(view, request);
} else {
    // 二进制资源无需编码信息
    if (!TextUtils.isEmpty(charset) || (isBinaryRes(mime))) {
        WebResourceResponse resourceResponse = new WebResourceResponse(mime, charset, httpURLConnection.getInputStream());
        resourceResponse.setStatusCodeAndReasonPhrase(statusCode, response);
        Map<String, String> responseHeader = new HashMap<String, String>();
        for (String key: headerKeySet) {
            // HttpUrlConnection可能包含key为null的报头，指向该http请求状态码
            responseHeader.put(key, httpURLConnection.getHeaderField(key));
        }
        resourceResponse.setResponseHeaders(responseHeader);
        return resourceResponse;
    } else {
        Log.e(TAG, "non binary resource for " + mime);
        return super.shouldInterceptRequest(view, request);
    }
}

/**
 * 从contentType中获取MIME类型
 * @param contentType
 * @return
 */
private String getMime(String contentType) {
    if (contentType == null) {
        return null;
    }
    return contentType.split(";")[0];
}

/**
 * 从contentType中获取编码信息
 * @param contentType
 * @return
 */
private String getCharset(String contentType) {
    if (contentType == null) {
        return null;
    }

    String[] fields = contentType.split(";");
    if (fields.length <= 1) {
        return null;
    }

    String charset = fields[1];
    if (!charset.contains("=")) {
        return null;
    }
    charset = charset.substring(charset.indexOf("=") + 1);
    return charset;
}

/**
 * 是否是二进制资源，二进制资源可以不需要编码信息
 * @param mime
 * @return
 */
private boolean isBinaryRes(String mime) {
    if (mime.startsWith("image")
            || mime.startsWith("audio")
            || mime.startsWith("video")) {
        return true;
    } else {
        return false;
    }
}

5 总结

综上所述，在WebView场景下的请求拦截逻辑如下所示：

5.1 【不可用场景】

• API Level < 21的设备
• POST请求
• 无法获取到MIME类型的请求
• 无法获取到编码的非二进制文件请求

5.2【可用场景】

前提条件：

• API Level >= 21
• GET请求
• 可以获取到MIME类型以及编码信息请求或是可以获取到MIME类型的二进制文件请求

可用场景：

• 普通HTTP请求
• HTTPS请求
• SNI请求
• HTTP报头中不含cookie的重定向请求

5.3 完整代码

HTTPDNS+WebView最佳实践完整代码请参考：GithubDemo

如果我的文章对您有帮助，不妨点个赞鼓励一下(^_^)