Httpclient 使用和性能测试

上篇，通过简介和架构图，我们对HttpClient有了初步的了解。
本篇我们展示HttpClient的简单使用，同时为了说明httpclient的使用性能，我们将Httpclient的同步和异步模式与apache的Httpclient4作比较。。

1. HttpClient示例代码

以下基本是官方示例，分别展示了如何使用Get和Post请求。

HttpClient client = HttpClient.newBuilder()
    .version(Version.HTTP_1_1)	//可以手动指定客户端的版本，如果不指定，那么默认是Http2
    .followRedirects(Redirect.NORMAL)	//设置重定向策略
    .connectTimeout(Duration.ofSeconds(20))	//连接超时时间
    .proxy(ProxySelector.of(new InetSocketAddress("proxy.example.com", 80)))	//代理地址设置
    .authenticator(Authenticator.getDefault()) 
    //.executor(Executors.newFixedThreadPoolExecutor(8))  //可手动配置线程池
    .build();   

HttpRequest request = HttpRequest.newBuilder()       
    .uri(URI.create("https://foo.com/"))    //设置url地址
    .GET()
    .build();
HttpResponse<String> response = client.send(request, BodyHandlers.ofString());  	//同步发送
System.out.println(response.statusCode()); 	//打印响应状态码
System.out.println(response.body());  


HttpRequest request = HttpRequest.newBuilder()
       .uri(URI.create("https://foo.com/")) 
       .timeout(Duration.ofMinutes(2))	//设置连接超时时间
       .header("Content-Type", "application/json")	
       .POST(BodyPublishers.ofFile(Paths.get("file.json")))    //设置请求体来源
       .build();   
client.sendAsync(request, BodyHandlers.ofString()) 		//异步发送
    .thenApply(HttpResponse::body) 	//发送结束打印响应体
    .thenAccept(System.out::println);  

可以看到，应用编写的代码相对流畅自然。不过，也有几个注意点

• Http连接池不支持手动配置，默认是无限复用的
• 重试次数不支持手动配置
• 不指定Http客户端或请求的版本，会默认使用Http2模式进行连接，受挫后会进行降级
• 请求的同步发送模式（send）实际上会后台另开线程

短短的几行代码只是实现了功能，那么，它的性能如何呢？我们把它和业界标杆——Apache 的HttpClient作对比。

2. 服务器测试代码编写

为了简便，使用node.js的http模块运行一个简易的服务器。该服务器驻守在8080端口，每收到一个请求，停留500ms后返回响应。

let http = require("http")
let server = http.createServer()
server.addListener("request", (req, res) => {
    if (req.url.startsWith("/")) {
        //接到任意请求，停留0.5秒后返回
        setTimeout(() => {
            res.end("haha")
        }, 500)
    }
}
)
server.listen(8080, () => console.log("启动成功！"))

使用node运行该js文件，提示已启动成功

3. JDK httpclient 和apache Httpclient 测试代码

首先定义公共的测试接口：

public interface Tester {

    //测试参数
    class TestCommand {

    }

    /**
     * 测试主方法
     * @param testCommand 测试参数
     */
    void test(TestCommand testCommand) throws Exception;

    /**
     * 重复测试多次
     * @param testName 测试名称
     * @param times 测试次数
     * @param testCommand 每次测试的参数
     */
    default void testMultipleTimes(String testName, int times, TestCommand testCommand) throws Exception{
        long startTime = System.currentTimeMillis();
        System.out.printf(" ----- %s开始，共%s次 -----\n", testName, times);
        for (int i = 0; i < times; i++) {
            long currentStartTime = System.currentTimeMillis();
            test(testCommand);
            System.out.printf("第%s次测试用时：%sms\n", i + 1, (System.currentTimeMillis() - currentStartTime));
        }
        long usedTime = System.currentTimeMillis() - startTime;
        System.out.printf("%s次测试共用时:%sms,平均用时：%sms\n", times, usedTime, usedTime / times);
    }
}

定义测试类，包含三个静态嵌套类，分别用作JDK httpclient的异步模式、同步模式和apache Httpclient的同步模式

public class HttpClientTester {

    /** Http请求的真正测试参数*/
    static class HttpTestCommand extends Tester.TestCommand {

        /**目的url*/
        String url;
        /**单次测试请求次数*/
        int requestTimes;
        /**请求线程数*/
        int threadCount;

        public HttpTestCommand(String url, int requestTimes, int threadCount) {
            this.url = url;
            this.requestTimes = requestTimes;
            this.threadCount = threadCount;
        }
    }


    static class BlocklyHttpClientTester implements Tester {

        @Override
        public void test(TestCommand testCommand) throws Exception {
            HttpTestCommand httpTestCommand = (HttpTestCommand) testCommand;
            testBlockly(httpTestCommand.url, httpTestCommand.requestTimes,httpTestCommand.threadCount);
        }

        /**
         * 使用JDK Httpclient的同步模式进行测试
         * @param url 请求的url
         * @param times 请求次数
         * @param threadCount 开启的线程数量
         * @throws ExecutionException
         * @throws InterruptedException
         */
        void testBlockly(String url, int times, int threadCount) throws ExecutionException, InterruptedException {
            threadCount = threadCount <= 0 ? 1 : threadCount;
            ExecutorService executorService = Executors.newFixedThreadPool(threadCount);
            HttpClient client = HttpClient.newBuilder().build();
            Callable<String> callable1 = () -> {
                HttpRequest request = HttpRequest.newBuilder(URI.create(url)).GET().build();
                HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
                return response.body();
            };
            List<Future<String>> futureList1 = new ArrayList<>();
            for (int i = 0; i < times; i++) {
                Future<String> future1 = executorService.submit(callable1);
                futureList1.add(future1);
            }
            for (Future<String> stringFuture : futureList1) {
                //阻塞直至所有请求返回
                String s = stringFuture.get();
            }
            executorService.shutdown();
        }
    }


    static class NonBlocklyHttpClientTester implements Tester {


        @Override
        public void test(TestCommand testCommand) throws Exception {
            HttpTestCommand httpTestCommand = (HttpTestCommand) testCommand;
            testNonBlockly(httpTestCommand.url, httpTestCommand.requestTimes);
        }

        /**
         * 使用JDK Httpclient的异步模式进行测试
         * @param url 请求的url
         * @param times 请求次数
         * @throws InterruptedException
         */
        void testNonBlockly(String url, int times) throws InterruptedException {
            //给定16个线程，业务常用 2 * Runtime.getRuntime().availableProcessors()
            ExecutorService executor = Executors.newFixedThreadPool(16);
            HttpClient client = HttpClient.newBuilder()
                    .executor(executor)
                    .build();
            //使用倒计时锁来保证所有请求完成
            CountDownLatch countDownLatch = new CountDownLatch(times);
            HttpRequest request = HttpRequest.newBuilder(URI.create(url)).GET().build();
            while (times-- >= 0) {
                client.sendAsync(request, HttpResponse.BodyHandlers.ofString())
                        .whenComplete((stringHttpResponse, throwable) -> {
                            if (throwable != null) {
                                throwable.printStackTrace();
                            }
                            if (stringHttpResponse != null) {
                                stringHttpResponse.body();
                            }
                            countDownLatch.countDown();
                        });
            }

            //阻塞直至所有请求完成
            countDownLatch.await();
            executor.shutdown();
        }
    }

    static class ApacheHttpClientTester implements Tester {

        @Override
        public void test(TestCommand testCommand) throws Exception {
            HttpTestCommand httpTestCommand = (HttpTestCommand) testCommand;
            testBlocklyWithApacheClient(httpTestCommand.url, httpTestCommand.requestTimes,httpTestCommand.threadCount);
        }
        /**
         * 使用Apache HttpClient进行测试
         * @param url 请求的url
         * @param times 使用时长
         * @param threadCount 开启的线程数量
         * @throws ExecutionException
         * @throws InterruptedException
         */
        void testBlocklyWithApacheClient(String url, int times, int threadCount) throws ExecutionException, InterruptedException {

            threadCount = threadCount <= 0 ? 1 : threadCount;
            ExecutorService executorService = Executors.newFixedThreadPool(threadCount);
            PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
            //设置apache Httpclient连接复用无限制，体现其最大性能
            connectionManager.setDefaultMaxPerRoute(Integer.MAX_VALUE);
            connectionManager.setMaxTotal(Integer.MAX_VALUE);
            CloseableHttpClient httpClient = HttpClientBuilder.create().setConnectionManager(connectionManager).build();
            Callable<String> callable1 = () -> {
                HttpGet httpGet = new HttpGet(url);
                CloseableHttpResponse response = httpClient.execute(httpGet);
                return EntityUtils.toString(response.getEntity());
            };
            List<Future<String>> futureList1 = new ArrayList<>();
            for (int i = 0; i < times; i++) {
                Future<String> future1 = executorService.submit(callable1);
                futureList1.add(future1);
            }
            for (Future<String> stringFuture : futureList1) {
                //阻塞直至所有请求返回
                String s = stringFuture.get();
            }
            executorService.shutdown();
        }
    }

测试的main方法：

    public static void main(String[] args) {
        try {
            //
            HttpTestCommand testCommand = new HttpTestCommand("http://localhost:8080", 200, 16);
            //每个测试重复3轮，减少误差
            final int testTimes = 3;
            new BlocklyHttpClientTester().testMultipleTimes("JDK HttpClient同步模式测试", testTimes, testCommand);
            new NonBlocklyHttpClientTester().testMultipleTimes("JDK HttpClient异步模式测试", testTimes, testCommand);
            new ApacheHttpClientTester().testMultipleTimes("Apache Httpclient同步模式测试", testTimes, testCommand);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

4. 测试结果

----- JDK HttpClient同步模式测试开始，共3次 -----
第1次测试用时：4414ms
第2次测试用时：3580ms
第3次测试用时：3620ms
3次测试共用时:11620ms,平均用时：3873ms
----- JDK HttpClient异步模式测试开始，共3次 -----
第1次测试用时：568ms
第2次测试用时：595ms
第3次测试用时：579ms
3次测试共用时:1742ms,平均用时：580ms
----- Apache Httpclient同步模式测试开始，共3次 -----
第1次测试用时：3719ms
第2次测试用时：3557ms
第3次测试用时：3574ms
3次测试共用时:10851ms,平均用时：3617ms

可见，Httpclient同步模式与apacheHttpclient同步模式性能接近；异步模式由于充分利用了nio非阻塞的特性，在线程数相同的情况下，效率大幅优于同步模式。

需要注意的是，此处的“同步”“异步”并非I/O模型中的同步，而是指编程方式上的同步/异步。

5. 总结

通过以上示例代码，可以看出HttpClient具有编写流畅、性能优良的特点，也有可定制性不足的遗憾。

下一节，我们将深入客户端的构建和启动过程，接触选择器管理者这一角色，探寻它和Socket通道的交互的交互过程。