ApiClient throttles requests in async mode

[andrewshuttlewood-masabi]

OkHttp by default only permits 5 connections to a given host in async mode. This means that in a high traffic scenario, we are capped to just 5 concurrent requests unless we manage a pool of ApiClients or create one per request, or ignore the async methods.

The number can be changed by configuring a dispatcher on the ApiClient builder to have a greater number of maxRequests and maxRequestsPerHost. I think there are a number of possible fixes:

• The max requests is set suitably high automatically (ideally unbounded for us)
• The max requests/max requests per host is configurable inside the merchant config
• The client can pass in an optional Function to configure the http client builder

We did a quick test with a local fork to set this higher and this helped it work for us, the change was just:

+            Dispatcher dispatcher = new Dispatcher();
+            dispatcher.setMaxRequests(500);
+            dispatcher.setMaxRequestsPerHost(500);
           try {
               httpClient = classHttpClient.newBuilder()
+                     .dispatcher(dispatcher)

It looks like there is no significant negative impact to just setting this to Integer.MAX_VALUE - so that might be a reasonable option.