When I run the AllTests.xjs file under Unit Test for Applications \ Aqua Data Server \ AquaScripts \ AllTests.xjs, I get the following error on some of the tests when using ADStudio 3.05 build 16229.
JavaException: org.apache.commons.httpclient.ConnectionPoolTimeoutException: Timeout waiting for connection
However, when I run the same script in ADStudio 3.01, I do not get these connection pool timeout exceptions. Could this somehow be related to the WebClient changes made recently?
This problem seems to have something to do with the recent WebClient changes. In the WebClient changes made a few days ago, I also replaced the connection manager used by the Apache HttpClient, changed from SimpleHttpConnectionManager to MultiThreadedHttpConnectionManager. The reason is SimpleHttpConnectionManager only provides access to a single HttpConnection which is an abstraction of an HTTP InputStream and OutputStream pair,
I did some study but could not figure out why this exception is thrown. However, by looking into methods implemented by MultiThreadedHttpConnectionManager class and changed some of its default settings, the problem seemed to be taken care of.
setDefaultMaxConnectionsPerHost(30); // default is 2
setMaxTotalConnections(150); // default is 20
In addition, whenever a request is submitted to AQWebClient, it now will try to close connections that has idled 5+ minutes:
long idleTimeout = 5 * 60 * 1000; // 5 minutes in milliseconds
connectionManager.closeIdleConnections(idleTimeout);
I have run AllTests.xjs quite a few times with changes made above, exception is no longer be thrown.
Again, the root cause of this problem is still not clear at the moment. The unit test scripts implemented by the AQWebClient package have examples that access multiple servers with multiple requests goes to each server, the exception never be thrown there.
For the 3 parameters mentioned above, we might want to add APIs to AQWebClient so that application can set/get their values.
Sending src-rep\src\lib\aqua-core\com\aquafold\aquacore\open\net\webbrowser\CoreWebClient.java
Sending src-rep\src\lib\aqua-core\com\aquafold\aquacore\open\net\webbrowser\CoreWebHttpClient.java
Transmitting file data ..
Committed revision 16277.
Hi Fung, I'd like to discuss the MultiThreadedHttpConnectionManager. I am not sure we want this. Depending on how this works, it could limit the usage on ADServer.
Hi Fung, I'd like to discuss the MultiThreadedHttpConnectionManager. I am not sure we want this. Depending on how this works, it could limit the usage on ADServer.
Discussed with Niels; to make it a bit clear, here is the relationship amongst AQWebClient/ApacheHttpCleint/ApacheConnectionManager in turns of instance references:
AQWebClient <1-to-1-mapping> ApacheHttpClient <1-to-1-mapping> ApacheConnectionManager
This means if 3 AQWebClient objects are created in an AQUA script, each of these 3 objects will have its own ApacheHttpClient instances created, and each of these ApacheHttpClient instances would have their own ApacheConnectionManager instantiated as well, as shown below:
AQWebClient_1 --> ApacheHttpClient_1 --> ApacheConnectionManager_1
AQWebClient_2 --> ApacheHttpClient_2 --> ApacheConnectionManager_2
AQWebClient_3 --> ApacheHttpClient_3 --> ApacheConnectionManager_3
Apache HttpClient provides two concrete connection manager implementations: SimpleHttpConnectionManager and MultiThreadedHttpConnectionManager. SimpleHttpConnectionManager seems to only maintain one single connection object and access to that connection object is not thread-safe; MultiThreadedHttpConnectionManager tracks a set of connection objects and access to these connection objects seem to be thread-safe (as implied by its class name). When we have time, maybe we should spend more time on this issue and find out why exception is thrown.
Discussed with Niels; to make it a bit clear, here is the relationship amongst AQWebClient/ApacheHttpCleint/ApacheConnectionManager in turns of instance references:
AQWebClient <1-to-1-mapping> ApacheHttpClient <1-to-1-mapping> ApacheConnectionManager
This means if 3 AQWebClient objects are created in an AQUA script, each of these 3 objects will have its own ApacheHttpClient instances created, and each of these ApacheHttpClient instances would have their own ApacheConnectionManager instantiated as well, as shown below:
AQWebClient_1 --> ApacheHttpClient_1 --> ApacheConnectionManager_1
AQWebClient_2 --> ApacheHttpClient_2 --> ApacheConnectionManager_2
AQWebClient_3 --> ApacheHttpClient_3 --> ApacheConnectionManager_3
Apache HttpClient provides two concrete connection manager implementations: SimpleHttpConnectionManager and MultiThreadedHttpConnectionManager. SimpleHttpConnectionManager seems to only maintain one single connection object and access to that connection object is not thread-safe; MultiThreadedHttpConnectionManager tracks a set of connection objects and access to these connection objects seem to be thread-safe (as implied by its class name). When we have time, maybe we should spend more time on this issue and find out why exception is thrown.
Issue #4879 |
Closed |
Fixed |
Resolved |
Completion |
No due date |
No fixed build |
No time estimate |
This problem seems to have something to do with the recent WebClient changes. In the WebClient changes made a few days ago, I also replaced the connection manager used by the Apache HttpClient, changed from SimpleHttpConnectionManager to MultiThreadedHttpConnectionManager. The reason is SimpleHttpConnectionManager only provides access to a single HttpConnection which is an abstraction of an HTTP InputStream and OutputStream pair,
I did some study but could not figure out why this exception is thrown. However, by looking into methods implemented by MultiThreadedHttpConnectionManager class and changed some of its default settings, the problem seemed to be taken care of.
setDefaultMaxConnectionsPerHost(30); // default is 2
setMaxTotalConnections(150); // default is 20
In addition, whenever a request is submitted to AQWebClient, it now will try to close connections that has idled 5+ minutes:
long idleTimeout = 5 * 60 * 1000; // 5 minutes in milliseconds
connectionManager.closeIdleConnections(idleTimeout);
I have run AllTests.xjs quite a few times with changes made above, exception is no longer be thrown.
Again, the root cause of this problem is still not clear at the moment. The unit test scripts implemented by the AQWebClient package have examples that access multiple servers with multiple requests goes to each server, the exception never be thrown there.
For the 3 parameters mentioned above, we might want to add APIs to AQWebClient so that application can set/get their values.
Sending src-rep\src\lib\aqua-core\com\aquafold\aquacore\open\net\webbrowser\CoreWebClient.java
Sending src-rep\src\lib\aqua-core\com\aquafold\aquacore\open\net\webbrowser\CoreWebHttpClient.java
Transmitting file data ..
Committed revision 16277.