-
Notifications
You must be signed in to change notification settings - Fork 73
Description
Describe the bug
I am creating an adaptor to interface a CMIS service to store content in an S3 store (with folder data in a relational database).
The problem I encountered is when I am reading content from the S3, i.e. Resource.getInputStream.
In the AWS S3 Client implementation, there is a HttpClient connection pool (default is 50 connections).
When I start to download multiple content from the S3 store via the CMIS interface, it will cause an Exception: timeout in getting connection from pool.
Searching the web I found that it could be caused by the InputStream not closed by the CMIS interface.
Please refer to: aws/aws-sdk-java#1405
To Reproduce
Need to to link the CMIS interface to the S3 store.
Create a SpringBootApplication with the following and import the spring-content-cmis, spring-content-s3, spring-content-jpa packages.
@EnableCmis
@import(JpaLockingAndVersioningConfig.class)
@EnableJpaRepositories
@EnableS3Stores
Define a S3 client bean:
@bean
public S3Client client() throws URISyntaxException {
AwsCredentialsProvider provider = new AwsCredentialsProvider() {
@OverRide
public AwsCredentials resolveCredentials() {
return AwsBasicCredentials.create(s3accessKey, s3secretKey);
}
};
return S3Client.builder()
.endpointOverride(new URI(s3Url))
.region(Region.AP_SOUTH_1)
.credentialsProvider(provider)
.build();
}
Using the CMIS interface download some CMIS file multiple times (at least 50 which is the default max connections).
Expected behavior
No limit on the number of file or content downloaded.
Additional context
Please refer to: aws/aws-sdk-java#1405
If I change the getInputStream() method in the SimpleStorageResource.java, to load the entire content from input stream this.amazonS3.getObject(getObjectRequestBuilder.build()) into a Byte Array, then close the InputStream and finally, return an InputSteam based on the byte array, it will not encounter the timeout on connection pool exception.
What is the proper way to handle the contentStream for CMIS or S3 object inputStream which needs to be closed quickly, to avoid a connection pool issue ?