Test to skip the Content-Length in ApacheHttpRequestFactory should be case insensitive

[outsideMyBox]

There is a test in ApacheHttpRequestFactory to skip the Content-Length and avoid the HttpClient4 to complain if it's already present.
The problem is that the test is case sensitive whereas headers are not according to the rfc2616 standard.
For example the header 'content-length' is not skipped

Describe the bug

The test should be case insensitive.

Expected Behavior

Skip any header in the list independently of their case.

Current Behavior

A request with 'content-length' leads to an exception:

[...]
Caused by: org.apache.http.ProtocolException: Content-Length header already present
[...]

Your Environment

• AWS Java SDK version used: 1.11.873
• JDK version used: 11
• Operating System and version: Ubuntu

[debora-ito]

@outsideMyBox can you tell us more about your use case? How is the content-length header being added?

[outsideMyBox]

Hi @debora-ito ,
it happens when copying an object with its metadata from a source to another destination.
Interestingly it works on AWS S3 (as far as I remember, the Content-Length was not already present in the source's metadata), but it doesn't on our client's premises, who uses an alternative S3 provider and other settings (e.g. load balancers). A first investigation showed that a load balancer returns a lowercase header 'content-length'.

[debora-ito]

Understood. We don't guarantee that SDK operations will work with third party S3 providers, we need to guarantee that it is working with S3 directly, which it is. But I'll bring this up to the team, and will change to a feature request.

[debora-ito]

Quick update: we've added this task to the our backlog.

[debora-ito]

@outsideMyBox have you tried using Java SDK 2.x? It changed the way it handles request headers and you can plug in different http clients.

Although this was added to the backlog, the team focus is in releasing features for 2.x and this would have a very low priority.

[outsideMyBox]

@debora-ito Thanks for the hint. Unfortunately, switching to the SDK 2.0 would involve too much refactoring on our side relative to the time we can spend at the moment.

[mosche]

@outsideMyBox run into a similar issue when testing.
A workaround was to configure the client to fix that header using a RequestHandler2

AmazonS3ClientBuilder builder = AmazonS3ClientBuilder.standard();
builder.withRequestHandlers(new RequestHandler2() {
      public void beforeRequest(Request<?> request) {
        Map<String, String> headers = request.getHeaders();
        if (!StringUtils.isAllLowerCase(HttpHeaders.CONTENT_LENGTH) && headers.containsKey(HttpHeaders.CONTENT_LENGTH)) {
          headers.remove(HttpHeaders.CONTENT_LENGTH.toLowerCase());
        }
        super.beforeRequest(request);
      }
    });

[wyljpn]

Hi, @mosche
I encountered the same issue when running the below code.

public static class MinioS3ClientBuilderFactory extends DefaultS3ClientBuilderFactory {
  @Override
  public AmazonS3ClientBuilder createBuilder(S3Options s3Options) {
    AmazonS3ClientBuilder builder = super.createBuilder(s3Options);
    builder.withPathStyleAccessEnabled(true);
    return builder;
  }
}

// START MinIO configurations        
options.as(AwsOptions.class).setAwsServiceEndpoint("http://minio-client.test.svc.cluster.local:9000");
AWSCredentialsProvider credentialsProvider = new AWSStaticCredentialsProvider(new BasicAWSCredentials("minio", "miniopassword"));
options.as(AwsOptions.class).setAwsCredentialsProvider(credentialsProvider);
Class<? extends DefaultS3ClientBuilderFactory> builderFactory = MinioS3ClientBuilderFactory.class;
options.as(S3Options.class).setS3ClientFactoryClass(builderFactory);
// END MinIO configurations

Pipeline p = Pipeline.create(options);
p.apply(TextIO.read().from("s3://jin/alarm_monitor_4.csv"))
 .apply(ParDo.of(new filterData()))
 .apply(ParDo.of(new responseApiFn()))
 .apply(TextIO.write().to("s3://jin/apiIdResponse").withSuffix(".csv").withoutSharding());

Error:

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.0.0:java (default-cli) on project api: An exception occured while executing the Java class. java.io.IOException: com.amazonaws.SdkClientException: Unable to execute HTTP request: null: ClientProtocolException: Content-Length header already present -> [Help 1]

Seems It could upload the temporary files (something like s3://jin/.temp-beam), but couldn't move it to s3://jin/apiIdResponse.csv

And, after adding builder.withRequestHandlers, it got another error complaining that it want a Content-Length in http header.

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.0.0:java (default-cli) on project api: An exception occured while executing the Java class. java.util.concurrent.ExecutionException: java.io.IOException: Failed closing channel to s3://jin/.temp-beam-1c3bc976-129b-47d2-a9d7-9be7fe835cd6/6dce7af262ab539b-c7b3-4720-8e28-582cf801cf72: com.amazonaws.services.s3.model.AmazonS3Exception: You must provide the Content-Length HTTP header. (Service: Amazon S3; Status Code: 411; Error Code: MissingContentLength; Request ID: 1716BF6FFB6436AE; S3 Extended Request ID: cebfa887-476d-4883-b6c5-2724abc0b4a9; Proxy: null), S3 Extended Request ID: cebfa887-476d-4883-b6c5-2724abc0b4a9 -> [Help 1]

Related libraries:

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk</artifactId>
    <version>1.12.305</version>
</dependency>

<dependency>
    <groupId>org.apache.commons</groupId>
      <artifactId>commons-lang3</artifactId>
      <version>3.12.0</version>
  </dependency>

<dependency>
  <groupId>org.apache.httpcomponents</groupId>
  <artifactId>httpclient</artifactId>
  <version>4.5.13</version>
</dependency>

<dependency>
    <groupId>org.apache.beam</groupId>
    <artifactId>beam-sdks-java-io-amazon-web-services</artifactId>
    <version>2.40.0</version>
</dependency>

Do you have anything idea about how to fix it?

[wyljpn]

I added a condition and it works! Remove the "content-length" only when it equals "0"

if (!StringUtils.isAllLowerCase(HttpHeaders.CONTENT_LENGTH) && headers.containsKey(HttpHeaders.CONTENT_LENGTH) && headers.get("content-length").equals("0")) {
                    headers.remove(HttpHeaders.CONTENT_LENGTH.toLowerCase());
                }

[debora-ito]

@outsideMyBox We don't have plans to support this in v1 before going into Maintenance Mode, so I'll go ahead and close this issue.

The workaround is to use RequestHandlers to change the header, like the examples in this thread show.

If this issue is still relevant in v2 please open a new issue in the v2 repo.

Reference:

• Announcing end-of-support for AWS SDK for Java v1.x effective December 31, 2025 - blog post

[github-actions]

This issue is now closed.

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.