Error While Copying files through different AWS buckets and region

[LamaBimal]

I am trying to copy the files from one bucket to another with different AWS Accounts and regions. Please find the AWS and region details as follows:

Source
AWS Account ID: 9858675xxxxx
Bucket: <source_bucket>
Region: us-east-1

Destination:
AWS Account ID: 2791777xxxxx
Bucket: <dest_bucket>
Region: us-west-1

While performing the copy operation with AWS S3 Client, I have getting the following error as
The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'us-west-1' Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: Q5YDECPR6BPSS3A6; S3 Extended Request ID: qpKP6yqa7L8RDpghM3O57HigraSpoBU8JrksSnF9k8D5liEkPzflkKQbwLXewqCvPystHsk0pGQ=

It is working fine while performing the copy operation through AWS Cli. It's only issue for the AWS SDK.

AWS Java SDK S3 Version: 1.11.525

Could you please suggest on this?

[debora-ito]

Can you share a sample code I can use to reproduce?

[LamaBimal]

Hello @debora-ito ,

Please find the sample code as follows:

package com.test.awsS3.Utils;

import com.amazonaws.AmazonClientException;
import com.amazonaws.ClientConfiguration;
import com.amazonaws.auth.DefaultAWSCredentialsProviderChain;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.S3ResponseMetadata;
import com.amazonaws.services.s3.model.AmazonS3Exception;
import com.amazonaws.services.s3.model.CopyObjectRequest;

public class S3FileCopy {
public AmazonS3 s3Client;
public void AwsConnectionManager() {

    System.out.println("Starting AWS connection .. ");
    String S3_REGION = "us-east-1";

    DefaultAWSCredentialsProviderChain credentials = new DefaultAWSCredentialsProviderChain();

    ClientConfiguration clientConfig = new ClientConfiguration()
            .withMaxConnections(100)
            .withConnectionTimeout(120 * 1000)
            .withMaxErrorRetry(5);
    clientConfig.setSignerOverride("AWSS3V4SignerType");

    AmazonS3ClientBuilder amazonS3ClientBuilder = AmazonS3ClientBuilder.standard()
            .withCredentials(credentials);
    amazonS3ClientBuilder.setClientConfiguration(clientConfig);
    amazonS3ClientBuilder.setRegion(S3_REGION);
    amazonS3ClientBuilder.setForceGlobalBucketAccessEnabled(true);

    s3Client = amazonS3ClientBuilder.build();

    System.out.println("AWS Connection successful...");

}
public void copyFile(String srcBucket, String destBucket, String srcKey, String destKey){
    try{


        CopyObjectRequest copyObjRequest = new CopyObjectRequest(
                srcBucket, srcKey, destBucket, destKey);
        System.out.println("*** Copy Object Request ****");
        System.out.println(copyObjRequest);
        System.out.println(copyObjRequest.getCustomRequestHeaders());
        System.out.println("***** end ********");
        s3Client.copyObject(copyObjRequest);
        S3ResponseMetadata md = s3Client.getCachedResponseMetadata(copyObjRequest);
        System.out.println("File Copying From " + srcBucket + "/" + srcKey + "To " + destBucket + "/" + destKey);
        System.out.println("Extended Request ID : " + md.getHostId() + " Request ID : " + md.getRequestId());
        System.out.println("Finished copying ------------------ ");


    }catch(AmazonS3Exception s3Exception){
        s3Exception.printStackTrace();
    }catch(AmazonClientException clientException){
        clientException.printStackTrace();
    }catch(Exception ex){
        ex.printStackTrace();
    }

}
public static void main(String[] args){
    String srcBucket = "dev-incoming-******"; //us-east-1 in AWS Account A
    String srcKey = "Test/20230714/testforfileMovement.txt";
    String destBucket = "om-*****"; // us-west-1 in AWS Account B
    String destKey = "Test/20230808/testforfileMovement.txt";
    S3FileCopy s3FileCopy = new S3FileCopy();
    s3FileCopy.AwsConnectionManager();
    s3FileCopy.copyFile(srcBucket,destBucket,srcKey, destKey);
}

}

Scenario: I am configuring the default region as us-east-1. And I am trying to copy the files from us-east-1 of AWS Account A to us-west-1 of AWS Account B.

Following is the error occurred:

com.amazonaws.services.s3.model.AmazonS3Exception: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'us-west-1' (Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: 2NM03YBET2YBZD6V; S3 Extended Request ID: EYk+g0Y8y+1eEMCvcmgUGXMlaNEt5VLw9z2CTX7a4kO1chUFxFEZ/YDGOO+mf5B8VbykJ2QLvlI=), S3 Extended Request ID: EYk+g0Y8y+1eEMCvcmgUGXMlaNEt5VLw9z2CTX7a4kO1chUFxFEZ/YDGOO+mf5B8VbykJ2QLvlI=
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1712)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1367)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4914)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4860)
at com.amazonaws.services.s3.AmazonS3Client.copyObject(AmazonS3Client.java:1996)
at com.test.awsS3.Utils.S3FileCopy.copyFile(S3FileCopy.java:51)
at com.test.awsS3.Utils.S3FileCopy.main(S3FileCopy.java:74)

Thanks,

[LamaBimal]

Hi @debora-ito ,

I have identified that the issue was raised by the following line:

clientConfig.setSignerOverride("AWSS3V4SignerType");

What's the usage of this signer? And Is it required case or not while configuring the client?

Thanks,

[debora-ito]

Normally you don't have to set the signer, the SDK will work fine with the default signer.
Glad you are not blocked by this anymore.