在处理超过5GB大小的文件时,可以使用AWS SDK提供的MultipartUpload功能来解决。MultipartUpload可以将大文件分割成多个小部分进行并发上传,并在服务器端进行合并。
下面是一个使用Java SDK的代码示例:
import com.amazonaws.AmazonServiceException;
import com.amazonaws.SdkClientException;
import com.amazonaws.auth.DefaultAWSCredentialsProviderChain;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.InitiateMultipartUploadRequest;
import com.amazonaws.services.s3.model.InitiateMultipartUploadResult;
import com.amazonaws.services.s3.model.PartETag;
import com.amazonaws.services.s3.model.UploadPartRequest;
import com.amazonaws.services.s3.model.UploadPartResult;
import com.amazonaws.services.s3.model.CompleteMultipartUploadRequest;
import com.amazonaws.services.s3.model.CompleteMultipartUploadResult;
import com.amazonaws.services.s3.model.AbortMultipartUploadRequest;
import java.io.File;
import java.util.ArrayList;
import java.util.List;
public class UploadLargeFile {
public static void main(String[] args) {
String bucketName = "your-bucket-name";
String keyName = "your-file-key";
String filePath = "path-to-your-file";
int partSize = 5 * 1024 * 1024; // 5MB
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
.withCredentials(DefaultAWSCredentialsProviderChain.getInstance())
.build();
try {
// Step 1: Initialize multipart upload
InitiateMultipartUploadRequest initRequest = new InitiateMultipartUploadRequest(bucketName, keyName);
InitiateMultipartUploadResult initResponse = s3Client.initiateMultipartUpload(initRequest);
String uploadId = initResponse.getUploadId();
// Step 2: Upload parts
long fileSize = new File(filePath).length();
long filePosition = 0;
int partNumber = 1;
List partETags = new ArrayList<>();
while (filePosition < fileSize) {
// Calculate part size
long partSizeRemaining = fileSize - filePosition;
int currentPartSize = (int) Math.min(partSize, partSizeRemaining);
// Upload part
UploadPartRequest uploadRequest = new UploadPartRequest()
.withBucketName(bucketName)
.withKey(keyName)
.withUploadId(uploadId)
.withPartNumber(partNumber)
.withFileOffset(filePosition)
.withFile(filePath)
.withPartSize(currentPartSize);
UploadPartResult uploadResult = s3Client.uploadPart(uploadRequest);
partETags.add(uploadResult.getPartETag());
filePosition += currentPartSize;
partNumber++;
}
// Step 3: Complete multipart upload
CompleteMultipartUploadRequest completeRequest = new CompleteMultipartUploadRequest(bucketName, keyName, uploadId, partETags);
CompleteMultipartUploadResult completeResult = s3Client.completeMultipartUpload(completeRequest);
System.out.println("Upload complete. ETag: " + completeResult.getETag());
} catch (AmazonServiceException e) {
// Handle Amazon service errors
e.printStackTrace();
} catch (SdkClientException e) {
// Handle client errors
e.printStackTrace();
}
// Step 4: Abort multipart upload if it fails or is not completed
s3Client.abortMultipartUpload(new AbortMultipartUploadRequest(bucketName, keyName, uploadId));
}
}
在上面的示例中,bucketName
是S3存储桶的名称,keyName
是要上传的文件在存储桶中的键,filePath
是要上传的文件路径,partSize
是每个上传的部分的大小。可以根据需要调整这些参数。
代码中的注释提供了每个步骤的解释。首先,我们初始化上传,然后根据文件大小将文件分割为多个部分进行上传。最后,我们完成上传并获取ETag。
如果上传失败或没有完成,还提供了一个捕获异常的部分,以便可以中止上传。
请确保您已正确配置AWS SDK的凭据,以便能够访问和上传到S3存储桶中。