Use AWS-SDK to upload files more than 5 MB to the S3 bucket

Yaohua
2 min readDec 24, 2020

Why I can’t upload the video to my S3 bucket?

In one project, we implemented the uploading function within S3 to collect sound data from the clients. It was pretty easy and didn’t take too much time by simply setup the CORS in the bucket, generate a key for S3Access, and use the s3.upload function in AWS-SDK.

Amazing! It can even show the progress bar easily!

Until that day, we want the users to upload the video……..

403 Forbidden

The first time I got 403 forbidden when I upload the video, I was shocked. Since it’s been a long time since the last time I changed this project, I’m so worried that it actually can never work!!! (Of course not since we still have users keep recording their sound)

After checking the network, I found it use the POST method in the first request whenever the file is over 5MB.

All forbidden!
Add `POST` into the AllowedMethods list in our S3 bucket.

Ok great. Now the first request passed, but the put request got canceled.

The second request canceled!

Till now, I’m totally stuck. I tried to allow all the methods in CORS. I tried several methods online. I also tried every configuration in the SDK.

How would it possible that even the AWS-CLI worked, BUT I WERE STILL FORBIDDEN!

ETAG

Actually, during that time, I kept seeing the words ETAG. I even tried the ETAG configuration in code several times. However, it just liked a miracle. The answer pump up when I randomly google

s3 upload more than 5mb etag cors

And this is the first article from 2013

https://stackoverflow.com/questions/20815141/fineuploader-large-files-to-amazon-s3

Nailed it. We need to expose ETAG because it’s the identity for the cache server!

Conclusion

To those who scroll down to the bottom for the answer, you just need to configure

"ExposeHeaders": [ "Etag"],"AllowedMethods": ["PUT", "POST"]

to make the s3.upload function works.

--

--

Yaohua

Full-Stack Engineer | Telehealth industry | Dreamer | Builder | Coach