S3 Pres Signed URLs

In this article, we explore a powerful AWS feature—pre-signed URLs. We begin by outlining the problem they solve and then delve into practical use cases.

Overview

Imagine you have an AWS account with a private S3 bucket that only authenticated IAM users can access. Although IAM users can retrieve or upload objects based on their permissions, sharing specific files with external users poses a challenge. Creating a new AWS account for every external user isn’t scalable, and making the bucket public would expose sensitive data. Pre-signed URLs address this dilemma.

By generating a pre-signed URL, an authenticated IAM user embeds their access credentials within the URL. This URL can be shared with a public user, permitting specific actions—such as downloading or uploading objects on the private bucket. When a request is made using the pre-signed URL, AWS S3 validates the embedded credentials and authorizes the action without exposing direct access to your account.

Use Case 1: Video Streaming

Consider a video hosting platform similar to Netflix or any membership-based service. In this scenario, your web server manages thousands of gigabytes of video files, yet you store the files in S3 for scalability and cost efficiency.

When a paying customer requests to watch a video, your server generates a pre-signed URL on behalf of that customer. The URL contains the credentials of a specific IAM user (for instance, user X) and is returned to the customer. AWS S3 then processes the request as if it came from user X, allowing secure access to the private video content.

Use Case 2: Direct File Uploads

Pre-signed URLs simplify not only file retrieval but also file uploads. For example, when updating a profile picture on a website, you traditionally upload the image to an API running on an EC2 instance, which then transfers the file to an S3 bucket.

With pre-signed URLs, your API can generate a URL that allows the file to be uploaded directly to S3. This method bypasses the API server, reducing server load and potentially accelerating the upload process.

Important Considerations

When generating a pre-signed URL, always specify an expiration time. These URLs are only valid for a limited duration, with a maximum expiration time of seven days when using IAM credentials. This limitation helps ensure temporary and controlled access.

It is crucial to understand that a pre-signed URL does not grant additional permissions beyond those already associated with the generating IAM user. If the IAM user lacks access to a specific object in the S3 bucket, any pre-signed URL they create will also be unable to access that object.

To illustrate, consider a case where a user without proper S3 bucket access generates a pre-signed URL. Any request made using that URL will be denied since the embedded credentials do not authorize access.

Summary

Pre-signed URLs in AWS offer a secure, time-limited way to grant specific permissions (such as downloading or uploading objects) to private S3 buckets. The AWS API uses the embedded credentials from the pre-signed URL to process requests. Consequently, if the generating IAM user lacks permission for a target object, the pre-signed URL will not succeed in granting access.