Then you create an empty object for each short URL (a random 5-character ID) and attach the “website redirect” metadata with the target (long) URL. It’s a little known feature: S3 can act as a redirection engine.įirst, you need to activate “website hosting” on your S3 bucket. This next graphic shows a generated short URL ready to copy and paste. The following graphic shows the admin page. Because you don’t need to manage your own servers anymore, it’s easy to build applications that are natively scalable, secure, resilient, and very cost effective. In this post, you’ll see how to build your own private URL shortener using a serverless architecture, which leaves you with full control over your application while leveraging managed AWS services like AWS Lambda, AWS API Gateway, and S3. This is when I realized it was super easy to build a private one. As this content is often confidential, I tend to avoid using public URL shortener services. It’s more convenient and user-friendly to generate a shortened URL. However, S3 pre-signed URLs are generally over 250 characters and may be split in several lines by email clients. To avoid sending heavy attachments in email, I first upload the content on Amazon S3, then send a pre-signed URL to allow the customer to securely download the content. TinyURL is a URL-shortening web service that creates shorter aliases for long URLs.As a solutions architect, I often need to send large files to customers, such as architecture diagrams and service presentations. When clicking the short link, users are redirected to the original URL. This service is useful because short links save space and allow users to type long URLs more easily. When designing a TinyURL-like application, there are some functional and non-functional requirements to consider. When given a URL, our service will generate a shorter alias of the original URL.Shortened links should not be predictable in any manner.URL redirection should happen in real-time with minimal latency.If the service fails, all the short links will not be functional. Short links will expire after a default timespan, but users are able to specify the expiration time.Users should have the option to pick a custom short link for their URL.The short link should redirect users to the original link.This new link will be extensively shortened so that it can be easily copied and pasted. There will be a large number of redirection requests compared to new URL shortenings. What would be Queries Per Second (QPS) for our system? New URLs shortenings per second:ĥ00 million / (30 days * 24 hours * 3600 seconds) = ~200 URLs/sĬonsidering the 100:1 read/write ratio, URLs redirections per second will be:ġ00 * 200 URLs/s = 20K/s Storage estimates Traffic estimatesĪssuming, we will have 500 million new URL shortenings per month with a 100:1 read/write ratio, we can expect 50 billion redirections during the same period: Let’s assume a 100:1 ratio between read and write. Since we expect to have 500 million new URLs every month, the total number of objects we expect to store will be 30 billion:ĥ00 million * 5 years * 12 months = 30 billion Let’s assume we store every URL shortening request (and associated shortened link) for five years. We will need 15TB of total storage:ģ0 billion * 500 bytes = 15 TB Bandwidth estimatesįor write requests, since we expect 200 new URLs every second, total incoming data for our service will be 100KB per second:įor read requests, since we expect ~20K URLs redirections every second, the total outgoing data for our service would be 10MB per second:Ģ0K * 500 bytes = ~10 MB/s Memory estimates Let’s assume that each stored object will be approximately 500 bytes (just a ballpark estimate - we will dig into it later). To cache 20% of these requests, we will need 170GB of memory: Since we have 20K requests per second, we will be getting 1.7 billion requests per day:Ģ0K * 3600 seconds * 24 hours = ~1.7 billion If we want to cache some of the hot URLs that are frequently accessed, how much memory will we need to store them? If we follow the 80-20 rule, meaning 20% of the URLs generate 80% of the traffic, we would like to cache these hot URLs. One thing to note here is that since there will be a lot of duplicate requests (of the same URL), our actual memory usage will be less than 170GB. api_dev_key (string): The API developer key of a registered account.Below is an example of the definitions of the APIs for creating and deleting URLs without service: createURL(api_dev_key, original_url, custom_alias=None, user_name=None, expire_date=None) We can use REST APIs to expose the functionality of our service. Original_url (string): Original URL to be shortened.This will be used to throttle users based on their allocated quota.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |