Building a URL Shortener with Rate Limiting and Redis in Go

URL shorteners like Bitly or TinyURL are incredibly popular tools, but have you ever wondered what goes into building one? In this blog, we’ll dive into how I built a custom URL shortener in Go with rate limiting and a Redis database for data storage and IP tracking. We’ll cover the core features, the tech stack, and the challenges I encountered along the way.

Project Overview

This URL shortener application accepts long URLs from users, generates shorter, unique links, and allows users to customize the shortened alias if desired. The server redirects any visitor who uses the shortened URL to the original long URL.

Here's a quick overview of the primary components:

• 
  WebServer: Handles routing and requests using the Fiber framework.


• 
  Rate Limiter: Manages user requests to prevent abuse, limiting the number of requests an IP can make in a given timeframe.


• 
  URL Validator: Ensures URLs are in a valid format and ready for storage.


• 
  URL Generator: Generates unique short links for each long URL or uses custom aliases provided by users.


• 
  Redis Database: Stores URL mappings and IP rate limits.

With these features in mind, let’s break down the implementation.

docker-compose up -d

DB_ADDR="localhost:6379"

DB_PASSWORD=""

APP_PORT=":6767"

DOMAIN="localhost:6767"

APP_QUOTA=10

go run main.go

Now, the application is live, and you can start shortening URLs!

Testing the URL Shortener

Shortening a URL

Send a POST request to /api/v1 with the following JSON payload:

{

    "url": "https://example.com",

    "short": "exmpl",

    "expiry": 24

}

Accessing a Shortened URL

Use the generated short URL, like http://localhost:6767/exmpl, to be redirected to https://example.com.

Key Learnings

1. 
   Using Redis for Rate Limiting: Redis is incredibly fast, and using it for both URL storage and rate limiting was efficient and effective.


2. 
   Building a REST API with Fiber: Fiber's simplicity and performance are well-suited for building APIs in Go.


3. 
   Error Handling and Validation: Ensuring robust error handling and URL validation was essential to providing a user-friendly experience.

Future Improvements

There are a few features and optimizations I’d like to add in the future:

• 
  Admin Dashboard: A UI to track URL usage and monitor statistics.


• 
  Detailed Analytics: Track click counts, referrers, and user demographics.


• 
  Scalability: Deploy the app on a cloud provider and use distributed Redis for handling more extensive data.

Conclusion

Building this URL shortener was a rewarding experience and a great way to explore Go, Fiber, and Redis. This project provides a solid foundation, whether you're learning about backend development or exploring Go’s potential in web services.

If you'd like to see the code in action, check out the GitHub repository here. Let me know your thoughts or if you have suggestions for improving the project!