1. Imagine You’re a Dog 🐕

Hey, buddy! Imagine you’re a dog living in a house.

Every morning, your owner gives you food. You eat, wag your tail, and wait for more.

But one day, you become super excited and start barking:

“Food! Food! Food! Give me more food!”

Your owner has a problem. If you keep demanding food every second, the entire food supply might disappear!

So your owner creates a simple rule:

“You can have only 5 biscuits every 10 seconds.”

Now you understand the rule. You can eat your biscuits, but once you reach the limit, you must wait until more biscuits become available.

Congratulations! You have just understood the basic idea of a RateLimiter.

In software engineering, a RateLimiter controls how many requests a system can accept within a specified time period.

For example:

• Maximum 5 requests every 10 seconds.
• Maximum 100 requests every minute.
• Maximum 1,000 requests every hour.

Just like your owner controls your biscuits, a RateLimiter controls the traffic entering a system.

2. Why Do We Need a RateLimiter?

Now imagine your house has three dogs:

• Dog 1: Order Service
• Dog 2: Product Service
• Dog 3: Payment Service

Your owner asks Dog 1 to bring a product from Dog 2.

In microservices, this is similar to one service calling another service through a network.

Let’s understand the normal flow.

Client
   |
   v
Order Service
   |
   v
Product Service
   |
   v
Product Details

Everything works perfectly when the Product Service responds quickly.

But what happens if Dog 2 falls asleep?

Imagine the Product Service takes 60 seconds to respond.

The Order Service must wait for the response. Its thread remains occupied while waiting.

Now imagine hundreds of clients are placing orders simultaneously.

Many Order Service threads may become blocked while waiting for the slow Product Service.

Eventually, the Order Service might run out of available threads and start rejecting requests.

The original problem was inside the Product Service, but now the Order Service is affected too.

This is called a cascading failure.

A RateLimiter can help control how many requests are sent to a service. It helps prevent excessive traffic from overwhelming the system, although it does not, by itself, fix a slow or unhealthy downstream service.

3. What Exactly Is a RateLimiter?

Let’s return to our dog example.

Your owner says:

“You can eat only 5 biscuits every 10 seconds.”

In a microservice, the equivalent rule could be:

“Allow only 5 requests during a 10-second period.”

If the limit has not been reached, the request can be accepted.

If the limit has been reached, the system can reject the request or make it wait, depending on the implementation and configuration.

A RateLimiter is useful for:

• Protecting APIs from excessive traffic.
• Controlling sudden traffic spikes.
• Managing access to expensive operations.
• Helping prevent services from being overloaded.
• Enforcing API usage limits.

Important: Rate limiting is one part of fault tolerance. It works alongside other patterns such as Bulkhead, TimeLimiter, CircuitBreaker, and Retry.

4. Types of Rate-Limiting Algorithms

Now let’s imagine your owner wants to control your biscuits in different ways.

There are several possible rules.

4.1 Fixed Window Counter

Your owner creates a clock and says:

“You can eat 5 biscuits every 10 seconds.”

The counter starts again at the beginning of every 10-second window.

For example:

Time window	Biscuits allowed
0–10 seconds	5
10–20 seconds	5
20–30 seconds	5

Suppose you eat 5 biscuits between seconds 0 and 9. You cannot eat another biscuit during that window.

When the next window begins, your counter resets.

The problem: What if you eat 5 biscuits at the very end of one window and another 5 immediately at the beginning of the next?

You could consume 10 biscuits in a very short period, even though the configured limit is 5 per 10-second window.

This is the boundary problem of the Fixed Window Counter algorithm.

4.2 Sliding Log

Now your owner becomes smarter.

Instead of resetting the counter at fixed intervals, your owner remembers the exact time you eat every biscuit.

For each new biscuit request, the owner checks how many biscuits you ate during the previous 10 seconds.

If you already ate 5, the next biscuit is rejected.

If fewer than 5 were eaten during that period, you can have another.

For example:

Last 10 seconds:

Biscuit 1  -> Accepted
Biscuit 2  -> Accepted
Biscuit 3  -> Accepted
Biscuit 4  -> Accepted
Biscuit 5  -> Accepted
Biscuit 6  -> Rejected

As old timestamps fall outside the 10-second window, they stop counting.

The advantage: It handles the window-boundary problem more accurately.

The disadvantage: It must maintain timestamps for accepted requests and clean up expired records. This can require more memory and processing.

4.3 Sliding Window Counter with Sub-Windows

Your owner now divides the 10-second period into smaller sections.

For example:

Main window = 10 seconds

Sub-window 1 = 0–2 seconds
Sub-window 2 = 2–4 seconds
Sub-window 3 = 4–6 seconds
Sub-window 4 = 6–8 seconds
Sub-window 5 = 8–10 seconds

Each sub-window stores the number of biscuits consumed during that period.

Suppose the counts are:

Sub-window 1 = 1
Sub-window 2 = 1
Sub-window 3 = 1
Sub-window 4 = 1
Sub-window 5 = 0

Total = 4
Limit = 5

You have eaten only 4 biscuits during the active window, so another biscuit can be accepted.

As time moves forward, the main window slides and expired sub-windows stop contributing to the count.

The advantage: It uses counters instead of storing every individual request timestamp.

The disadvantage: It requires more bookkeeping than a basic Fixed Window Counter and uses memory to maintain sub-window counts.

4.4 Sliding Window Counter with Weighted Window

Now imagine your owner wants a simpler approximation without tracking every small sub-window.

The owner considers two fixed windows:

• The previous window.
• The current window.

The previous window’s requests are given a weight based on how much of that window overlaps the current sliding period.

For example:

Previous window requests = 5
Weight = 90%

Current window requests = 0

Estimated total = 0 + (5 × 0.90)
                = 4.5

If the implementation rounds this to 5 and the configured limit is 5, the next request will be rejected.

A little later, the previous window might contribute only 80% of its count:

Previous window requests = 5
Weight = 80%

Current window requests = 0

Estimated total = 0 + (5 × 0.80)
                = 4

The request may now be accepted because the estimated count is below the limit.

The advantage: It uses less detailed tracking than a Sliding Log.

The disadvantage: It assumes that the distribution of requests within the previous window can be approximated by a weighting calculation. The result is an estimate, not an exact count.

4.5 Token Bucket

This is one of the easiest algorithms to understand.

Imagine your owner has a bucket that can hold a maximum of 4 biscuits.

Every time you request a biscuit, one biscuit is removed from the bucket.

When the bucket is empty, your request cannot be accepted until a new biscuit becomes available.

Your owner refills the bucket at a fixed rate.

For example:

Bucket capacity = 4 tokens
Refill rate = 1 token every 6 seconds

Request 1 -> Accepted
Request 2 -> Accepted
Request 3 -> Accepted
Request 4 -> Accepted
Request 5 -> Rejected if no token is available

After the refill interval, a new token becomes available.

If the bucket is already full, additional tokens are discarded.

The advantage: Token Bucket allows a limited burst of requests while controlling the longer-term request rate.

The disadvantage: If the bucket capacity is too high, a large burst can still be accepted after a period of inactivity.

Resilience4j’s RateLimiter uses a permission-based model with configurable refresh periods and permissions. The exact behavior depends on the configured settings.

4.6 Leaky Bucket

Now imagine a different rule.

Your owner accepts biscuit requests and places them in a queue.

The owner gives you biscuits at a fixed rate, regardless of how quickly you ask.

For example:

Incoming requests
       |
       v
  +----------+
  |  Queue   |
  | RQ1      |
  | RQ2      |
  | RQ3      |
  +----------+
       |
       v
Processed at a controlled rate

If the queue is full, additional requests may be rejected.

The advantage: It smooths out bursts by processing queued requests at a controlled rate.

The disadvantage: Requests may experience additional latency while waiting in the queue. The queue size must also be chosen carefully.

Which Algorithm Should You Remember?

Algorithm	Simple explanation	Main drawback
Fixed Window	Reset the counter every period	Bursts at window boundaries
Sliding Log	Remember each request’s timestamp	Higher memory and cleanup cost
Sliding Window Counter	Count requests in smaller time windows	Additional bookkeeping
Weighted Window	Estimate traffic using weighted counts	Approximation may be inaccurate
Token Bucket	Requests consume available tokens	Allows bursts up to bucket capacity
Leaky Bucket	Queue requests and process them at a controlled rate	Queueing adds latency

5. Implement RateLimiter in Spring Boot Using Resilience4j

Now let’s stop talking about biscuits and write some Java code.

Imagine we have two microservices:

• Order Service: Receives requests from clients.
• Product Service: Returns product details.

The Order Service calls the Product Service through a Feign client.

We want to allow only 2 calls per 10-second refresh period.

Step 1: Add the dependency

For a Spring Boot 3 application, add Resilience4j to your Maven pom.xml.

<dependency>
    <groupId>io.github.resilience4j</groupId>
    <artifactId>resilience4j-spring-boot3</artifactId>
    <version>2.1.0</version>
</dependency>

Use a compatible, maintained Resilience4j version for your project.

Step 2: Create the Product Feign Client

This interface allows the Order Service to call the Product Service.

@FeignClient(name = "product-service")
public interface ProductClient {

    @GetMapping("/products/{id}")
    String getProductById(
        @PathVariable("id") String id
    );
}

Think of the Feign client as your dog sending a message to another dog:

“Hey, Product Service! Please give me the details for product 101.”

Step 3: Create the Order Service

Now we apply the @RateLimiter annotation to the method that calls the Product Service.

import io.github.resilience4j.ratelimiter.annotation.RateLimiter;
import org.springframework.stereotype.Service;

@Service
public class OrderService {

    private final ProductClient productClient;

    public OrderService(ProductClient productClient) {
        this.productClient = productClient;
    }

    @RateLimiter(
        name = "productRateLimiter",
        fallbackMethod = "rateLimitedFallback"
    )
    public String invokeProductAPI(String id) {

        return productClient.getProductById(id);
    }

    public String rateLimitedFallback(
            String id,
            Throwable t) {

        return "Rate limit exceeded. Try again later.";
    }
}

Let’s understand the important parts.

@RateLimiter

This annotation applies rate-limiting behavior to the method.

name = "productRateLimiter"

This is the name of the RateLimiter instance whose settings we will define in the configuration.

fallbackMethod = "rateLimitedFallback"

When the call fails in a way handled by the configured fallback mechanism, this method provides an alternative response.

The fallback method must have a compatible return type and the original method’s parameters, optionally followed by a compatible exception parameter.

The annotated method should be public and managed by Spring so the framework’s proxy-based interception can work.

Step 4: Configure RateLimiter

Open application.properties and add:

server.port=8081
spring.application.name=order-service

resilience4j.ratelimiter.instances.productRateLimiter.limitForPeriod=2
resilience4j.ratelimiter.instances.productRateLimiter.limitRefreshPeriod=10s
resilience4j.ratelimiter.instances.productRateLimiter.timeoutDuration=1s

Let’s understand these properties.

Property	Meaning
limitForPeriod=2	Allow 2 permissions per refresh period
limitRefreshPeriod=10s	Refresh the permissions every 10 seconds
timeoutDuration=1s	Wait up to 1 second to acquire a permission before rejecting the attempt

The timeout controls how long the caller waits for a permission. It does not mean that the Product Service API itself has a one-second timeout.

Step 5: Create the Controller

@RestController
@RequestMapping("/orders")
public class OrderController {

    private final OrderService orderService;

    public OrderController(OrderService orderService) {
        this.orderService = orderService;
    }

    @GetMapping("/{id}")
    public String getOrder(@PathVariable String id) {
        return orderService.invokeProductAPI(id);
    }
}

Now send requests to:

GET http://localhost:8081/orders/101

The controller invokes the Order Service method, which attempts to call the Product Service through the configured RateLimiter.

Step 6: Understand the Output

Suppose the Product Service is available and each request succeeds.

Request 1 -> Accepted
Request 2 -> Accepted
Request 3 -> Rate limit exceeded
Request 4 -> Rate limit exceeded

After the permission refresh period, permissions become available again.

After the refresh period:

Request 5 -> Accepted
Request 6 -> Accepted
Request 7 -> Rate limit exceeded

Remember that the refresh period is not a rolling 10-second Sliding Log window. It is the configured permission refresh period used by Resilience4j.

Also, the fallback above returns a message, but a real REST API should generally return an appropriate HTTP status, such as 429 Too Many Requests, when the request has exceeded its allowed rate.

6. Can We Use RateLimiter With RestTemplate or RestClient?

Yes!

The concept remains the same if the application uses RestTemplate or RestClient instead of Feign.

You can apply the RateLimiter to the Spring-managed service method that makes the downstream API call.

The important point is that the annotated method must be invoked through the appropriate Spring proxy for annotation-based interception to work.

7. Can We Create Our Own RateLimiter?

Yes. We can create custom rate-limiting logic using Spring AOP.

Imagine you want to write an annotation that looks like this:

@CustomRateLimiter(
    limit = 5,
    windowInSeconds = 60
)
public String getProducts() {
    return "List of products";
}

First, define the custom annotation:

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface CustomRateLimiter {

    int limit();

    int windowInSeconds();
}

Next, create an aspect:

@Aspect
@Component
public class RateLimiterAspect {

    @Around("@annotation(customRateLimiter)")
    public Object rateLimit(
            ProceedingJoinPoint pjp,
            CustomRateLimiter customRateLimiter)
            throws Throwable {

        // 1. Check the current request count.
        // 2. Reject the request if the limit is reached.
        // 3. Record an accepted request.
        // 4. Execute the original method.

        return pjp.proceed();
    }
}

The code above demonstrates the structure, not a complete RateLimiter. The actual counting, concurrency control, window expiration, and rejection logic must be implemented.

For a single-instance application, a thread-safe in-memory implementation may be sufficient. For a distributed application, multiple service instances need a shared or coordinated limiting strategy if the limit is intended to apply globally.

8. Where Does RateLimiter Fit in Fault-Tolerant Microservices?

RateLimiter is one of several patterns used to protect microservices.

A useful conceptual learning sequence is:

1. RateLimiter: Control how many requests are admitted.
2. Bulkhead: Isolate resources so one overloaded operation does not consume everything.
3. TimeLimiter: Bound how long an operation is allowed to wait.
4. CircuitBreaker: Stop repeated calls to a dependency that is failing.
5. Retry: Attempt a failed operation again when doing so is appropriate.

These patterns solve different problems and can work together. Their actual ordering depends on the application, the type of failure, and the resilience framework configuration.

For example, a Retry policy can multiply the number of downstream calls. Rate limiting should be designed with those additional attempts in mind so that retries do not overwhelm an already struggling service.

9. Real-World Example: An Online Shopping Application

Imagine an online shopping website during a major sale.

Thousands of customers are trying to view products at the same time.

Without appropriate traffic controls, the Product Service may become overloaded.

A RateLimiter can restrict the number of requests admitted to a particular operation.

For example:

Customers
    |
    v
API Gateway
    |
    v
Order Service
    |
    v
RateLimiter
    |
    v
Product Service

The exact placement depends on what you want to protect.

An API Gateway rate limit can protect the public API from excessive client traffic. A service-level RateLimiter can control calls to a particular operation or dependency.

For a production system, you must also decide whether limits apply per user, per API key, per service instance, or globally across the application.

10. Key Takeaways

Let’s finish our dog lesson.

Remember these five things:

• A RateLimiter controls how many requests can be admitted within a configured time period.
• It can help protect services from excessive traffic and sudden bursts.
• Fixed Window, Sliding Log, Sliding Window Counter, Weighted Window, Token Bucket, and Leaky Bucket use different approaches to traffic control.
• Spring Boot applications can use Resilience4j’s @RateLimiter annotation and configuration properties.
• RateLimiter is not a replacement for timeouts, circuit breakers, bulkheads, or proper monitoring.

The easiest way to remember it:

A dog should not be allowed to eat unlimited biscuits at once. Similarly, a microservice should not necessarily accept unlimited requests at once.

A sensible limit helps the system remain more stable under heavy traffic.

And now, my friend, you’re ready for the next lesson!

In the next article, we’ll explore Bulkhead, TimeLimiter, CircuitBreaker, and Retry in Spring Boot Microservices using the same simple dog-friendly approach.

Until then, keep coding and keep learning! 🐶💻