Rate limiting API requests

You have released a public API and you want to avoid abuse ? It is possible to limit access to a specific number of requests within periods of time. For example, if a user sends two successive requests when you only allow one request every 5 seconds, he will get the following answer :

Rate limit exceeded, please try again in 4 seconds.

How does it work ?

The user’s IP address can be used to uniquely identify the user. When it sends a request, the IP is stored with an expiration time stamp. The next time he will send a request, you check whether it is allowed to receive a response by comparing the stored time stamp with the current date and time.

Technical solutions

You could use a database like MySQL or SQLite to save the data but this is not recommended. :

• if your web application traffic is high, it will consume a lot of system resources and increase access times.
• the recorded IP addresses should be purged to avoid unnecessarily inflating the database.

Redis is therefore recommended for this use case. This software is a key/value database running in memory. If you do not have the required system to install a Redis server, there is an alternative : Redix. This one does not work in memory because it is based on BadgerDB. However, it remains very fast and easy to use.

Code sample

Here is the PHP source code to limit access to your API by allowing a request every 5 seconds for any user and using Redix.

Installing the Redis/Redix client :

composer require predis/predis

Download Redix depending on your operating system, then start the service :

./redix_linux_amd64

The following answer indicates that Redix is listening on ports 6380 for RESP protocol and 7090 for HTTP protocol.

In your API, add the following code to the header :

<?php
require_once 'class.ratelimit.redix.php';

$rl = new RateLimit();
$waitfor = $rl->getSleepTime($_SERVER['REMOTE_ADDR']);
if ($waitfor>0) {
    echo 'Rate limit exceeded, please try again in '.$waitfor.'s';
    exit;    
    }

// Your API response
echo 'API response';

The source code for the script class.ratelimit.redix.php is :

<?php
require_once __DIR__.'/vendor/autoload.php';
Predis\Autoloader::register();

class RateLimit {

    private $redis;
    const RATE_LIMIT_SECS = 5; // allow 1 request every x seconds

    public function __construct() {
        $this->redis = new Predis\Client([
            'scheme' => 'tcp',
            'host'   => 'localhost', // or the server IP on which Redix is running
            'port'   => 6380
            ]);
        }

    /**
     * Returns the number of seconds to wait until the next time the IP is allowed
     * @param ip {String}
     */
    public function getSleepTime($ip) {
        $value = $this->redis->get($ip);
        if(empty($value)) {
            // if the key doesn't exists, we insert it with the current datetime, and an expiration in seconds
            $this->redis->set($ip, time(), self::RATE_LIMIT_SECS*1000);
            return 0;
            } 
        return self::RATE_LIMIT_SECS - (time() - intval(strval($value)));
        } // getSleepTime

    } // class RateLimit

Conclusion

This solution is relatively easy to implement. The TTL command can also be used in the getSleepTime function. It returns the time remaining to live of a data before it expires. But it is not currently implemented in Redix.

To allow more requests, for example 180 in 15 minutes, Redis offers a solution for limiting APIs using the parameters INCR and EXPIRE.

Ressources

• Redis commands