Rate limiting

In computer networks, rate limiting is used to control the rate of requests sent or received by a network interface controller. It can be used to prevent DoS attacks and limit web scraping.

Research indicates flooding rates for one zombie machine are in excess of 20 HTTP GET requests per second, legitimate rates much less.

Rate limiting should be used along with throttling pattern to minimize the number of throttling errors.

Hardware appliances can limit the rate of requests on layer 4 or 5 of the OSI model.

Rate limiting can be induced by the network protocol stack of the sender due to a received ECN-marked packet and also by the network scheduler of any router along the way.

While a hardware appliance can limit the rate for a given range of IP-addresses on layer 4, it risks blocking a network with many users which are masked by NAT with a single IP address of an ISP.

Deep packet inspection can be used to filter on the session layer but will effectively disarm encryption protocols like TLS and SSL between the appliance and the protocol server (i.e. web server).

Protocol servers using a request / response model, such as FTP servers or typically Web servers may use a central in-memory key-value database, like Redis or Aerospike, for session management. A rate limiting algorithm is used to check if the user session (or IP address) has to be limited based on the information in the session cache.