Write in front

In< [high concurrency] the interviewer asked me how to use Nginx to limit the current, and I got the Offer easily! >In this article, we mainly introduced how to use Nginx for current limiting to avoid the system being overwhelmed by large flow. In addition, Nginx has many powerful functions, such as load balancing, caching, black-and-white lists, grayscale publishing, etc. Today, let's discuss these powerful functions supported by Nginx!

Nginx installation

Note: take CentOS 6.8 server as an example. Install Nginx as root.

1. installation dependent environment

yum -y install wget gcc-c++ ncurses ncurses-devel cmake make perl bison openssl openssl-devel gcc* libxml2 libxml2-devel curl-devel libjpeg* libpng* freetype* autoconf automake zlib* fiex* libxml* libmcrypt* libtool-ltdl-devel* libaio libaio-devel  bzr libtool

2. install openssl

wget https://www.openssl.org/source/openssl-1.0.2s.tar.gz
tar -zxvf openssl-1.0.2s.tar.gz
cd /usr/local/src/openssl-1.0.2s
./config --prefix=/usr/local/openssl-1.0.2s
make
make install

3. install pcre

wget https://ftp.pcre.org/pub/pcre/pcre-8.43.tar.gz
tar -zxvf pcre-8.43.tar.gz
cd /usr/local/src/pcre-8.43
./configure --prefix=/usr/local/pcre-8.43
make
make install

4. install zlib

wget https://sourceforge.net/projects/libpng/files/zlib/1.2.11/zlib-1.2.11.tar.gz
tar -zxvf zlib-1.2.11.tar.gz
cd /usr/local/src/zlib-1.2.11
./configure --prefix=/usr/local/zlib-1.2.11
make
make

5. install Nginx

wget http://nginx.org/download/nginx-1.17.2.tar.gz
tar -zxvf nginx-1.17.2.tar.gz
cd /usr/local/src/nginx-1.17.2
./configure --prefix=/usr/local/nginx-1.17.2 --with-openssl=/usr/local/src/openssl-1.0.2s --with-pcre=/usr/local/src/pcre-8.43 --with-zlib=/usr/local/src/zlib-1.2.11 --with-http_ssl_module
make
make install

It should be noted that when installing Nginx, the source code decompression directories of openssl, pcre and zlib are specified. The full path of the Nginx configuration file after installation is: /usr/local/Nginx-1.17.2/conf/Nginx Conf.

Nginx load balancing configuration

1. load balancing configuration

http {
       ......
    upstream real_server {
       server 192.168.103.100:2001 weight=1;  #Polling servers and access weights
       server 192.168.103.100:2002 weight=2;
    }
 
    server {
        listen  80;
 
        location / {
            proxy_pass http://real_server;
        }
    }
}

2. failed retry configuration

upstream real_server {
   server 192.168.103.100:2001 weight=1 max_fails=2 fail_timeout=60s;
   server 192.168.103.100:2002 weight=2 max_fails=2 fail_timeout=60s;
}

It means fail_timeout failed Max_ After failures requests, the upstream server is considered unavailable, and the service address is kicked out. Fail_ After the timeout time, the server will be added to the survival list again for retry.

Nginx current limiting configuration

1. configuration parameters

limit_req_zone instruction setting parameters

limit_req_zone $binary_remote_addr zone=mylimit:10m rate=10r/s;

• Limit_ Req_ The zone is defined in the http block, $binary_remote_addr means to save the binary form of the client IP address.
• Zone defines the shared memory area for IP status and URL access frequency. zone=keyword identifies the name of the zone and the size of the zone followed by a colon. The status information of 16000 IP addresses is about 1MB, so the region in the example can store 160000 IP addresses.
• Rate defines the maximum request rate. The rate in the example cannot exceed 10 requests per second.

2. set current limit

location / {
        limit_req zone=mylimit burst=20 nodelay;
        proxy_pass http://real_server;
}

burst queue size. nodelay does not limit the time between individual requests.

3. unlimited white list

geo $limit {
default              1;
192.168.2.0/24  0;
}
 
map $limit $limit_key {
1 $binary_remote_addr;
0 "";
}
 
limit_req_zone $limit_key zone=mylimit:10m rate=1r/s;
 
location / {
        limit_req zone=mylimit burst=1 nodelay;
        proxy_pass http://real_server;
}

In the above configuration, the IP access of the 192.168.2.0/24 network segment is not limited to current, and other current is limited.

The meaning of the numbers after IP:

• 24 indicates subnet mask: 255.255.255.0
• 16 indicates subnet mask: 255.255.0.0
• 8 indicates subnet mask: 255.0.0.0

Nginx cache configuration

1. browser cache

expire for static resource cache

location ~*  .(jpg|jpeg|png|gif|ico|css|js)$ {
   expires 2d;
}

Expires and cache control are added to the Response Header,

Static resources include (general cache)

• Common and unchanging images, such as logo, icon, etc
• js, css static files
• Downloadable content, media files

Negotiation cache (add_header ETag/Last-Modified value)

• HTML file
• Frequently replaced pictures
• Frequently modified js and css files
• Basically unchanged API interface

No cache required

• User privacy and other sensitive data
• Frequently changing api data interfaces

2. proxy layer cache

//Cache path. inactive indicates the cache time. After expiration, the cache will be cleaned up
proxy_cache_path /data/cache/nginx/ levels=1:2 keys_zone=cache:512m inactive = 1d max_size=8g;
 
location / {
    location ~ \.(htm|html)?$ {
        proxy_cache cache;
        proxy_cache_key    $uri$is_args$args;     //HASH with this variable value as KEY
        //The X-Cache field can be seen in the header of the HTTP response. The contents can include hit, miss, express, etc
        add_header X-Cache $upstream_cache_status;
        proxy_cache_valid 200 10m;
        proxy_cache_valid any 1m;
        proxy_pass  http://real_server;
        proxy_redirect     off;
    }
    location ~ .*\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$ {
        root /data/webapps/edc;
        expires      3d;
        add_header Static Nginx-Proxy;
    }
}

Create a file directory on the local disk. According to the settings, the requested resources are cached in this directory in the form of K-V. the KEY needs to be defined by itself (the hash value of the url is used here). At the same time, you can specify the cache duration of a certain content as needed, such as 10 minutes for the cache with the status code of 200, 5 minutes for the cache with the status code of 301302, and 1 minute for all other content.
You can clean up the cache through the function of purger.

The browser cache should be disabled during AB test / personalization requirements.

Nginx blacklist

1. general configuration

location / {
    deny  192.168.1.1;
    deny 192.168.1.0/24;
    allow 10.1.1.0/16;
    allow 2001:0db8::/32;
    deny  all;
}

2. Lua+Redis dynamic blacklist (OpenResty)

Installation and operation

yum install yum-utils
yum-config-manager --add-repo https://openresty.org/package/centos/openresty.repo
yum install openresty
yum install openresty-resty
 check
yum --disablerepo="*" --enablerepo="openresty" list available
 working
service openresty start

Configuration (/usr/local/openresty/nginx/conf/nginx.conf)

lua_shared_dict ip_blacklist 1m;
 
server {
    listen  80;
 
    location / {
        access_by_lua_file lua/ip_blacklist.lua;
        proxy_pass http://real_server;
    }
}

Lua script (ip_blacklist.lua)

local redis_host    = "192.168.1.132"
local redis_port    = 6379
local redis_pwd     = 123456
local redis_db = 2
 
-- connection timeout for redis in ms.
local redis_connection_timeout = 100
 
-- a set key for blacklist entries
local redis_key     = "ip_blacklist"
 
-- cache lookups for this many seconds
local cache_ttl     = 60
 
-- end configuration
 
local ip                = ngx.var.remote_addr
local ip_blacklist      = ngx.shared.ip_blacklist
local last_update_time  = ip_blacklist:get("last_update_time");
 
-- update ip_blacklist from Redis every cache_ttl seconds:
if last_update_time == nil or last_update_time < ( ngx.now() - cache_ttl ) then
 
  local redis = require "resty.redis";
  local red = redis:new();
 
  red:set_timeout(redis_connect_timeout);
 
  local ok, err = red:connect(redis_host, redis_port);
  if not ok then
    ngx.log(ngx.ERR, "Redis connection error while connect: " .. err);
  else
    local ok, err = red:auth(redis_pwd)
    if not ok then
      ngx.log(ngx.ERR, "Redis password error while auth: " .. err);
    else
        local new_ip_blacklist, err = red:smembers(redis_key);
        if err then
            ngx.log(ngx.ERR, "Redis read error while retrieving ip_blacklist: " .. err);
        else
        ngx.log(ngx.ERR, "Get data success:" .. new_ip_blacklist)
          -- replace the locally stored ip_blacklist with the updated values:
            ip_blacklist:flush_all();
          for index, banned_ip in ipairs(new_ip_blacklist) do
            ip_blacklist:set(banned_ip, true);
          end
          -- update time
          ip_blacklist:set("last_update_time", ngx.now());
      end
    end
  end
end
 
if ip_blacklist:get(ip) then
  ngx.log(ngx.ERR, "Banned IP detected and refused access: " .. ip);
  return ngx.exit(ngx.HTTP_FORBIDDEN);
end

Nginx grayscale Publishing

1. realize grayscale publishing according to cookies

Query the version value according to the Cookie. If the version value is v1, forward to host1, and v2, forward to host2. If they do not match, forward to the default configuration.

upstream host1 {
   server 192.168.2.46:2001 weight=1;  #Polling servers and access weights
   server 192.168.2.46:2002 weight=2;
}
 
upstream host2 {
   server 192.168.1.155:1111  max_fails=1 fail_timeout=60;
}
 
upstream default {
   server 192.168.1.153:1111  max_fails=1 fail_timeout=60;
}
 
map $COOKIE_version $group {
   ~*v1$ host1;
   ~*v2$ host2;
   default default;
}
 
lua_shared_dict ip_blacklist 1m;
 
server {
    listen  80;
 
    #set $group "default";
    #if ($http_cookie ~* "version=v1"){
    #    set $group host1;
    #}
    #if ($http_cookie ~* "version=v2"){
    #    set $group host2;
    #}
 
    location / {
        access_by_lua_file lua/ip_blacklist.lua;
        proxy_pass http://$group;
    }
}

2. realize gray-scale publishing according to the incoming IP

server {
  ...............
  set $group default;
  if ($remote_addr ~ "192.168.119.1") {
      set $group host1;
  }
  if ($remote_addr ~ "192.168.119.2") {
      set $group host2;
  }

3. more fine grained grayscale Publishing

Reference: https://github.com/sunshinelyz/ABTestingGateway

OK, let's stop here today! Don't forget to read and forward it to others, so that more people can see it, learn and progress together!!

Write at the end

If you think binghe's written well, please search on wechat and pay attention to the "binghe technology" wechat official account to learn from binghe about high concurrency, distribution, micro services, big data, Internet and cloud native technologies. The "binghe technology" wechat official account has updated a large number of technical topics, and each technical article is full of dry goods! Many readers have successfully switched to big factories by reading the wechat official account article of "ice river technology" and hanging interviewers; Many readers have also achieved a technological leap and become the technical backbone of the company! If you want to improve your ability, achieve a leap in technical ability, enter a large factory, get promoted and get a raise, then pay attention to the wechat official account of "ice river technology" and update the dry goods of super hard core technology every day, so that you will no longer be confused about how to improve your technical ability!