Skip to content

Instantly share code, notes, and snippets.

@HJianBo

HJianBo/emqx_nginx_proxy-英文翻译-0331(1).md

Created April 1, 2019 14:04
Show Gist options
  • Save HJianBo/1612d252b0da5957831828a9d5fc8197 to your computer and use it in GitHub Desktop.
Save HJianBo/1612d252b0da5957831828a9d5fc8197 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
emqx_nginx_proxy-英文翻译-0331(1).md

EMQ X 消息服务器 Nginx 反向代理

前言

很多使用 EMQ X 消息服务器的童鞋，会发现随着业务量的增长会考虑到 EMQ X 节点的扩容，多节点之后，如果节点前没有挂载代理，就需要在客户端指定连接哪台 EMQ X 节点，如果其中一台节点升级，还需要在客户端重新做设置，但如果在集群节点挂载反向代理，就可以通过反向代理将连接分配到另外节点上，从而避免在客户端的更改和重新部署。So，今天 EMQ君就聊聊 Nginx 反向代理的事吧!

Nginx安装

依赖安装

  • 安装pcre、zlib、openssl 
    $ wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.42.tar.gz
$ tar -zxf pcre-8.42.tar.gz
$ cd pcre-8.42
$ ./configure
$ make
$ sudo make install

$ wget http://zlib.net/zlib-1.2.11.tar.gz
$ tar -zxf zlib-1.2.11.tar.gz
$ cd zlib-1.2.11
$ ./configure
$ make
$ sudo make install

wget -O openssl.tar.gz -c https://github.com/openssl/openssl/archive/OpenSSL_1_0_2l.tar.gz
tar zxf openssl.tar.gz
mv openssl-OpenSSL_1_0_2l/ openssl

下载安装Nginx

  • 源码编译安装Nginx 
    $ wget https://nginx.org/download/nginx-1.14.0.tar.gz
$ tar zxf nginx-1.14.0.tar.gz
$ cd nginx-1.14.0

./configure --sbin-path=/usr/local/nginx/nginx --conf-path=/usr/local/nginx/nginx.conf --pid-path=/usr/local/nginx/nginx.pid --with-pcre=../pcre-8.42 --with-zlib=../zlib-1.2.11 --with-http_ssl_module --with-stream --with-stream_ssl_module --with-openssl=/opt/openssl
  • 编译安装 
    make && make install

EMQ X节点集群

EMQ X 的节点集群，可参阅官方文档：http://emqtt.com/docs/v2/cluster.html

Nginx TCP/SSL 反向代理设置

TCP 反向代理设置

  • 反向代理设置： 
    $ mkdir -p  /usr/local/nginx/tcp.d/

$ cat <<- 'EOF' >> /usr/local/nginx/nginx.conf
include /usr/local/nginx/tcp.d/*.conf;
EOF

$ vim emqx_tcp_nginx.conf

stream
{
log_format proxy '$remote_addr [$time_local] '
                 '$protocol $status $bytes_sent $bytes_received '
                 '$session_time "$upstream_addr" '
                 '"$upstream_bytes_sent" "$upstream_bytes_received" "$upstream_connect_time"';

    access_log /var/log/nginx/tcp-access.log proxy ;
    open_log_file_cache off;
upstream mqtt1883 {
    #zone tcp_servers 64k;
    #hash $remote_addr;
    server 192.168.1.10:1883 weight=1;
    server 192.168.1.13:1883 weight=1;
}
server {
        listen       1883;
        proxy_send_timeout 2h;
        proxy_read_timeout 2h;
        proxy_connect_timeout 150s;
        proxy_timeout 150s;
        proxy_pass mqtt1883;
        proxy_buffer_size 3M;
        tcp_nodelay on;
}
}
    客户端通过连接 地址，Nginx将连接负载到 EMQ X 节点，EMQ 君测试200个客户端连接效果，读者可以看到200个连接已经被分布到2个EMQ X 节点上了：

SSL 反向代理设置

以下配置文件中，证书直接使用的 EMQ X 自带证书，EMQ X 自带证书目录 emqx/etc/certs。 ``` $ cat emqx_ssl_nginx.conf

stream{
  upstream backend{
    #   hash $remote_addr consistent;
        least_conn;
        server 192.168.1.10:1883 weight=1;
        server 192.168.1.13:1883 weight=1;
        }

  server {
        listen 8883 ssl;
        proxy_send_timeout 2h;
        proxy_read_timeout 2h;
        proxy_connect_timeout 150s;
        proxy_timeout 150s;
        proxy_pass backend;
        proxy_buffer_size 3M;
        ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers         AES128-SHA:AES256-SHA:RC4-SHA:DES-CBC3-SHA:RC4-MD5;
        ssl_certificate     /root/nginx-certs/certs/cert.pem;
        ssl_certificate_key /root/nginx-certs/certs/key.pem;
        ssl_session_cache   shared:SSL:10m;
        ssl_session_timeout 10m;
        }
}
```

客户端通过 SSL 方式连接 地址，Nginx将连接以 TCP 方式负载到 EMQ X 节点。

总结

以上配置，简明的介绍了关于 Nginx 在 TCP/SSL 反向代理的配置，在实际生产环境当中，可根据实际情况，来调整 Nginx 配置参数。读者可直接参考本文结合 Nginx 官网文档进行测试，enjoy it o

EMQ X Message Server Nginx Reverse Proxy

Foreword

Many newcomers who use the EMQ X message server will consider the expansion of the EMQ X node as the business grows. When you have multiple nodes and there is no proxy before the node, you need to specify which EMQ X node to be connected by the client. If one of the nodes is upgraded, you need to reset on the client. However, if the reverse proxy is loaded on the cluster node, you can assign the connection to the other node through the reverse proxy, thus avoiding changes and redeployments on the client. Mr. EMQ is going to about Nginx reverse proxy today.

Nginx Installation

Dependent installation

Install pcre、zlib、openssl

$ wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.42.tar.gz
$ tar -zxf pcre-8.42.tar.gz
$ cd pcre-8.42
$ ./configure
$ make
$ sudo make install
  
$ wget http://zlib.net/zlib-1.2.11.tar.gz
$ tar -zxf zlib-1.2.11.tar.gz
$ cd zlib-1.2.11
$ ./configure
$ make
$ sudo make install
  
$ wget -O openssl.tar.gz -c https://github.com/openssl/openssl/archive/OpenSSL_1_0_2l.tar.gz
$ tar zxf openssl.tar.gz
$ mv openssl-OpenSSL_1_0_2l/ openssl

Download and install Nginx

Source code compile and install Nginx

$ wget https://nginx.org/download/nginx-1.14.0.tar.gz
$ tar zxf nginx-1.14.0.tar.gz
$ cd nginx-1.14.0
  
$ ./configure --sbin-path=/usr/local/nginx/nginx --conf-path=/usr/local/nginx/nginx.conf --pid-path=/usr/local/nginx/nginx.pid --with-pcre=../pcre-8.42 --with-zlib=../zlib-1.2.11 --with-http_ssl_module --with-stream --with-stream_ssl_module --with-openssl=/opt/openssl

Compile and install

make && make install

EMQ X node cluster

The node cluster of EMQ X can be found in the official documentation：http://emqtt.com/docs/v2/cluster.html

Nginx TCP/SSL reverse proxy settings

TCP reverse proxy settings

reverse proxy settings：

$ mkdir -p  /usr/local/nginx/tcp.d/
  
$ cat <<- 'EOF' >> /usr/local/nginx/nginx.conf
  include /usr/local/nginx/tcp.d/*.conf;
  EOF
  
$ vim emqx_tcp_nginx.conf
  
  stream
  {
  log_format proxy '$remote_addr [$time_local] '
                   '$protocol $status $bytes_sent $bytes_received '
                   '$session_time "$upstream_addr" '
                   '"$upstream_bytes_sent" "$upstream_bytes_received" "$upstream_connect_time"';
  
      access_log /var/log/nginx/tcp-access.log proxy ;
      open_log_file_cache off;
  upstream mqtt1883 {
      #zone tcp_servers 64k;
      #hash $remote_addr;
      server 192.168.1.10:1883 weight=1;
      server 192.168.1.13:1883 weight=1;
  }
  server {
          listen       1883;
          proxy_send_timeout 2h;
          proxy_read_timeout 2h;
          proxy_connect_timeout 150s;
          proxy_timeout 150s;
          proxy_pass mqtt1883;
          proxy_buffer_size 3M;
          tcp_nodelay on;
  }
  }

The client connects the address , and Nginx distribute the connect to the EMQ X node. Mr. EMQ test 200 client connections, and the reader can see that 200 connections have been distributed to the 2 EMQ X nodes.

SSL reverse proxy settings

In the following configuration file, the EMQ X comes with a certificate directly used by the certificate, with the certificate directory emqx/etc/certs.

$ cat emqx_ssl_nginx.conf
stream{
  upstream backend{
    #   hash $remote_addr consistent;
        least_conn;
        server 192.168.1.10:1883 weight=1;
        server 192.168.1.13:1883 weight=1;
        }

  server {
        listen 8883 ssl;
        proxy_send_timeout 2h;
        proxy_read_timeout 2h;
        proxy_connect_timeout 150s;
        proxy_timeout 150s;
        proxy_pass backend;
        proxy_buffer_size 3M;
        ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers         AES128-SHA:AES256-SHA:RC4-SHA:DES-CBC3-SHA:RC4-MD5;
        ssl_certificate     /root/nginx-certs/certs/cert.pem;
        ssl_certificate_key /root/nginx-certs/certs/key.pem;
        ssl_session_cache   shared:SSL:10m;
        ssl_session_timeout 10m;
        }
}

The client connects to the address via SSL, and Nginx will distribute the connection to the EMQ X node in TCP mode.

Summary

The above configuration briefly introduces the configuration of Nginx in the TCP/SSL reverse proxy. In the actual working environment, the Nginx configuration parameters can be adjusted. Readers can do the test by reference to this article and the documentation from Nginx official website, enjoy it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment