Nginx as a TCP/UDP relay

Author: Solène

Date: 24 February 2021

Tags: openbsd nginx networking

Comment on Mastodon

Introduction

In this tutorial I will explain how to use Nginx as a TCP or UDP relay as an alternative to Haproxy or Relayd. This mean nginx will be able to accept requests on a port (TCP/UDP) and relay it to another backend without knowing about the content. It also permits to negociates a TLS session with the client and relay to a non-TLS backend. In this example I will explain how to configure Nginx to accept TLS requests to transmit it to my Gemini server Vger, Gemini protocol has TLS as a requirement.

I will explain how to install and configure Nginx and how to parse logs to obtain useful information. I will use an OpenBSD system for the examples.

It is important to understand that in this context Nginx is not doing anything related to HTTP.

Installation

On OpenBSD we need the package nginx-stream, if you are unsure about which package is required on your system, search which package provide the file ngx_stream_module.so . To enable Nginx at boot, you can use rcctl enable nginx.

Nginx stream module core documentation

Nginx stream module log documentation

Configuration

The default configuration file for nginx is /etc/nginx/nginx.conf , we will want it to listen on port 1965 and relay to 127.0.0.1:11965.

worker_processes  1;

load_module modules/ngx_stream_module.so;

events {
   worker_connections 5;
}

stream {
    log_format basic '$remote_addr $upstream_addr [$time_local] '
                     '$protocol $status $bytes_sent $bytes_received '
                     '$session_time';

    access_log logs/nginx-access.log basic;

    upstream backend {
        hash $remote_addr consistent;
        server 127.0.0.1:11965;
    }
    server {
        listen 1965 ssl;
        ssl_certificate /etc/ssl/perso.pw:1965.crt;
        ssl_certificate_key /etc/ssl/private/perso.pw:1965.key;
        proxy_pass backend;
    }
}

In the previous configuration file, the backend defines the destination, multiples servers could be defined, with weights and timeouts, there is only one in this example.

The server block will tell on which port Nginx should listen and if it has to handle TLS (which is named ssl because of history), usual TLS configuration can be used here, then for a request, we have to tell to which backend Nginx have to relay the connections.

The configuration file defines a custom log format that is useful for TLS connections, it includes remote host, backend destination, connection status, bytes transffered and duration.

Log parsing

Using awk to calculate time performance

I wrote a quite long shell command parsing the log defined earlier that display the number of requests, and median/min/max session time.

$ awk '{ print $NF }' /var/www/logs/nginx-access.log | sort -n |  awk '{ data[NR] = $1 } END { print "Total: "NR" Median:"data[int(NR/2)]" Min:"data[2]" Max:"data[NR] }'
Total: 566 Median:0.212 Min:0.000 Max:600.487

Find bad clients using awk

Sometimes in the logs there are clients that obtains a status 500, meaning the TLS connection haven't been established correctly. It may be some scanner that doesn't try a TLS connection, if you want to get statistics about those and see if it would be worth to block them if they do too many attempt, it is easy to use awk to get the list.

awk '$(NF-3) == 500 { print $1 }' /var/www/logs/nginx-access.log

Using goaccess for real time log visualization

It is also possible to use the program Goaccess to view logs in real time with many information, it is really an awesome program.

goaccess --date-format="%d/%b/%Y" \
         --time-format="%H:%M:%S" \
         --log-format="%h %r [%d:%t %^] TCP %s %^ %b %L" /var/www/logs/nginx-access.log

Goaccess official website

Conclusion

I was using relayd before trying Nginx with stream module, while relayd worked fine it doesn't provide any of the logs Nginx offer. I am really happy with this use of Nginx because it is a very versatile program that shown to be more than a http server over time. For a minimal setup I would still recommend lighter daemon such as relayd.

-- Response ended

-- Page fetched on Thu Apr 25 03:46:16 2024