intel/intel-optimized-nginx repository overview

⁠Introduction

Nginx* is a high-performance HTTP/HTTPS and reverse proxy web server based on a BSD-like license. Nginx uses SSL/TLS to enhance web access security. Intel has introduced the Crypto-NI software solution which is based on 3rd generation Intel® Xeon® Scalable Processors (Codename Ice Lake/Whitley). It can effectively improve the security of web access. Intel_Asynch_Nginx is an Intel optimized version Nginx, used by Intel to support Async hardware and software acceleration for https.

The main software used in this solution are IPP Cryptography Library, Intel Multi-Buffer Crypto for IPsec Library (intel-ipsec-mb) and Intel® QuickAssist Technology (Intel® QAT), which provide batch submission of multiple SSL requests and parallel asynchronous processing mechanism based on the new instruction set, greatly improving the performance. Intel® QuickAssist Accelerator is a PCIe card that needs to be inserted into the PCIe slot in the server at the start.

⁠Docker Image

• intel/intel-optimized-nginx:qathw:
• intel/intel-optimized-nginx:qatsw:

The docker images use Intel optimized async version of Nginx which can use QAT engine async operations to accelerate https performance from https://github.com/intel/asynch_mode_nginx

⁠How to use this image

⁠Generate certificates:

Create the TLS key and certificate for enabling encryption:

openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout <path>/key_rsa2048.key -out <path>/cert_rsa2048.crt -subj "/C=US/ST=State/L=Locality/O=Company/OU=Section/CN=company.com"

⁠Run this image

• qathw: sudo docker run -it --cap-add=IPC_LOCK --security-opt seccomp=unconfined $(for i in ls /dev/vfio/*; do echo --device $i; done) -v <path>/key_rsa2048.key:/keys/key_rsa2048.key -v <path>/cert_rsa2048.crt:/certs/cert_rsa2048.crt intel/intel-optimized-nginx:qathw

• qatsw: sudo docker run -it -v <path>/key_rsa2048.key:/keys/key_rsa2048.key -v <path>/cert_rsa2048.crt:/certs/cert_rsa2048.crt intel-optimized-nginx:qatsw

⁠Configure & run nginx

For information on the syntax of the nginx configuration files, see the official documentation⁠ (specifically the Beginner's Guide⁠) and Asynch Mode for NGINX* readme⁠. Below is an example conf file for asynch_mode_nginx with qatengine enabled. Edit this file inside the container, then run nginx with nginx -c <location of the conf file>

user root;
worker_processes 4;

# nginx_use_qat_engine=true
#start_qat_engine_block
load_module modules/ngx_ssl_engine_qat_module.so;
ssl_engine {
        use_engine qatengine;
        default_algorithms ALL;
        qat_engine {
                qat_offload_mode async;
                qat_notify_mode poll;
                qat_external_poll_interval 1;
        }
}
#end_qat_engine_block

worker_rlimit_nofile 1000000;

events {
        use epoll;
        worker_connections 8192;
        multi_accept on;
        accept_mutex on;
}

http {
        ssl_buffer_size 65536;
        include /usr/local/share/nginx/conf/mime.types;
        default_type application/octet-stream;
                sendfile on;
                access_log off;

        server {
                listen 4400 ssl reuseport backlog=131072 so_keepalive=off;
                tcp_nopush on;
                tcp_nodelay on;
                ssl_verify_client off;
                ssl_session_tickets off;
                access_log  off;
                lingering_close off;
                lingering_time 1;

                server_name example.com;
                ssl_asynch on;

                ssl_certificate  /certs/cert_rsa2048.crt;
                ssl_certificate_key /keys/key_rsa2048.key;

                ssl_session_timeout 300s;

                ssl_protocols TLSv1.3;

                ssl_conf_command Ciphersuites TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_SHA256:TLS_CHACHA20_POLY1305_SHA256;
                ssl_prefer_server_ciphers on;

                location / {
                        index index.html index.htm;
                }

        }
}

⁠QAT In-tree Driver Setup For 4xxx Device

Check System Prerequisites

• Platform must have Intel® QuickAssist Technology QAT device such as "4xxx"
• QAT Physical Functions (PF's) can be determined as under:

lspci -d 8086:4942
76:00.0 Co-processor: Intel Corporation Device 4942 (rev 40)
...

or

lspci -d 8086:4940
6b:00.0 Co-processor: Intel Corporation Device 4940 (rev 40)
...

• Firmware must be available.

  check that these files exist:

  /lib/firmware/qat_4xxx.bin or /lib/firmware/qat_4xxx.bin.xz

  /lib/firmware/qat_4xxx_mmp.bin or /lib/firmware/qat_4xxx_mmp.bin.xz

  if not, download form:

  https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/qat_4xxx.bin
  https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/qat_4xxx_mmp.bin
  

  Copy

  On updating these files run

  sudo rmmod qat_4xxx
  sudo modprobe qat_4xxx
  sudo dracut --force
  

  Copy

  to update kernel modules and initramfs.

Required Kernel Information:

• Linux kernel v5.11+ (This is for crypto, for compression use v5.17+)
• Fedora 34+ (for compression use 36+)
• RHEL 8.4+ (for compression use 9.0+)

BIOS Settings

• Intel VT-d and SR-IOV must be enabled in the platform (BIOS).

Grub Settings

• sudo grubby --update-kernel=DEFAULT --args="intel_iommu=on vfio-pci.disable_denylist=1 iommu=pt default_hugepagesz=2M hugepagesz=2M hugepages=4096"
• Reboot system

• sudo vim /etc/default/grub
• move to GRUB_CMDLINE_LINUX add intel_iommu=on vfio-pci.disable_denylist=1 iommu=pt default_hugepagesz=2M hugepagesz=2M hugepages=4096
• sudo update-grub
• Reboot system

Install QATLib

# Install QATLib
sudo dnf install -y qatlib-devel

# Add your user to qat group and re-login to make the change effective
sudo usermod -a -G qat `whoami`
sudo su -l $USER

# Make sure qat service is started properly and ready for use.
sudo systemctl stop qat.service
sudo systemctl enable qat.service
sudo systemctl restart qat.service
sudo systemctl status qat.service

Other Distributions: Building From Source

# Install dependencies
sudo dnf update -y
sudo dnf install -y gcc systemd-devel automake autoconf libtool
sudo dnf install -y openssl-devel zlib-devel yasm

# Install dependencies
sudo apt update -y
sudo apt install -y build-essential cmake g++ pkg-config wget make yasm nasm libboost-all-dev libnl-genl-3-dev zlib1g zlib1g-dev
apt install -y systemd m4 pkg-config libudev-dev libssl-dev autoconf libtool tar git

Build & install

git clone https://github.com/intel/qatlib
cd qatlib
./autogen.sh
./configure --prefix=/usr --enable-service
make -j
sudo make install

# Make sure qat service is started properly and ready to use
sudo systemctl stop qat.service
sudo systemctl enable qat.service
sudo systemctl restart qat.service
sudo systemctl status qat.service

⁠References

• https://github.com/intel/qatlib/blob/main/INSTALL⁠

• intel-device-plugins-for-kubernetes/Dockerfile at main · intel/intel-device-plugins-for-kubernetes · GitHub⁠

• https://doc.dpdk.org/guides/cryptodevs/qat.html⁠

• https://github.com/intel/3rd-party-codes-for-qat-crypto-base-external-release⁠

• https://github.com/intel/3rd-party-codes-for-qat-crypto-base-external-release/blob/main/Release.md⁠