Skip to content

Nginx performance degradation in Gramine-SGX attributed to SHA256 hashing #1712

@sahason

Description

@sahason

Description of the problem

Poor performance of Nginx in Gramine-SGX is seen due to ~60% overhead of mbedtls_sha256_update. Please see the attached perf report.
We used wrk benchmarking tool to compare the performance of Nginx on Native and Gramine-SGX with threads 1 and 64. The wrk command used: wrk -t64 -c300 -d30s http://127.0.0.1:8002/random/10K.1.html. The statistics of Throughput (Requests/sec) shows a poor performance of Nginx in Garmine-SGX.

Threads Gramine-SGX (Requests/Sec) Native (Requets/Sec) Native vs Gramine-SGX
1 44145.58 86392.24 -48.90
64 427658.63 947360.14 -54.86

Few observations:

  1. The performance degradation is more for bigger file size.
  2. Keeping the folder containing 10K.1.html under allowed files in manifest file improves the performance.
  3. Commenting out SHA256 calls in copy_and_verify_trusted_files improves the performance a lot. Please check below table.
Threads Gramine-SGX (Requests/Sec) Native (Requests/Sec) Native vs Gramine-SGX
64 876094.83 943265.29 -7.12

Please suggest how can we improve the throughput.

With 64 threads:
perf-report

Steps to reproduce

Build and run Nginx server:

  1. cd CI-Examples/nginx
  2. Modify nginx-gramine.conf.template with the below data
worker_processes auto; 

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;

events {
    worker_connections  768;
}

http {
    include            mime.types;
    default_type       application/octet-stream;
    keepalive_timeout  13;
    access_log  off;
    client_body_buffer_size 80k;
    client_max_body_size 9m;
    client_header_buffer_size 1k;
    client_body_timeout 10;
    client_header_timeout 10;
    send_timeout 10;
    open_file_cache max=1024 inactive=10s;
    open_file_cache_valid 60s;
    open_file_cache_min_uses 2;
    open_file_cache_errors on;
    sendfile           on;
    #keepalive_timeout  65;

    # a single HTTP/HTTPS server
    server {
        listen 8002;
        listen  8444 ssl;
        server_name 127.0.0.1;

        ssl_certificate            server.crt;
        ssl_certificate_key        server.key;
        ssl_session_cache          shared:SSL:10m;
        ssl_session_timeout        10m;
        ssl_protocols              TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers                HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers  on;

        location / {
            root   html;
            index  index.html index.htm;
        }

        # redirect server error pages to the static page /50x.html
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

        access_log off;
    }
}

daemon off;
  1. make SGX=1
  2. gramine-sgx nginx

Run benchmark:
wrk -t64 -c300 -d30s http://127.0.0.1:8002/random/10K.1.html

Expected results

No response

Actual results

No response

Gramine commit hash

1f72aaf

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions