This repository contains the artifacts for the paper "On the Security of SSH Client Signatures", accepted at the ACM Conference on Computer and Communications Security (CCS) 2025.
We recommend having a machine with at least 16 CPU cores and 64 GB of RAM available. This is required to run the Elasticsearch stack and the various analysis tools included in this repository. Running with less than the recommended hardware may result in degraded performance or unpredictable results. Also, we recommend having at least 50 GB of free disk space to avoid running out of space during experiments.
We ran our experiments on a server with 2 AMD EPYC 7763 64-core CPUs and 2 TB of RAM.
- Ubuntu 24.04 LTS. We recommend a fresh installation to avoid possible side effects.
- Docker 28.3.3 or newer.
- Golang 1.25.0 or newer.
- Python 3.11.
- SageMath 10.3 installed as a Python library.
To ease installation, we provide a setup script (scripts/00_setup_env.sh) that
automates the installation of the required software and dependencies. It is
designed to be run on a fresh Ubuntu 24.04 LTS installation. Note that the
docker containers require support for modern CPU architectures, please use
host-passthrough or similar for the CPU mode in your VM if you do not run on
native hardware. In particular, it will perform the following steps:
- Install system packages
- Latest version of Docker
- Python 3.11 with venv module installed
- SageMath 10.3 dependencies
- Install Golang 1.25.0
- Sets up a Python virtual environment in
venv/ - Install required Python dependencies for the evaluation scripts and tools into the virtual environment
- Install SageMath 10.3 as a Python library
- Build the key_scraper and nonce_sampler tools using
go build - Set up Docker containers for the Elasticsearch and MongoDB infrastructure
- Copy the auto-generated Elasticsearch CA certificate to the host
(
code/key_scraper/ca.crt)
Important
After running the setup script, you must restart your system to ensure all changes take effect.
Tip
Building SageMath 10.3 as a Python library can take a significant amount of
time, please be patient. You may track the progress by tailing the log file
(tail -f logs/setup_env.log).
For evaluation, accounts on GitHub, GitLab, and Launchpad are required to access the API (GitHub and GitLab only) and to upload generated SSH keys while testing for public key upload restrictions. Free accounts are sufficient for this purpose. Uploaded SSH keys should be removed immediately after testing.
To test basic functionality after running scripts/00_setup_env.sh, you can
perform the following checks:
-
Verify that Docker, Golang, and Python are installed and accessible from the command line:
docker --version go version python3.11 --version
Expected output (exact versions for Docker and Python can differ):
user@ccs25ae:~/SSH-Client-Signatures-Artifacts$ docker --version Docker version 28.4.0, build d8eb465 user@ccs25ae:~/SSH-Client-Signatures-Artifacts$ go version go version go1.25.0 linux/amd64 user@ccs25ae:~/SSH-Client-Signatures-Artifacts$ python3.11 --version Python 3.11.13
-
Verify that the Python virtual environment can be activated and, in particular, that you can load the SageMath library:
. venv/bin/activate python -c "from sage.all import Primes; print(Primes())" deactivate
Expected output:
user@ccs25ae:~/SSH-Client-Signatures-Artifacts$ . venv/bin/activate (venv) user@ccs25ae:~/SSH-Client-Signatures-Artifacts$ python -c "from sage.all import Primes; print(Primes())" Set of all prime numbers: 2, 3, 5, 7, ... (venv) user@ccs25ae:~/SSH-Client-Signatures-Artifacts$ deactivate
-
Verify that the Elasticsearch stack is running and that both, Elasticsearch and Kibana, are available:
docker ps curl -u elastic:elasticsearchpass --cacert code/key_scraper/ca.crt "https://localhost:9200/_cluster/health?pretty" curl "http://localhost:5601/api/status"
Expected output:
user@ccs25ae:~/SSH-Client-Signatures-Artifacts$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES aebdf9a4be7e docker.elastic.co/kibana/kibana:9.1.2 "/bin/tini -- /usr/l…" 11 minutes ago Up 10 minutes (healthy) 127.0.0.1:5601->5601/tcp sshks-kibana-1 3aad35b1244d docker.elastic.co/elasticsearch/elasticsearch:9.1.2 "/bin/tini -- /usr/l…" 11 minutes ago Up 10 minutes (healthy) 9200/tcp, 9300/tcp sshks-es03-1 5a79993a8353 docker.elastic.co/elasticsearch/elasticsearch:9.1.2 "/bin/tini -- /usr/l…" 11 minutes ago Up 10 minutes (healthy) 9200/tcp, 9300/tcp sshks-es02-1 57f0fb088fdd docker.elastic.co/elasticsearch/elasticsearch:9.1.2 "/bin/tini -- /usr/l…" 11 minutes ago Up 10 minutes (healthy) 127.0.0.1:9200->9200/tcp, 9300/tcp sshks-es01-1 87fa1a955f51 mongo:latest "docker-entrypoint.s…" 11 minutes ago Up 10 minutes 0.0.0.0:27017->27017/tcp, [::]:27017->27017/tcp sshks-mongodb-1 user@ccs25ae:~/SSH-Client-Signatures-Artifacts$ curl -u elastic:elasticsearchpass --cacert code/key_scraper/ca.crt "https://localhost:9200/_cluster/health?pretty" { "cluster_name": "sshks-cluster", "status": "green", "timed_out": false, "number_of_nodes": 3, "number_of_data_nodes": 3, "active_primary_shards": 40, "active_shards": 80, "relocating_shards": 0, "initializing_shards": 0, "unassigned_shards": 0, "unassigned_primary_shards": 0, "delayed_unassigned_shards": 0, "number_of_pending_tasks": 0, "number_of_in_flight_fetch": 0, "task_max_waiting_in_queue_millis": 0, "active_shards_percent_as_number": 100.0 } user@ccs25ae:~/SSH-Client-Signatures-Artifacts$ curl "http://localhost:5601/api/status" {"status":{"overall":{"level":"available"}}}
Warning
If Elasticsearch is unreachable, verify that the vm.max_map_count
limit is set to 262144 or higher. To access the current limit, run
sysctl -n vm.max_map_count. To increase the limit, run
sysctl -w vm.max_map_count=262144 and persist the setting by adding or
updating the vm.max_map_count=262144 setting in the /etc/sysctl.conf file.
When running scripts/00_setup_env.sh to set up the environment, the script
will take care of this.
The following major claims are made with regard to the artifacts:
- The
key_scrapertool (code/key_scraper) is capable of collecting SSH public keys from multiple platforms (GitHub, GitLab, Launchpad) systematically. - The evaluation pipeline (
code/key_scraper/scripts) can identify and analyze weak SSH keys based on the methodology described in Section 3.2 in the paper. - The
07-generate-test-keys.pyscript can generate a diverse set of SSH keys violating certain properties for testing upload restrictions. At the time of artifact evaluation, these keys were able to reproduce Table 3 in the paper. - The
nonce_samplertool (code/nonce_sampler) can test ECDSA and EdDSA signatures generated by SSH clients and agents for determinism and potential bias. It can find the biased nonce vulnerability in PuTTY <= 0.80 (CVE-2024-31497). - The success rate of the sieve_pred algorithm in the context of the PuTTY biased nonce vulnerability (521-bit nonces, 9 bits biased) is given by Figure 5 in the paper. In particular, the attack always succeeds when at least 60 signatures with biased nonces are available.
Tip
Each claim can be proven by the corresponding experiment with the same number, that is, claim 1 can be proven by E1, claim 2 by E2, and so on.
Note
Our paper is based on two full runs of the key scraper evaluation pipeline (claims 1 and 2) in June 2023 and January 2025. Exact reproduction of the results in the paper is not possible due to the changing nature of the source data. Additionally, a full run will require over a month of real time due to API limits. To account for this, the experiments E1 and E2 are designed to demonstrate the capabilities of the proposed tools and methodologies at a small scale reasonable for evaluation.
For running the artifacts, you can use the provided scripts in the scripts/
directory. Each script is designed to perform a specific task in the evaluation
pipeline.
Note
- Estimated time required: 24 hours (adjustable)
- Interaction required: yes, initial API token generation (estimate: 10 minutes)
- Other requirements: Internet access, API tokens on GitHub / GitLab
- Proves claim: C1 via inspection of the scraped data
To collect SSH public keys from GitHub, GitLab and Launchpad, you can use the
scripts/01_run_scraper.sh script. This script will configure and then start
the key scraper and collect SSH public keys from the platforms for a duration of
24 hours. To adjust the execution time, pass a duration value to the script,
e.g. scripts/01_run_scraper.sh 2h (any duration value compatible with the
timeout utility is supported, see
the corresponding man page). Adjust the
timeout in the script as needed (or remove entirely).
To run the scraper on GitHub and GitLab, you will need to provide a personal
access token to authenticate with the API of each service. Refer to
GitHub Docs
and GitLab Docs
for more information on how to generate one. For GitHub, we used a classic
access token rather than a fine-grained one - if you decide to generate a
fine-grained token, selecting public repository access and no account
permissions should be sufficient. For GitLab, you must assign at least
read_api and read_user scopes to the access token.
Note
Since our evaluation in January 2025, GitLab started to enforce stricter rate limits on their API, which may negatively affect the data collection process. See their announcement for further details. Our testing indicates that these rate limits only slow down, but not prevent, the data collection process.
After execution, the Elasticsearch database should be populated with user
records from all three platforms containing their SSH public keys. You can use
Kibana (available at http://localhost:5601) to explore the collected dataset.
Use elastic:elasticsearchpass as the credentials to log in. Navigate to the
"Discover" app (http://localhost:5601/app/discover) and create a new data view
(see
Elastic Docs)
with an sshks_users* index pattern. For the timestamp field, select
visitedAt.
Note
- Estimated time required: 1 hour (depending on available CPU cores)
- Interaction required: no
- Other requirements: none
- Proves claim: C2 via result files in
results/
Once you have collected a decent amount of SSH public keys, run the evaluation
by pipeline by calling scripts/02_evaluate_keys.sh. This script will perform a
full run of the evaluation pipeline on the keys collected by the key scraper
tool. In particular, it will perform the following steps:
- Activate the virtual environment in
venv - Extract the SSH keys from the user-oriented scraping results
(
01-extract-keys-sshks.py). Errors are logged toresults/01-extract-keys-sshks-errors.txt - Collect unique SSH keys and stores it into a separate index
(
01-extract-keys-sshks-unique.py). - Evaluate basic statistics for the dataset (
02-summarize-keys.py):results/02-summary-ecdsa-curve-dist.png- Distribution of ECDSA key curves used in the collected SSH keys (Figure 4 in the paper).results/02-summary-rsa-modulus-cdf.png- (Complementary) CDF of RSA key modulus sizes used in the collected SSH keys.results/02-summary-rsa-modulus-dist.png- Distribution of RSA key modulus sizes used in the collected SSH keys (Figure 3 in the paper).02-summary.tex- LaTeX table used as a base for Table 2 in the paper.
- Analyze the SSH keys according to the methodology described in Section 3.2 of the paper.
- Generate a report summarizing the findings of the evaluation pipeline.
- Collect users affected by weak keys from the Elasticsearch database for disclosure.
All results can be found in the results directory after running the script.
Additionally, the summarized findings are printed to stdout.
Note
- Estimated time required: 1 hour
- Interaction required: yes, uploading test keys to platforms (estimate: 20 minutes per platform)
- Other requirements: accessible account on GitHub, GitLab, and Launchpad
- Proves claim: C3 by comparison between the platforms' responses and Table 3 in the paper
To test the public key upload restrictions of various platforms, you can
generate SSH keys with specific properties (e.g., weak keys) and attempt to
upload them to the platforms. To generate the keys, run
scripts/03_generate_test_keys.sh. This will call the
07-generate-test-keys.py python script and store the generated keys in the
results directory. The resulting file contains one SSH key per line that can
be copied into the corresponding SSH key upload forms on
GitHub,
GitLab, and Launchpad
(https://launchpad.net/~*user*/+editsshkeys). If the key is accepted without
error and is visible in the account's list of SSH keys, we consider the upload
successful. Uploading each key to each platform can reproduce Table 3 in the
paper.
The file format of the resulting 07-test-keys.txt file is based on OpenSSH's
known_hosts file. In this format, each line contains a single key with an
explicit key type and comment. The format is <key type> <key> <comment> where
<key type> is either ssh-dss (for DSA keys), ssh-rsa (for RSA keys),
ecdsa-sha2-nistp256 (for ECDSA keys), or ssh-ed25519 (for Ed25519 keys). The
comment contains a short description of the key and allows matching the key to
the entries in Table 3.
Note
- Estimated time required: 5 - 30 minutes / client
- Interaction required: yes, installing clients and performing connections
- Proves claim: C4 by comparison between the output of the script and Table 4 in the paper
To test an SSH client or agent for nonce determinism and bias, run
scripts/04_measure_client_agent.sh. The script ask for which algorithm the
nonce_sampler tool should be invoked and whether the implementation to test is
a client or agent.
If client is selected, the tool will listen on port 2200 for incoming client connections. Connect to the tool using your preferred SSH client with the selected key configured for authentication (the full key path will be printed during script execution). During the first connection, the tool will try to determine the nonce generation algorithm used by the client. Manually terminate the SSH client connection if the tool is stuck in the signature collection phase (in these cases, the client continuously tries to authenticate in the same connection).
Afterward, the tool will measure nonce bias. Connect to port 2200 using the same
SSH client. Depending on the client, manual interaction with the client is
necessary as some clients do not support partial authentication or limit the
number of authentication attempts in a single connection. Once the required
number of signatures has been collected (indicated by the remaining: counter),
terminate any client connection and the tool will proceed to the nonce bias
measurement phase.
As an example, the following commands cause the local OpenSSH client to connect to the nonce_sampler tool using the NIST P-256 key (adjust as necessary):
ssh -i code/nonce_sampler/keys/id_ecdsa_nistp256 -p 2200 sample@localhost
# Abort after a few seconds using Ctrl+C
ssh -i code/nonce_sampler/keys/id_ecdsa_nistp256 -p 2200 sample@localhost
# Wait until remaining < 0, then terminate using Ctrl+CIf agent is selected, the tool will automatically try to connect to the UNIX socket given by the $SSH_AUTH_SOCK environment variable. Make sure the SSH agent is running before running the script. The tool will try to load the corresponding key into the agent using the SSH agent protocol. If the agent uses preconfigured keys only, make sure to load the key before running the script. To test a Windows-based agent, connect to the VM via SSH with agent forwarding enabled. This will automatically set the $SSH_AUTH_SOCK in the shell session.
As an example, the following commands can be used to start the OpenSSH ssh-agent. ssh-agent support adding keys dynamically through the SSH agent protocol. Hence, adding the keys before running the script is not necessary.
eval "$(ssh-agent -s)"When testing PuTTY 0.80, the script should output a biased nonce with the top 9 bits of the nonce being zero all the time and k_proto for the nonce generation method. For all other clients tested, the tool should not report any bias, while reporting the nonce generation method as in Table 4 in the paper. Similarly, the nonce generation method of an agent should coincide with the one given in Table 5.
Tip
PuTTY uses a proprietary key format (.ppk) for private keys and cannot use
keys generated by ssh-keygen directly. In the 00_setup_env.sh script,
ssh-keygen is used to bootstrap the evaluation keys. Therefore, to test
determinism and bias of a PuTTY client, these keys must be converted to the
PuTTY format before use. To do so, run the following command inside the
artifact's root directory. It will convert the private keys into PuTTY's .ppk
format, which must then be used inside PuTTY.
find code/nonce_sampler/keys -maxdepth 1 -type f -not -name \*.pub -not -name \*.ppk -exec puttygen {} -o {}.ppk -O private \;If the puttygen executable is not available in your PATH (for example, if
you build PuTTY from source), adjust the command to include the relative or
absolute path.
You can also find a pre-recorded demonstration of us running experiment E4 against a Windows-based PuTTY client in the supp_material folder.
We cannot provide the tested SSH clients and agents as part of these artifacts due to licensing restrictions. However, the following tables based on Tables 4 and 5 in the paper may be helpful in reproducing the results:
| Client Name | Version | OS | Nonce Scheme | Download Link |
|---|---|---|---|---|
| AbsoluteTelnet | 12.16 | Windows | Not RFC 6979 / k_proto | Link |
| AsyncSSH | 2.18.0 | Linux | Random | Link |
| Bitvise | 9.42 | Windows | Not RFC 6979 / k_proto | Link |
| Cyberduck | 9.0.1 | Windows | Random | Link |
| Dropbear | 2024.86 | Linux | Random | Link |
| Erlang/OTP SSH | 5.2.1 | Linux | Not RFC 6979 / k_proto | Link |
| FileZilla | 3.67.0 | Windows | Random | Link (current version only) |
| Golang x/crypto/ssh | 0.29.0 | Windows | Optional (Any) | Link |
| libssh | 0.11.1 | Linux | Random (OpenSSL) / RFC 6979 (MBedTLS) | Link |
| OpenSSH Portable | 9.9p1 | Linux | Random | Link |
| OpenSSH Portable | 9.9p1 | MacOS | Random | Link |
| Paramiko | 3.5.0 | Linux | Random | Link |
| PKIX-SSH | 15.3 | Linux | Random | Link |
| PortX | 2.2.12 | MacOS | Random | Link (current version only) |
| PuTTY | 0.80 | Windows | k_proto | Link |
| PuTTY | 0.81 | Windows | RFC 6979 | Link |
| SecureCRT | 9.5.2 | Windows | Random | Link (current version only) |
| Secure Shellfish | 2025.17 | MacOS | Not RFC 6979 / k_proto | Link (current version only) |
| ServerCat | 1.18 | MacOS | Not RFC 6979 / k_proto | Link (current version only) |
| SSH Term | 7.0.26 | MacOS | Not RFC 6979 / k_proto | Link (current version only) |
| Tectia SSH | 6.6.3.490 | Windows | Random | Link (current version only / trial) |
| Tera Term | 5.2 | Windows | Random | Link |
| Termius | 9.8.5 | Linux | Random | Link (current version only) |
| Termius | 9.21.2 | MacOS | Random | Link (current version only) |
| Win32 OpenSSH | 9.5.0.0 | Windows | Random | Link |
| WinSCP | 6.3.4 | Windows | RFC 6979 | Link |
| XShell 7 | 0170 | Windows | Random | Link (current version only / trial) |
| Agent Name | Version | OS | Nonce Scheme | Download Link |
|---|---|---|---|---|
| 1Password | 8.10.56 | Linux | No (EC)DSA support | Link (current version only / trial) |
| GnuPG | 2.4.4 | Linux | RFC 6979 | Link |
| Goldwarden | 0.3.6 | Linux | Random | Link |
| KeeAgent | 0.13.8 | Linux | Random | Link |
| MobaXTerm | 24.4 | Windows | RFC 6979 | Link (current version only) |
| OpenSSH | 9.6p1 | Linux | Random | Link |
| PKIX-SSH | 15.3 | Linux | Random | Link |
| PuTTY Pageant | 0.80 | Windows | k_proto | Link |
| PuTTY Pageant | 0.81 | Windows | RFC 6979 | Link |
| SecureCRT | 9.5.2 | Windows | Random | Link (current version only) |
| Tectia SSH | 6.6.3.490 | Windows | Random | Link (current version only / trial) |
| Termius | 9.9.0 | Linux | Random | Link (current version only) |
| Win32 OpenSSH | 9.5.0.0 | Windows | Random | Link |
| XShell 7 Xagent | 0170 | Windows | Random | Link (current version only / trial) |
Note
- Estimated time required: 6 hours
- Interaction required: no
- Proves claim: C5 by comparison between the output of the script and Figure 5 in the paper
To benchmark the success rate of the PuTTY vulnerability, run
scripts/05_bench_biased_nonce.sh. This will invoke the ecdsa_cli.py script
from the bdd-predicate repository with
the PuTTY-specific parameters, and with a varying number of available signatures
(ranging from 56 to 64 signatures).
Depending on the number of available CPU cores, each run may take significant
time to complete. With 16 cores on a desktop machine, each run may take around
15-20 minutes to complete. You can run
tail -f results/putty_attack/bdd-sieve_pred-$i.out (where $i is the number of
available signatures m in the current run) to monitor the progress of each
benchmark.
After completing the benchmarks, the success rate for each run will be printed to stdout. The data points should closely resemble the ones reported in the paper in Figure 5.
.
├── code
│ ├── env_docker # Dockerfiles for creating the Elasticsearch infrastructure used for evalutaion
│ ├── key_scraper # A tool written in Go that can collect SSH public keys from GitHub, GitLab, and Launchpad
| | └── scripts # Scripts for evaluating a dataset of SSH public keys gathered with the key_scraper tool
│ ├── nonce_sampler # A tool written in Go that can be used to analyze the determinism and bias of SSH client nonces
│ └── rsa_factorability_tool # Python implementation for an optimized batch-gcd algorithm used to find common factors
├── data
│ ├── key_scraper_results # Experimental results of the key_scraper used in the paper (excluding raw data)
│ └── putty_attack # Experimental results with regard to the PuTTY vulnerability
├── scripts
│ ├── 00_setup_env.sh # Script for setting up a fresh evaluation environment with all dependencies installed
│ ├── 01_run_scraper.sh # Runs the key_scraper tool on GitHub, Gitlab, and Launchpad for 24 hours
│ ├── 02_evaluate_keys.sh # Performs a full run of the evaluation pipeline on the keys collected by the key_scraper tool
│ ├── 03_generate_test_keys.sh # Generates test keys for testing public key upload restrictions
│ ├── 04_measure_client_agent.sh # Runs the nonce_sampler tool to measure client nonce determinism and bias
│ └── 05_bench_biased_nonce.sh # Benchmarks the success rate of the PuTTY vulnerability as described in the paper
├── supp_material # Contains additional material that did not make it into the final version of the paper but may be of interest
└── README.md
We use a variety of third party libraries and tools for the tools and scripts contained in this repository.
- GraphQL Client - github.com/Khan/genqlient
- Golang Elasticsearch Library - github.com/elastic/go-elasticsearch
- Scheduling Library - github.com/reugn/go-quartz
- Configuration Library - github.com/spf13/viper
- GraphQL Parser - github.com/vektah/gqlparser
- badkeys Tool - badkeys
- Cryptographic Library - cryptography
- Elliptic Curve Library - ECPy
- Python Elasticsearch Library - elasticsearch
- Multi-precision Arithmetic Library - gmpy2
- Plotting Library - matplotlib
- Multiprocessing Library - mpire
- NumPy - numpy
- ROCA Detection Library - roca-detect
- Progress Bar - tqdm
- badkeys Tool - badkeys
- Cryptographic Library - cryptography
- WSGI Web Application Framework - Flask
- Multi-precision Arithmetic Library - gmpy2
- Python MongoDB Library - pymongo
- WSGI Server - waitress
- Integer Factorization - primefac
- Progress Bar - tqdm
- Python Elasticsearch Library - elasticsearch
- Implementation of the Edwards25519 Curve - filippo.io/edwards25519
- Colorized Output - github.com/fatih/color
- Tabular Output - github.com/rodaine/table
- CLI Interface - github.com/urfave/cli
- Queue Implementation - go.linecorp.com/garr
- Cryptographic Library / SSH Implementation - golang.org/x/crypto (modified)