Skip to content

bundle of features #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
keldonin wants to merge 15 commits into
base: main
Choose a base branch
from
Open

bundle of features #21

Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
528bc74
feat: replacing boost::chrono by std::steady_clock and std::high_reso…
keldonin Dec 17, 2025
4c84726
feat: adding support for quantiles (p95,p98,p99)
keldonin Dec 17, 2025
9cbece0
feat: support for datapoints - normality test less sensitive to outli…
keldonin Dec 1, 2025
e4ccc63
feat: test case for RSA PSS
keldonin Dec 1, 2025
63dc78a
feat: when a key generation fails, move on and skip depending test ca…
keldonin Dec 4, 2025
6221601
feat: randomized input for seedrandom test
keldonin Dec 4, 2025
5d83064
feat: improve graph generation with enhanced grid and axis management
keldonin Dec 4, 2025
9294620
feat: add test case for measuring performance of finding objects
keldonin Dec 9, 2025
98e6824
fix: replace ctgmath with cmath and ensure consistent use of std::abs…
keldonin Dec 17, 2025
78eee3e
adapted changelog and version numbering
keldonin Dec 17, 2025
226a9ea
fixed incorrect standard deviation calculus reported by Copilot
keldonin Dec 17, 2025
25b1f49
gengraphs.py: removed unnecessary format methods in string parsing
keldonin Dec 17, 2025
94963ff
fix: change label for find objects case, to avoid gengraphs file gene…
keldonin Dec 18, 2025
63af2ba
fixes following PR review + adjusted cache size for quantile estimations
keldonin Jan 6, 2026
cb3c240
fixed issue with gengraph.py broken for producing compared curves
keldonin Jan 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,27 @@ All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
## UNRELEASED
### Added
- Docker buildx recipes and scripts
- `uv`support in Docker builds to improve reproducibility and speed of building
- log-normal distribution
- quantiles (p95,p98,p99)
- Lilliefors fitness test against both normal and log-normal distributions
- data points can now be recorded with new option (`-d`); data points can be extracted to a spreadsheet using the new utility `datapoints2xlsx`
- adds support for RSA PSS signature testing
- adds support for `C_FindObjects()` testing

### Changed
- using high resolution clocks and steady clocks from standard library, to improve precision of measurement in all environments (including VMs)
- introduces the use of units for automatic conversion and improved readability
- graphs improvements: latency percentiles have now a scale distinct from average, coloured curves and axis, coloured horizontal lines
- flow: when a key generation fails, the corresponding test cases are removed from the test cases, and the application no longer terminates
- `C_SeedRandom()` test case uses random numbers for the seed

### Fixed
- compilation warning under CLANG removed (using `std::abs` instead of `abs`)
- updated python dependencies to fix reported vulnerabilities

## 3.15.1 - 2025-11-26
### Fixed
Expand Down
2 changes: 1 addition & 1 deletion P11PERFTEST_VERSION
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
3.15.1
3.16.0-BETA
5 changes: 3 additions & 2 deletions buildx/Dockerfile.alpine321
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,8 @@ RUN uv venv
RUN uv pip install -r requirements.txt \
&& uv run pyinstaller scripts/json2xlsx.spec \
&& uv run pyinstaller scripts/gengraphs.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs
&& uv run pyinstaller scripts/datapoints2xlsx.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /tar_build/usr/local/share/doc/p11perftest \
Expand All @@ -157,7 +158,7 @@ RUN make distclean \
&& make install-strip DESTDIR=/apk_build

# Build the python executables using pyinstaller
RUN install -s -m 755 -t /apk_build/usr/bin dist/json2xlsx dist/gengraphs
RUN install -s -m 755 -t /apk_build/usr/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /apk_build/usr/share/doc/p11perftest \
Expand Down
5 changes: 3 additions & 2 deletions buildx/Dockerfile.amzn2023
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,8 @@ RUN uv venv
RUN uv pip install -r requirements.txt \
&& uv run pyinstaller scripts/json2xlsx.spec \
&& uv run pyinstaller scripts/gengraphs.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs
&& uv run pyinstaller scripts/datapoints2xlsx.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /tar_build/usr/local/share/doc/p11perftest \
Expand All @@ -147,7 +148,7 @@ RUN make distclean \
&& make install-strip DESTDIR=/rpm_build

# Build the python executables using pyinstaller
RUN install -s -m 755 -t /rpm_build/usr/bin dist/json2xlsx dist/gengraphs
RUN install -s -m 755 -t /rpm_build/usr/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /rpm_build/usr/share/doc/p11perftest \
Expand Down
8 changes: 4 additions & 4 deletions buildx/Dockerfile.deb12
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ RUN apt-get update && apt-get install -y \
libboost-dev \
libboost-program-options-dev \
libboost-system-dev \
libboost-timer-dev \
libssl-dev \
&& rm -rf /var/lib/apt/lists/*

Expand Down Expand Up @@ -129,7 +128,8 @@ RUN uv venv
RUN uv pip install -r requirements.txt \
&& uv run pyinstaller scripts/json2xlsx.spec \
&& uv run pyinstaller scripts/gengraphs.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs
&& uv run pyinstaller scripts/datapoints2xlsx.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /tar_build/usr/local/share/doc/p11perftest \
Expand All @@ -142,7 +142,7 @@ RUN make distclean \
&& make install-strip DESTDIR=/deb_build

# Build the python executables using pyinstaller
RUN install -s -m 755 -t /deb_build/usr/bin dist/json2xlsx dist/gengraphs
RUN install -s -m 755 -t /deb_build/usr/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /deb_build/usr/share/doc/p11perftest \
Expand Down Expand Up @@ -174,7 +174,7 @@ RUN . /meta/env \
&& echo "Section: misc" >> DEBIAN/control \
&& echo "Priority: optional" >> DEBIAN/control \
&& echo "Architecture: $PKG_ARCH" >> DEBIAN/control \
&& echo "Depends: libc6,libboost-chrono1.74.0,libboost-timer1.74.0,libboost-program-options1.74.0" >> DEBIAN/control \
&& echo "Depends: libc6,libboost-program-options1.74.0" >> DEBIAN/control \
&& echo "Maintainer: $PKG_MAINTAINER" >> DEBIAN/control \
&& echo "Description: A utility for testing PKCS#11 implementations" >> DEBIAN/control

Expand Down
8 changes: 4 additions & 4 deletions buildx/Dockerfile.deb13
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ RUN apt-get update && apt-get install -y \
libboost-dev \
libboost-program-options-dev \
libboost-system-dev \
libboost-timer-dev \
libssl-dev \
&& rm -rf /var/lib/apt/lists/*

Expand Down Expand Up @@ -129,7 +128,8 @@ RUN uv venv
RUN uv pip install -r requirements.txt \
&& uv run pyinstaller scripts/json2xlsx.spec \
&& uv run pyinstaller scripts/gengraphs.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs
&& uv run pyinstaller scripts/datapoints2xlsx.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /tar_build/usr/local/share/doc/p11perftest \
Expand All @@ -142,7 +142,7 @@ RUN make distclean \
&& make install-strip DESTDIR=/deb_build

# Build the python executables using pyinstaller
RUN install -s -m 755 -t /deb_build/usr/bin dist/json2xlsx dist/gengraphs
RUN install -s -m 755 -t /deb_build/usr/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /deb_build/usr/share/doc/p11perftest \
Expand Down Expand Up @@ -174,7 +174,7 @@ RUN . /meta/env \
&& echo "Section: misc" >> DEBIAN/control \
&& echo "Priority: optional" >> DEBIAN/control \
&& echo "Architecture: $PKG_ARCH" >> DEBIAN/control \
&& echo "Depends: libc6,libboost-chrono1.83.0,libboost-timer1.83.0,libboost-program-options1.83.0" >> DEBIAN/control \
&& echo "Depends: libc6,libboost-program-options1.83.0" >> DEBIAN/control \
&& echo "Maintainer: $PKG_MAINTAINER" >> DEBIAN/control \
&& echo "Description: A utility for testing PKCS#11 implementations" >> DEBIAN/control

Expand Down
5 changes: 3 additions & 2 deletions buildx/Dockerfile.ol8
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,8 @@ RUN uv venv
RUN uv pip install -r requirements.txt \
&& uv run pyinstaller scripts/json2xlsx.spec \
&& uv run pyinstaller scripts/gengraphs.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs
&& uv run pyinstaller scripts/datapoints2xlsx.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /tar_build/usr/local/share/doc/p11perftest \
Expand All @@ -150,7 +151,7 @@ RUN make distclean \
&& make install-strip DESTDIR=/rpm_build

# Build the python executables using pyinstaller
RUN install -s -m 755 -t /rpm_build/usr/bin dist/json2xlsx dist/gengraphs
RUN install -s -m 755 -t /rpm_build/usr/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /rpm_build/usr/share/doc/p11perftest \
Expand Down
5 changes: 3 additions & 2 deletions buildx/Dockerfile.ol9
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,8 @@ RUN uv venv
RUN uv pip install -r requirements.txt \
&& uv run pyinstaller scripts/json2xlsx.spec \
&& uv run pyinstaller scripts/gengraphs.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs
&& uv run pyinstaller scripts/datapoints2xlsx.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /tar_build/usr/local/share/doc/p11perftest \
Expand All @@ -151,7 +152,7 @@ RUN make distclean \
&& make install-strip DESTDIR=/rpm_build

# Build the python executables using pyinstaller
RUN install -s -m 755 -t /rpm_build/usr/bin dist/json2xlsx dist/gengraphs
RUN install -s -m 755 -t /rpm_build/usr/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /rpm_build/usr/share/doc/p11perftest \
Expand Down
8 changes: 4 additions & 4 deletions buildx/Dockerfile.ubuntu2204
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ RUN apt-get update && apt-get install -y \
libboost-dev \
libboost-program-options-dev \
libboost-system-dev \
libboost-timer-dev \
libssl-dev \
&& rm -rf /var/lib/apt/lists/*

Expand Down Expand Up @@ -130,7 +129,8 @@ RUN uv venv
RUN uv pip install -r requirements.txt \
&& uv run pyinstaller scripts/json2xlsx.spec \
&& uv run pyinstaller scripts/gengraphs.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs
&& uv run pyinstaller scripts/datapoints2xlsx.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /tar_build/usr/local/share/doc/p11perftest \
Expand All @@ -143,7 +143,7 @@ RUN make distclean \
&& make install-strip DESTDIR=/deb_build

# Build the python executables using pyinstaller
RUN install -s -m 755 -t /deb_build/usr/bin dist/json2xlsx dist/gengraphs
RUN install -s -m 755 -t /deb_build/usr/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /deb_build/usr/share/doc/p11perftest \
Expand Down Expand Up @@ -175,7 +175,7 @@ RUN . /meta/env \
&& echo "Section: misc" >> DEBIAN/control \
&& echo "Priority: optional" >> DEBIAN/control \
&& echo "Architecture: $PKG_ARCH" >> DEBIAN/control \
&& echo "Depends: libc6,libboost-chrono1.74.0,libboost-timer1.74.0,libboost-program-options1.74.0" >> DEBIAN/control \
&& echo "Depends: libc6,libboost-program-options1.74.0" >> DEBIAN/control \
&& echo "Maintainer: $PKG_MAINTAINER" >> DEBIAN/control \
&& echo "Description: A utility for testing PKCS#11 implementations" >> DEBIAN/control

Expand Down
8 changes: 4 additions & 4 deletions buildx/Dockerfile.ubuntu2404
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ RUN apt-get update && apt-get install -y \
libboost-dev \
libboost-program-options-dev \
libboost-system-dev \
libboost-timer-dev \
libssl-dev \
&& rm -rf /var/lib/apt/lists/*

Expand Down Expand Up @@ -133,7 +132,8 @@ RUN uv venv
RUN uv pip install -r requirements.txt \
&& uv run pyinstaller scripts/json2xlsx.spec \
&& uv run pyinstaller scripts/gengraphs.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs
&& uv run pyinstaller scripts/datapoints2xlsx.spec \
&& install -s -m 755 -t /tar_build/usr/local/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /tar_build/usr/local/share/doc/p11perftest \
Expand All @@ -146,7 +146,7 @@ RUN make distclean \
&& make install-strip DESTDIR=/deb_build

# Build the python executables using pyinstaller
RUN install -s -m 755 -t /deb_build/usr/bin dist/json2xlsx dist/gengraphs
RUN install -s -m 755 -t /deb_build/usr/bin dist/json2xlsx dist/gengraphs dist/datapoints2xlsx

# Install documentation
RUN mkdir -p /deb_build/usr/share/doc/p11perftest \
Expand Down Expand Up @@ -178,7 +178,7 @@ RUN . /meta/env \
&& echo "Section: misc" >> DEBIAN/control \
&& echo "Priority: optional" >> DEBIAN/control \
&& echo "Architecture: $PKG_ARCH" >> DEBIAN/control \
&& echo "Depends: libc6,libboost-chrono1.83.0,libboost-timer1.83.0,libboost-program-options1.83.0" >> DEBIAN/control \
&& echo "Depends: libc6,libboost-program-options1.83.0" >> DEBIAN/control \
&& echo "Maintainer: $PKG_MAINTAINER" >> DEBIAN/control \
&& echo "Description: A utility for testing PKCS#11 implementations" >> DEBIAN/control

Expand Down
4 changes: 1 addition & 3 deletions configure.ac
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,10 +63,8 @@ AC_CONFIG_HEADERS([config.h])
dnl Check for libraries, headers, data etc here.
AC_SEARCH_LIBS([dlopen], [dl dld], [], [AC_MSG_FAILURE([can't find dynamic linker lib])])
AX_PTHREAD(,[AC_MSG_ERROR[pthread is required to compile this project]])
AX_BOOST_BASE([1.62],, [AC_MSG_ERROR([p11perftest needs Boost, but it was not found in your system])])
AX_BOOST_BASE([1.66],, [AC_MSG_ERROR([p11perftest needs Boost, but it was not found in your system])])
AX_BOOST_PROGRAM_OPTIONS()
AX_BOOST_TIMER()
AX_BOOST_CHRONO()

PKG_CHECK_MODULES([BOTAN], [ botan-2 > 2.17.0 ])
PKG_CHECK_MODULES([LIBCRYPTO], [ libcrypto > 1 ])
Expand Down
5 changes: 3 additions & 2 deletions scripts/Makefile.am
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@

ACLOCAL_AMFLAGS = -I m4

bin_SCRIPTS = createkeys.sh generatekeys.py json2xlsx.py
bin_SCRIPTS = createkeys.sh generatekeys.py json2xlsx.py datapoints2xlsx.py

EXTRA_DIST = createkeys.sh generatekeys.py json2xlsx.py
EXTRA_DIST = createkeys.sh generatekeys.py json2xlsx.py datapoints2xlsx.py \
generatekeys.spec json2xlsx.spec datapoints2xlsx.spec
Loading