Skip to content

Reimplement REST interface using HTTP/2, HTTP3 #371

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 24, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -108,6 +108,8 @@ else()
option(OPENCMW_ENABLE_COVERAGE "Enable Coverage" OFF)
endif()
option(OPENCMW_ENABLE_CONCEPTS "Enable Concepts Builds" ${opencmw_MASTER_PROJECT})
option(OPENCMW_DEBUG_HTTP "Enable verbose HTTP output for debugging" OFF)
option(OPENCMW_PROFILE_HTTP "Enable verbose HTTP output for profiling" OFF)

# Very basic PCH example
option(ENABLE_PCH "Enable Precompiled Headers" OFF)
@@ -124,6 +126,14 @@ if(ENABLE_PCH)
<utility>)
endif()

if(OPENCMW_DEBUG_HTTP)
target_compile_definitions(opencmw_project_options INTERFACE -DOPENCMW_DEBUG_HTTP)
endif()

if(OPENCMW_PROFILE_HTTP)
target_compile_definitions(opencmw_project_options INTERFACE -DOPENCMW_PROFILE_HTTP)
endif()

if(OPENCMW_ENABLE_TESTING)
enable_testing()
message("Building Tests.")
83 changes: 70 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
@@ -29,7 +29,7 @@ where the frame-work takes care of most of the communication, [data-serialisatio
and buffering, settings management, Role-Based-Access-Control (RBAC), and other boring but necessary control system integrations
while still being open to expert-level modifications, extensions or improvements.

### General Schematic
## General Schematic

OpenCMW combines [ZeroMQ](https://zeromq.org/)'s [Majordomo](https://rfc.zeromq.org/spec/7/) with LMAX's [disruptor](https://lmax-exchange.github.io/disruptor/)
([C++ port](https://github.com/Abc-Arbitrage/Disruptor-cpp)) design pattern that both provide a very efficient lock-free mechanisms
@@ -38,7 +38,7 @@ for distributing, streaming and processing of data objects. A schematic outline

![OpenCMW architectural schematic](./assets/FAIR_microservice_schematic.svg)

### Glossary
## Glossary

_Majordomo Broker_ or _'Broker':_ is the central authority where multiple workers can register their services, allowing clients to perform get, set or subscriptions requests.
There can be multiple brokers for subset of services.
@@ -67,11 +67,11 @@ _Publisher:_ the [DataSourcePublisher](DataSourceExample.cpp) provides an interf
ring-buffer with events from OpenCMW, REST services or other sources.
While using disruptor ring-buffers is the preferred and most performing options, the client also supports classic patterns of registering call-back functions or returning `Future<reyly objects>` objects.

### OpenCMW Majordomo Protocol
## OpenCMW Majordomo Protocol

The OpenCMW Majordomo [protocol](docs/MajordomoProtocol.md) is based on the [ZeroMQ Majordomo protocol](https://rfc.zeromq.org/spec/7/), both extending and slightly modifying it (see [the comparison](docs/Majordomo_protocol_comparison.pdf)).

#### Service Names
### Service Names

Service names must always start with `/`. For consistency, this also applies to the built-in MDP broker services like `/mmi.service` (instead of `mmi.service` without leading slash as in ZeroMQ Majordomo). A service name is a non-empty alphanumerical string (also allowing `.`, `_`), that must start with `/` but not end with `/`. It contain additional `/` to denote a hierarchical structure.

@@ -84,7 +84,7 @@ Examples:
- `/DeviceName/Acquisition/` - invalid (trailing slash)
- `/a-service/` - invalid (`-` not allowed)

#### Topics
### Topics

The "topic" field (frame 5 in the [OpenCMW MDP protocol](docs/Majordomo_protocol_comparison.pdf)) specifies the topic for subscriptions and GET/SET requests. It contains a URI with the service name as path and optional query parameters to specify further requests parameters and filter criteria.

@@ -99,7 +99,7 @@ Note that the whole path is considered the service name, and that there's no add

See also the documentation for [mdp::Topic](src/core/include/Topic.hpp).

#### URL to Service/Topic Mapping (mds/mdp and REST)
### URL to Service/Topic Mapping (mds/mdp and REST)

With both the MDS/MDP-based ZeroMQ clients as well as the REST interface, a common scheme is used to map from mdp/hds/http(s) URLs used for subscriptions and requests to the OpenCMW service name and topic fields.

@@ -120,7 +120,7 @@ Other examples are:
- `mds://example.com:8080/DeviceName/Acquisition?signal=test` => service name `/DeviceName/Acquisition`, topic `/DeviceName/Acquisition?signal=test` (subscription via mds).
- `mdp://example.com:8080/dashboards/dashboard1?what=header` => service name `/dashboards/dashboard1`, topic `/dashboards/dashboard1?what=header` (Request via mdp).

### Compile-Time-Reflection
## Compile-Time-Reflection

The serialisers are based on a [compile-time-reflection](docs/CompileTimeSerialiser.md) that efficiently transform domain-objects to and from the given wire-format (binary, JSON, ...).
Compile-time reflection will become part of [C++23](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p0592r4.html) as described by [David Sankel et al. , “C++ Extensions for Reflection”, ISO/IEC CD TS 23619, N4856](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/n4856.pdf).
@@ -142,7 +142,7 @@ provides also an optional light-weight `constexpr` annotation template wrapper `
that in turn can be used to (re-)generate and document the class definition (e.g. for other programming languages or projects that do not have the primary domain-object definition at hand)
or to generate a generic [OpenAPI](https://swagger.io/specification/) definition. More details can be found [here](docs/CompileTimeSerialiser.md).

### Building from source
## Building from source

Note that building from source is only required if you want to modify opencmw-cpp itself.

@@ -151,7 +151,7 @@ In that case, rather take a look at the project [opencmw-cpp-example](https://gi

For concrete build instructions, please check the [build instructions page](docs/BuildInstructions.md).

### Example
## Example

For an example on how to implement a simple, first service using opencmw-cpp, please take a look at the project [opencmw-cpp-example](https://github.com/alexxcons/opencmw-cpp-example).

@@ -217,7 +217,7 @@ or RESTful (HTTP)-based high-level protocols, or through a simple RESTful web-in
[comment]: <> (The basic HTML rendering is based on XXX template engine and can be customised. For more efficient, complex and cross-platform)
[comment]: <> (UI designs it is planned to allow embedding of WebAssembly-based &#40;[WASM]&#40;https://en.wikipedia.org/wiki/WebAssembly&#41;&#41; applications.)

### Performance
## Performance

The end-to-end transmission achieving roughly 10k messages per second for synchronous communications and
about 140k messages per second for asynchronous and or publish-subscribe style data acquisition (TCP link via locahost).
@@ -241,17 +241,74 @@ Your mileage may vary depending on the specific domain-object, processing logic,
but you can check and compare the results for your platform using the [RoundTripAndNotifyEvaluation](RoundTripAndNotifyEvaluation.cpp)
and/or [MdpImplementationBenchmark](MdpImplementationBenchmark.cpp) benchmarks.

### Documentation
## Testing HTTP/3

The openCMW REST interface supports both HTTP/2 and HTTP/3. When connecting from a browser, the browser typically first connects
via HTTP/2. HTTP/2 responses contain a "alt-svc" header that informs the browser about the availability of HTTP/3. The browser
then should switch to HTTP/3. Note that if HTTP/3 fails for any reason (certificates, server error etc.), the browser might remember
that and not try HTTP/3 again. How to reset this depends on the browser. What I did in Google Chrome

1. open an private tab, open the developer console, go to "Network", enable the "Protocol" column
2. connect to the service; verify in the developer console that HTTP/3 is used (Protocol should switch to`H3` after the first `H2` request)
3. To reset browser status, close *all* private tabs and open a new one.

### SSL Certificates

QUIC/HTTP/3 requires the use of TLS, unencrypted servers are not possible.

At least in Google Chrome, the TLS stack used for QUIC seems quite separate from the normal settings, and its much stricter than for HTTP/1/2. *If
anything goes wrong here, Google Chrome will silently stick with HTTP/2, or show you an error, if you're lucky*.

Caveats:

1. The certificate must be trusted inside the Chrome Certificate store (e.g. on Mac, trusting it in Keychain might silence the warning for HTTP1/2,
but HTTP/3 will still fail. Add the server's public key under chrome://certificate-manager/.
2. The hostname in the certificate must match what you connect via in the browser. "Works-everywhere self-signed certificates" I couldn't get to work.
What I did: Create a certificate for hostname `foobar`, edit `/etc/hosts` to resolve `foobar` to the test host's IP address. Enter e.g. `https://foobar:8080`
in the browser, instead of the IP address.
3. For HTTP/2, you can ignore Chrome's warning with "Proceed anyway" or similar. This does not make the QUIC stack trust the certificate, HTTP/3 will
not be used.
4. To get the certificate's fingerprint in base64:
```
openssl x509 -in /path/to/demo_public.crt -noout -pubkey \
| openssl pkey -pubin -outform DER \
| openssl dgst -sha256 -binary \
| openssl enc -base64
```
3. Start Chrome with these parameters to make Chrome trust the certificate
```
.../Google\ Chrome --enable-quic \
--origin-to-force-quic-on=<yourhost>:8080 \
--ignore-certificate-errors-spki-list=<base64fingerprint> \
--user-data-dir=/tmp/quic-test-profile \
--no-sandbox
```

For verbose logging (netlog can be viewed in the [https://netlog-viewer.appspot.com/#import](netlog viewer)), add
```
--enable-logging=stderr --v=3 \
--log-net-log=netlog.json --quic-version=h3
```

### curl

curl needs to be built with QUIC/HTTP/3 enabled, which is not the case at least on Ubuntu 24.04. I used the docker image `badouralix/curl-http3`:

```
docker run --rm badouralix/curl-http3 curl -k -vvvv --http3 https://<yourhost>:8080/loadTest?topic=1&intervalMs=40&payloadSize=4096&nUpdates=100&LongPollingIdx=Next"
```

## Documentation

.... more to follow.

### Don't like Cpp?
## Don't like Cpp?

For prototyping applications or services that do not interact with hardware-based systems, a Java-based
[OpenCMW](https://github.com/fair-acc/opencmw-java) twin-project is being developed which follows the same functional style
but takes advantage of more concise implementation and C++-based type safety.

### Acknowledgements
## Acknowledgements

The implementation heavily relies upon and re-uses time-tried and well-established concepts from [ZeroMQ](https://zeromq.org/)
(notably the [Majordomo](https://rfc.zeromq.org/spec/7/) communication pattern, see [Z-Guide](https://zguide.zeromq.org/docs/chapter4/#Service-Oriented-Reliable-Queuing-Majordomo-Pattern)
135 changes: 108 additions & 27 deletions cmake/DependenciesNative.cmake
Original file line number Diff line number Diff line change
@@ -1,32 +1,118 @@
# Build a static version of openssl to link into
include(ExternalProject)
include(GNUInstallDirs)

set(OPENSSL_C_FLAGS "-O3 -march=x86-64-v3" CACHE STRING "OpenSSL custom CFLAGS" FORCE)
set(OPENSSL_CXX_FLAGS "-O3 -march=x86-64-v3" CACHE STRING "OpenSSL custom CXXFLAGS" FORCE)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-march=x86-64-v3 should not be hard-coded...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was there before, so I left it. Should remove -march alltogether?

set(OPENSSL_INSTALL_DIR "${CMAKE_BINARY_DIR}/openssl-install")
add_library(OpenSSL::Crypto STATIC IMPORTED GLOBAL)
add_library(OpenSSL::SSL STATIC IMPORTED GLOBAL)
add_dependencies(OpenSSL::Crypto PUBLIC openssl-build)
add_dependencies(OpenSSL::SSL PUBLIC openssl-build)
set_target_properties(OpenSSL::Crypto PROPERTIES

# Build custom OpenSSL with QUIC support
ExternalProject_Add(OpenSslProject
GIT_REPOSITORY https://github.com/openssl/openssl.git
GIT_TAG openssl-3.5.0 # 3.5.0 required for server-side QUIC support
GIT_SHALLOW ON
BUILD_BYPRODUCTS ${OPENSSL_INSTALL_DIR}/lib64/libcrypto.a ${OPENSSL_INSTALL_DIR}/lib64/libssl.a
CONFIGURE_COMMAND COMMAND ./Configure CFLAGS=${OPENSSL_C_FLAGS} CXXFLAGS=${OPENSSL_CXX_FLAGS} no-shared no-tests --prefix=${OPENSSL_INSTALL_DIR} --openssldir=${OPENSSL_INSTALL_DIR} linux-x86_64
UPDATE_COMMAND ""
BUILD_COMMAND make -j
INSTALL_COMMAND make install_sw # only installs software components (no docs, etc)
BUILD_IN_SOURCE ON
)

add_library(openssl-crypto-static STATIC IMPORTED GLOBAL)
add_dependencies(openssl-crypto-static OpenSslProject)
set_target_properties(openssl-crypto-static PROPERTIES
IMPORTED_LOCATION "${OPENSSL_INSTALL_DIR}/lib64/libcrypto.a"
INTERFACE_INCLUDE_DIRECTORIES "${OPENSSL_INSTALL_DIR}/include"
)
set_target_properties(OpenSSL::SSL PROPERTIES

add_library(openssl-ssl-static STATIC IMPORTED GLOBAL)
add_dependencies(openssl-ssl-static OpenSslProject)
set_target_properties(openssl-ssl-static PROPERTIES
IMPORTED_LOCATION "${OPENSSL_INSTALL_DIR}/lib64/libssl.a"
INTERFACE_INCLUDE_DIRECTORIES "${OPENSSL_INSTALL_DIR}/include"
)
get_target_property(libcryptoa OpenSSL::Crypto IMPORTED_LOCATION)
get_target_property(libcryptoaloc OpenSSL::Crypto LOCATION)
set(OPENSSL_C_FLAGS "-O3 -march=x86-64-v3" CACHE STRING "OpenSSL custom CFLAGS" FORCE)
set(OPENSSL_CXX_FLAGS "-O3 -march=x86-64-v3" CACHE STRING "OpenSSL custom CXXFLAGS" FORCE)
add_custom_command(
OUTPUT ${OPENSSL_INSTALL_DIR}/lib64/libcrypto.a ${OPENSSL_INSTALL_DIR}/lib64/libssl.a
COMMAND ${FETCHCONTENT_BASE_DIR}/openssl-source-src/Configure CFLAGS=${OPENSSL_C_FLAGS} CXXFLAGS=${OPENSSL_CXX_FLAGS} no-shared no-tests --prefix=${OPENSSL_INSTALL_DIR} --openssldir=${OPENSSL_INSTALL_DIR} linux-x86_64
COMMAND make -j
COMMAND make install_sw # only installs software components (no docs, etc)
COMMENT "Build openssl as a static library"
WORKING_DIRECTORY ${FETCHCONTENT_BASE_DIR}/openssl-source-build

option(ENABLE_NGHTTP_DEBUG "Enable verbose nghttp2 debug output" OFF)

ExternalProject_Add(Nghttp2Project
GIT_REPOSITORY https://github.com/nghttp2/nghttp2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why via ExternalProject_Add(...) and not the cmake fetch API?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using FetchContent_declare caused several issues with clashing target names (IIRC all the projects have a target "examples", for example), and also the various flags that need to be set would at least be confusing when set globally in the parent project.
Installing static libs + headers using ExternalProject_Add seemed cleaner and less error-prone.

GIT_TAG v1.65.0
GIT_SHALLOW ON
BUILD_BYPRODUCTS ${CMAKE_BINARY_DIR}/nghttp2-install/${CMAKE_INSTALL_LIBDIR}/libnghttp2.a
UPDATE_COMMAND ""
CMAKE_ARGS
-DCMAKE_INSTALL_PREFIX:PATH=${CMAKE_BINARY_DIR}/nghttp2-install
-DCMAKE_BUILD_TYPE:STRING=${CMAKE_BUILD_TYPE}
-DENABLE_LIB_ONLY:BOOL=ON
-DENABLE_HTTP3:BOOL=OFF
-DENABLE_DEBUG:BOOL=${ENABLE_NGHTTP_DEBUG}
-DBUILD_STATIC_LIBS:BOOL=ON
-BUILD_SHARED_LIBS:BOOL=OFF
-DENABLE_DOC:BOOL=OFF
)

add_library(nghttp2-static STATIC IMPORTED GLOBAL)
set_target_properties(nghttp2-static PROPERTIES
IMPORTED_LOCATION "${CMAKE_BINARY_DIR}/nghttp2-install/${CMAKE_INSTALL_LIBDIR}/libnghttp2.a"
INTERFACE_INCLUDE_DIRECTORIES "${CMAKE_BINARY_DIR}/nghttp2-install/include"
)
add_custom_target(openssl-build ALL
DEPENDS ${OPENSSL_INSTALL_DIR}/lib64/libcrypto.a ${OPENSSL_INSTALL_DIR}/lib64/libssl.a
add_dependencies(nghttp2-static Nghttp2Project)

ExternalProject_Add(Nghttp3Project
GIT_REPOSITORY https://github.com/ngtcp2/nghttp3.git
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here....

GIT_TAG v1.10.1
GIT_SHALLOW ON
BUILD_BYPRODUCTS ${CMAKE_BINARY_DIR}/nghttp3-install/${CMAKE_INSTALL_LIBDIR}/libnghttp3.a
UPDATE_COMMAND ""
CMAKE_ARGS
-DCMAKE_INSTALL_PREFIX:PATH=${CMAKE_BINARY_DIR}/nghttp3-install
-DCMAKE_BUILD_TYPE:STRING=${CMAKE_BUILD_TYPE}
-DENABLE_LIB_ONLY:BOOL=ON
-DENABLE_DEBUG:BOOL=${ENABLE_NGHTTP_DEBUG}
-DBUILD_STATIC_LIBS:BOOL=ON
-DBUILD_SHARED_LIBS:BOOL=OFF
-DENABLE_DOC:BOOL=OFF
)

ExternalProject_Add(NgTcp2Project
GIT_REPOSITORY https://github.com/ngtcp2/ngtcp2.git
GIT_TAG v1.13.0
GIT_SHALLOW ON
PREFIX ${CMAKE_BINARY_DIR}/ngtcp2-install
BUILD_BYPRODUCTS ${CMAKE_BINARY_DIR}/ngtcp2-install/${CMAKE_INSTALL_LIBDIR}/libngtcp2.a ${CMAKE_BINARY_DIR}/ngtcp2-install/${CMAKE_INSTALL_LIBDIR}/libngtcp2_crypto_ossl.a
UPDATE_COMMAND ""
CMAKE_ARGS
-DOPENSSL_ROOT_DIR:PATH=${OPENSSL_INSTALL_DIR}
-DENABLE_OPENSSL:BOOL=ON
-DCMAKE_INSTALL_PREFIX:PATH=${CMAKE_BINARY_DIR}/ngtcp2-install
-DCMAKE_BUILD_TYPE:STRING=${CMAKE_BUILD_TYPE}
-DENABLE_LIB_ONLY:BOOL=ON
-DENABLE_DEBUG:BOOL=${ENABLE_NGHTTP_DEBUG}
-DBUILD_STATIC_LIBS:BOOL=ON
-DBUILD_SHARED_LIBS:BOOL=OFF
DEPENDS openssl-crypto-static openssl-ssl-static
)

add_library(ngtcp2-static STATIC IMPORTED GLOBAL)
set_target_properties(ngtcp2-static PROPERTIES
IMPORTED_LOCATION "${CMAKE_BINARY_DIR}/ngtcp2-install/${CMAKE_INSTALL_LIBDIR}/libngtcp2.a"
INTERFACE_INCLUDE_DIRECTORIES "${CMAKE_BINARY_DIR}/ngtcp2-install/include"
)
add_dependencies(ngtcp2-static NgTcp2Project)

add_library(ngtcp2-crypto-ossl-static STATIC IMPORTED GLOBAL)
set_target_properties(ngtcp2-crypto-ossl-static PROPERTIES
IMPORTED_LOCATION "${CMAKE_BINARY_DIR}/ngtcp2-install/${CMAKE_INSTALL_LIBDIR}/libngtcp2_crypto_ossl.a"
INTERFACE_INCLUDE_DIRECTORIES "${CMAKE_BINARY_DIR}/ngtcp2-install/include"
)
add_dependencies(ngtcp2-crypto-ossl-static NgTcp2Project)

add_library(nghttp3-static STATIC IMPORTED GLOBAL)
set_target_properties(nghttp3-static PROPERTIES
IMPORTED_LOCATION "${CMAKE_BINARY_DIR}/nghttp3-install/${CMAKE_INSTALL_LIBDIR}/libnghttp3.a"
INTERFACE_INCLUDE_DIRECTORIES "${CMAKE_BINARY_DIR}/nghttp3-install/include"
)
add_dependencies(nghttp3-static Nghttp3Project)

add_library(mustache INTERFACE)
target_include_directories(mustache INTERFACE ${CMAKE_CURRENT_SOURCE_DIR}/3rd_party/kainjow)
@@ -38,6 +124,7 @@ FetchContent_Declare(
GIT_TAG v4.3.5 # latest as of 2025-03-27
)
set(ZMQ_BUILD_TESTS OFF CACHE BOOL "Build the tests for ZeroMQ")

# suppress warnings for missing zeromq dependencies by disabling some features
set(WITH_TLS OFF CACHE BOOL "TLS support for ZeroMQ WebSockets")
set(BUILD_SHARED OFF CACHE BOOL "Build cmake shared library")
@@ -56,12 +143,6 @@ FetchContent_Declare(
GIT_TAG v1.2.12 # latest v1.2.12
)

FetchContent_Declare(
openssl-source
GIT_REPOSITORY https://github.com/openssl/openssl.git
GIT_TAG openssl-3.4.1
)

FetchContent_MakeAvailable(cpp-httplib zeromq openssl-source)
FetchContent_MakeAvailable(cpp-httplib zeromq)

list(APPEND CMAKE_MODULE_PATH ${catch2_SOURCE_DIR}/contrib) # replace contrib by extras for catch2 v3.x.x
17 changes: 7 additions & 10 deletions concepts/client/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,13 +1,10 @@
if(NOT EMSCRIPTEN)
add_executable(RestSubscription_example RestSubscription_example.cpp)
target_link_libraries(
RestSubscription_example
PRIVATE core
client
opencmw_project_options
opencmw_project_warnings
assets::rest)
endif()
add_executable(LoadTest_client LoadTest_client.cpp)
target_link_libraries(
LoadTest_client
PRIVATE core
client
opencmw_project_options
opencmw_project_warnings)

add_executable(RestSubscription_client RestSubscription_client.cpp)
target_link_libraries(
Loading