Skip to content

Share connection between curl requests#170

Merged
ejoerns merged 1 commit into
rauc:masterfrom
rvdgracht:master-share
Jul 8, 2025
Merged

Share connection between curl requests#170
ejoerns merged 1 commit into
rauc:masterfrom
rvdgracht:master-share

Conversation

@rvdgracht

Copy link
Copy Markdown
Contributor

Using a curl_share between curl requests allows sharing an open connection and re-use cached DNS, PSL and TLS session id.

This change allows performing multiple requests without having to do re-perform the (full) TLS handshake.

For reference, on a stm32mp151c with OPTEE + pkcs11 TA a full TLS handshake takes ~8 seconds. Mostly due to small pager pool (internal sram) available for OPTEE.

With this change a mTLS curl request take around 60ms after the initial connection has been established.

@rvdgracht

Copy link
Copy Markdown
Contributor Author

I have some concerns about sharing the connection (CURL_LOCK_DATA_CONNECT) between the mainloop (poller) and the download thread. The documentation states: "It is not supported to share connections between multiple concurrent threads." I did place locking on the shared data including CURL_LOCK_DATA_CONNECT though.
I have succesfully tested this. So the mainloop can poll for status while the download thread is streaming data over the
same connection. Maybe someone with more extensive libcurl knowledge can verify this.

@Bastian-Krause

Copy link
Copy Markdown
Member

I think inter-thread connection sharing as it's implemented here should be okay. Maybe we should add -fsanitize=thread here, so we stumble upon any issues:

- CFLAGS="-fsanitize=address -fsanitize=leak -g" LDFLAGS="-fsanitize=address -fsanitize=leak"

@Bastian-Krause Bastian-Krause added the enhancement New feature or request label Feb 15, 2024
@rvdgracht

Copy link
Copy Markdown
Contributor Author

Maybe we should add -fsanitize=thread here

Added.

@Bastian-Krause

Copy link
Copy Markdown
Member

Maybe we should add -fsanitize=thread here

Added.

TSAN works by inserting instrumentation into compiled code to detect race conditions. We use glib from Debian in CI, meaning those libraries are not built with TSAN instrumentation, so memory accesses cannot be tracked. This leads to false positives. So suggesting to enable the sanitizer was not a good idea after all.

I'll drop the commit.

Bastian-Krause
Bastian-Krause previously approved these changes Dec 16, 2024
@Bastian-Krause
Bastian-Krause dismissed their stale review December 18, 2024 10:58

Additional changes required.

@Bastian-Krause

Copy link
Copy Markdown
Member

I've changed the PR slightly:

  • use GMutexes instead of pthread_mutex_t, add mutex init/clear
  • moved curl_share_setopt() calls to set_default_curl_opts() guarded with g_once_init_enter()/g_once_init_leave()
  • add g_return_if_fail() for return value of curl_share_init()
  • added asserts in curl lock/unlock functions

Bastian-Krause
Bastian-Krause previously approved these changes Dec 18, 2024

@ejoerns ejoerns left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me overall. Just a few notes left about possible enhancements I see.

Comment thread src/hawkbit-client.c Outdated
Comment thread src/hawkbit-client.c
Comment thread src/hawkbit-client.c Outdated
Using a curl_share between curl requests allows sharing an open connection
and re-use cached DNS, PSL and TLS session id.

This change allows performing multiple requests without having to do
re-perform the (full) TLS handshake.

For reference, on a stm32mp151c with OPTEE + pkcs11 TA a full TLS
handshake takes ~8 seconds. Mostly due to small pager pool (internal sram)
available for OPTEE.

With this change a mTLS curl request take around 60ms after the initial
connection has been established.

Signed-off-by: Robin van der Gracht <robin@protonic.nl>
Signed-off-by: Bastian Krause <bst@pengutronix.de>
@ejoerns
ejoerns merged commit 375dc8e into rauc:master Jul 8, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants