Skip to content

Improve error handling in SecureAccept and SsecureConnect - #3468

Merged
josesimoes merged 7 commits into
nanoframework:mainfrom
josesimoes:fix-tls-error-parsing
Jul 15, 2026
Merged

Improve error handling in SecureAccept and SsecureConnect#3468
josesimoes merged 7 commits into
nanoframework:mainfrom
josesimoes:fix-tls-error-parsing

Conversation

@josesimoes

@josesimoes josesimoes commented Jul 14, 2026

Copy link
Copy Markdown
Member

Description

  • Now returning detailed SSL error and mbedTLS error code.
  • Update declarations and callers.
  • Update return checks to use the new codes.
  • Update enum declaration in System.Net.

Motivation and Context

How Has This Been Tested?

Screenshots

Types of changes

  • Improvement (non-breaking change that improves a feature, code or algorithm)
  • Bug fix (non-breaking change which fixes an issue with code or algorithm)
  • New feature (non-breaking change which adds functionality to code)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Config and build (change in the configuration and build system, has no impact on code or features)
  • Dev Containers (changes related with Dev Containers, has no impact on code or features)
  • Dependencies/declarations (update dependencies or assembly declarations and changes associated, has no impact on code or features)
  • Documentation (changes or updates in the documentation, has no impact on code or features)

Checklist

  • My code follows the code style of this project (only if there are changes in source code).
  • My changes require an update to the documentation (there are changes that require the docs website to be updated).
  • I have updated the documentation accordingly (the changes require an update on the docs in this repo).
  • I have read the CONTRIBUTING document.
  • I have tested everything locally and all new and existing tests passed (only if there are changes in source code).

@josesimoes josesimoes added the Area: Common libs Everything related with common libraries label Jul 14, 2026
@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: ASSERTIVE

Plan: Pro

Run ID: 95e18608-a89e-4db1-afad-6290f2cb89e8

📥 Commits

Reviewing files that changed from the base of the PR and between a5b5252 and eb9dddf.

📒 Files selected for processing (2)
  • src/PAL/COM/sockets/Sockets_debugger.cpp
  • src/PAL/COM/sockets/ssl/MbedTLS/ssl_connect_internal.cpp

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes
    • Improved TLS connection and acceptance error handling with structured handshake error codes.
    • Certificate verification failures now provide more specific cryptographic error details.
    • Invalid security contexts and hostname configuration failures are reported clearly.
    • SSL debug connections now correctly reflect handshake success or failure.
  • New Features
    • TLS handshake APIs now return SslError and expose the underlying mbedTLS error code for easier diagnostics.

Walkthrough

Changes

SSL handshake error propagation

Layer / File(s) Summary
SSL error and API contracts
src/PAL/Include/nanoPAL_Sockets.h, src/PAL/COM/sockets/ssl/ssl_functions.h, src/DeviceInterfaces/System.Net/sys_net_native.h
Adds handshake-specific SslError values and changes SSL handshake APIs to return SslError with an mbedTLS error output parameter.
MbedTLS handshake status mapping
src/PAL/COM/sockets/ssl/MbedTLS/*
Maps invalid contexts, hostname failures, certificate verification failures, and other handshake failures to structured statuses and error codes.
SSL API forwarding and stubs
src/PAL/COM/sockets/ssl/ssl.cpp, src/PAL/COM/sockets/ssl/ssl_stubs.cpp
Propagates the new API contract through native implementations and returns SslError_None from stubs.
Handshake consumers and exception mapping
src/DeviceInterfaces/System.Net/sys_net_native_System_Net_Security_SslNative.cpp, src/PAL/COM/sockets/Sockets_debugger.cpp
Uses SslError for handshake handling, maps failures to CLR or cryptographic exceptions, and handles uncategorized statuses explicitly.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant SecureConnect
  participant SSL_Connect
  participant MbedTLS
  participant ExceptionMapper
  SecureConnect->>SSL_Connect: start handshake with mbedtlsCode
  SSL_Connect->>MbedTLS: perform TLS handshake
  MbedTLS-->>SSL_Connect: handshake status and error code
  SSL_Connect-->>SecureConnect: return SslError
  SecureConnect->>ExceptionMapper: map non-success SslError
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title matches the TLS error-handling changes, though it contains a typo and is a bit long.
Description check ✅ Passed The description clearly matches the changes to SSL error codes, callers, and enum updates.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/PAL/COM/sockets/Sockets_debugger.cpp (1)

462-482: 🩺 Stability & Availability | 🔴 Critical | 🏗️ Heavy lift

Do not remove the non-blocking retry loop.

By removing the do/while loop that checked for blocking states (SOCK_EWOULDBLOCK / SOCK_TRY_AGAIN), the SSL handshake for the debugger will immediately fail and abruptly close the socket if it cannot complete in a single pass.

TLS handshakes require multiple round trips, so non-blocking sockets will invariably return a "would block" state. You must restore a retry loop to allow the handshake to continue, ensuring that blocking states are properly intercepted rather than treated as a fatal sslErr != SslError_None.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/PAL/COM/sockets/Sockets_debugger.cpp` around lines 462 - 482, Restore the
non-blocking retry loop around SSL_Connect in the debugger SSL handshake,
continuing while the result represents SOCK_EWOULDBLOCK or SOCK_TRY_AGAIN and
only treating other non-success results as fatal. Keep socket/context cleanup
after the retry loop and preserve m_usingSSL assignment only for a successful
handshake.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@src/DeviceInterfaces/System.Net/sys_net_native_System_Net_Security_SslNative.cpp`:
- Around line 187-190: Remove the stale sslErr < 0 check immediately following
WaitEvents in the retry loop, since sslErr is not updated there and the
condition can prematurely terminate retries. Preserve the loop’s existing retry
behavior for SOCK_EWOULDBLOCK and SOCK_TRY_AGAIN.

In `@src/PAL/COM/sockets/ssl/MbedTLS/ssl_accept_internal.cpp`:
- Line 30: Update the context initialization and sanity check in the SSL accept
function to validate both context and context->ssl before dereferencing either.
Return SslError_HandshakeBadContext when either pointer is NULL, then assign ssl
only after the combined check, matching the safe flow used by
ssl_connect_internal.cpp.
- Around line 51-59: Restore non-blocking mode before handshake failure returns:
in ssl_accept_internal.cpp lines 51-59 and ssl_connect_internal.cpp lines 58-66,
set nonblock to 1 and call SOCK_ioctl in each failure path, including
certificate verification failure, before returning the corresponding SslError.

---

Outside diff comments:
In `@src/PAL/COM/sockets/Sockets_debugger.cpp`:
- Around line 462-482: Restore the non-blocking retry loop around SSL_Connect in
the debugger SSL handshake, continuing while the result represents
SOCK_EWOULDBLOCK or SOCK_TRY_AGAIN and only treating other non-success results
as fatal. Keep socket/context cleanup after the retry loop and preserve
m_usingSSL assignment only for a successful handshake.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7b773199-86e7-4eba-9164-42b54a9e2834

📥 Commits

Reviewing files that changed from the base of the PR and between 5a06b11 and beeff95.

📒 Files selected for processing (9)
  • src/DeviceInterfaces/System.Net/sys_net_native.h
  • src/DeviceInterfaces/System.Net/sys_net_native_System_Net_Security_SslNative.cpp
  • src/PAL/COM/sockets/Sockets_debugger.cpp
  • src/PAL/COM/sockets/ssl/MbedTLS/ssl_accept_internal.cpp
  • src/PAL/COM/sockets/ssl/MbedTLS/ssl_connect_internal.cpp
  • src/PAL/COM/sockets/ssl/ssl.cpp
  • src/PAL/COM/sockets/ssl/ssl_functions.h
  • src/PAL/COM/sockets/ssl/ssl_stubs.cpp
  • src/PAL/Include/nanoPAL_Sockets.h

Comment thread src/DeviceInterfaces/System.Net/sys_net_native_System_Net_Security_SslNative.cpp Outdated
Comment thread src/PAL/COM/sockets/ssl/MbedTLS/ssl_accept_internal.cpp
Comment thread src/PAL/COM/sockets/ssl/MbedTLS/ssl_accept_internal.cpp

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@src/DeviceInterfaces/System.Net/sys_net_native_System_Net_Security_SslNative.cpp`:
- Around line 91-97: The blocking handshake paths no longer use timeout stack
state. In SecureAccept at
src/DeviceInterfaces/System.Net/sys_net_native_System_Net_Security_SslNative.cpp
lines 91-97, remove the timeout_ms/hbTimeout setup, the unused timeout handling,
and the Timeout stack pop; apply the same removal of timeout/hbTimeout setup,
unused timeout handling, and stack pop in SecureConnect at lines 164-170,
leaving the blocking SSL handshake calls intact.
- Around line 531-533: Update the SslError switch in the SSL initialization
function so the default branch explicitly returns failure instead of breaking
and continuing to stack.SetResult_I4(sslContext). Preserve the existing handling
for recognized SslError values and ensure unrecognized or future errors cannot
return the invalid SSL context as success.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: ASSERTIVE

Plan: Pro

Run ID: a5099a69-baec-4cbb-9f1c-2820bf7b6e0c

📥 Commits

Reviewing files that changed from the base of the PR and between beeff95 and 92a7d89.

📒 Files selected for processing (2)
  • src/DeviceInterfaces/System.Net/sys_net_native_System_Net_Security_SslNative.cpp
  • src/PAL/COM/sockets/ssl/MbedTLS/ssl_accept_internal.cpp

- Now returning detailed SSL error and mbedTLS error code.
- Update declarations and callers.
- Update return checks to use the new codes.
- Update enum declaration in System.Net.
@josesimoes
josesimoes force-pushed the fix-tls-error-parsing branch from 17c0c2a to a5b5252 Compare July 15, 2026 11:06

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/PAL/COM/sockets/Sockets_debugger.cpp`:
- Around line 462-482: Remove the outer do/while retry loop and its accompanying
non-blocking socket comment around SSL_Connect. Invoke SSL_Connect once,
preserving mbedtlsCode and sslErr for the existing terminal error handling
because ssl_connect_internal now performs the handshake retries internally.

In `@src/PAL/COM/sockets/ssl/MbedTLS/ssl_connect_internal.cpp`:
- Around line 29-32: Extend the initial validation in ssl_connect_internal to
also check context->ssl for NULL before calling mbedtls_ssl_set_hostname. Return
SslError_HandshakeBadContext for either invalid context or SSL state, matching
the safety behavior of ssl_accept_internal.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: ASSERTIVE

Plan: Pro

Run ID: 930339ea-d878-4aa3-85fe-6a94a303f6b7

📥 Commits

Reviewing files that changed from the base of the PR and between 92a7d89 and a5b5252.

📒 Files selected for processing (9)
  • src/DeviceInterfaces/System.Net/sys_net_native.h
  • src/DeviceInterfaces/System.Net/sys_net_native_System_Net_Security_SslNative.cpp
  • src/PAL/COM/sockets/Sockets_debugger.cpp
  • src/PAL/COM/sockets/ssl/MbedTLS/ssl_accept_internal.cpp
  • src/PAL/COM/sockets/ssl/MbedTLS/ssl_connect_internal.cpp
  • src/PAL/COM/sockets/ssl/ssl.cpp
  • src/PAL/COM/sockets/ssl/ssl_functions.h
  • src/PAL/COM/sockets/ssl/ssl_stubs.cpp
  • src/PAL/Include/nanoPAL_Sockets.h

Comment thread src/PAL/COM/sockets/Sockets_debugger.cpp
Comment thread src/PAL/COM/sockets/ssl/MbedTLS/ssl_connect_internal.cpp Outdated
@josesimoes
josesimoes merged commit 51945c3 into nanoframework:main Jul 15, 2026
40 checks passed
@josesimoes
josesimoes deleted the fix-tls-error-parsing branch July 15, 2026 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: Common libs Everything related with common libraries Type: enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants