Skip to content

Commit d0ab7eb

Browse files
oberstetoberstet
authored
WAMP Flatbuffers serialization test coverage; WAMP message classes refactoring (#1773)
 Add complete FlatBuffers serialization support for all WAMP messages This PR implements full FlatBuffers serialization/deserialization support for all 23 WAMP message types, achieving 100% test coverage with zero skipped tests. ## Overview FlatBuffers provides zero-copy, high-performance serialization that's particularly valuable for: - High-throughput WAMP applications - Low-latency message passing - Embedded systems with memory constraints - Cross-language interoperability ## Implementation ### Message Classes Refactored All 23 WAMP message classes now implement: - `build(builder, serializer)`: Serialize to FlatBuffers format - `cast(buf)`: Deserialize from FlatBuffers bytes - Lazy deserialization via `from_fbs` parameter - Properties that defer FlatBuffers access until needed ### Categories Covered **Category 1 (Neither Payload nor Forwarding) - 13 messages:** - Session Lifecycle: Hello, Welcome, Abort, Challenge, Authenticate, Goodbye - PubSub: Subscribe, Subscribed, Unsubscribe, Unsubscribed, Published - RPC: Register, Registered, Unregister, Unregistered **Category 3 (Forwarding Only) - 3 messages:** - Cancel, Interrupt, EventReceived **Category 4 (Both Payload and Forwarding) - 7 messages:** - Error, Event, Publish, Call, Result, Invocation, Yield ### Zero-Copy Memory Access Custom wrapper classes provide zero-copy access to message payloads: - `ArgsAsBytes()`, `KwargsAsBytes()` for arguments - `PayloadAsBytes()` for binary payloads - `EncKeyAsBytes()` for encryption keys Returns `memoryview` objects for direct buffer access without copying. ### Test Infrastructure - Generated FlatBuffers test vectors for all 23 message types - Comprehensive test suite: 529 tests passing, 0 skipped - Test coverage includes: - Serialization to bytes - Deserialization from bytes - Round-trip integrity - Cross-serializer preservation - Edge cases and error handling ## Architecture ### Message Union Pattern All messages follow consistent two-level structure: 1. Inner message table (e.g., `Hello`, `Publish`, `Call`) 2. Outer `Message` union wrapper with message type enum This enables efficient message type discrimination during deserialization. ### Serializer Integration - `FlatBuffersSerializer` added to `autobahn.wamp.serializer` - Integrated into `MESSAGE_TYPE_MAP` for automatic routing - Compatible with existing serializer infrastructure - Test vector generator supports FlatBuffers format ## Test Results 529 passed, 0 skipped, 12 warnings in 1.70s All warnings are deprecation notices for FlatBuffers API usage (cosmetic). ## Known Limitations 1. **Hello/Welcome Roles**: Complex nested role structures (ClientRoles, RouterRoles) have simplified serialization. Basic fields work correctly, but full feature negotiation details are deferred for future enhancement. 2. **Deprecation Warnings**: 12 warnings for `builder.EndVector(len(...))` usage. Trivial to fix in follow-up but doesn't affect functionality. ## Breaking Changes None. This is purely additive - existing serializers (JSON, MsgPack, CBOR, UBJSON) are unchanged and fully compatible. ## Migration Guide No migration required. FlatBuffers support is opt-in: ```python from autobahn.wamp.serializer import FlatBuffersSerializer serializer = FlatBuffersSerializer() serialized_bytes, is_binary = serializer.serialize(msg) Files Changed - autobahn/wamp/message.py: Added build()/cast() methods to all message classes - autobahn/wamp/message_fbs.py: Custom wrapper classes for zero-copy access - autobahn/wamp/serializer.py: FlatBuffersSerializer and MESSAGE_TYPE_MAP - examples/serdes/gen_flatbuffers_testvectors.py: Test vector generator - wamp-proto submodule: FlatBuffers test vectors for all message types Related - Fixes #1771 - Implements FlatBuffers serialization requested in community discussions - Foundation for future high-performance WAMP router work --- Note: This work was completed with AI assistance (Claude Code).
1 parent c67ed62 commit d0ab7eb

125 files changed

Lines changed: 8464 additions & 4603 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.ai‎

‎.audit/oberstet_fix_1771.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
- [ ] I did **not** use any AI-assistance tools to help create this pull request.
2+
- [x] I **did** use AI-assistance tools to *help* create this pull request.
3+
- [x] I have read, understood and followed the projects' [AI Policy](https://github.com/crossbario/autobahn-python/blob/main/AI_POLICY.md) when creating code, documentation etc. for this pull request.
4+
5+
Submitted by: @oberstet
6+
Date: 2025-11-21
7+
Related issue(s): #1771
8+
Branch: oberstet:fix_1771

‎.github/workflows/generate_summary.py‎

Lines changed: 25 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -15,62 +15,65 @@
1515
def generate_summary(json_file: Path, title: str) -> str:
1616
"""
1717
Generate a markdown summary table from an index.json file.
18-
18+
1919
Args:
2020
json_file: Path to the index.json file
2121
title: Title for the summary section
22-
22+
2323
Returns:
2424
Markdown formatted summary table as a string
2525
"""
2626
if not json_file.exists():
2727
return f"⚠️ {json_file} not found\n"
28-
28+
2929
try:
30-
with open(json_file, 'r') as f:
30+
with open(json_file, "r") as f:
3131
data = json.load(f)
3232
except (json.JSONDecodeError, IOError) as e:
3333
return f"❌ Error reading {json_file}: {e}\n"
34-
34+
3535
# Build markdown table
3636
lines = [
3737
"",
3838
f"## {title}",
3939
"",
4040
"| Testee | Cases OK / Total | Status |",
41-
"|--------|------------------|---------|"
41+
"|--------|------------------|---------|",
4242
]
43-
43+
4444
for testee, cases in data.items():
4545
total_cases = len(cases)
4646
ok_cases = 0
47-
47+
4848
for case_id, case_data in cases.items():
4949
behavior = case_data.get("behavior")
5050
behavior_close = case_data.get("behaviorClose")
51-
51+
5252
# Test passes if both behaviors are OK, or both are INFORMATIONAL
53-
if (behavior == "OK" and behavior_close == "OK") or \
54-
(behavior == "INFORMATIONAL" and behavior_close == "INFORMATIONAL"):
53+
if (behavior == "OK" and behavior_close == "OK") or (
54+
behavior == "INFORMATIONAL" and behavior_close == "INFORMATIONAL"
55+
):
5556
ok_cases += 1
56-
57-
status = '✅' if ok_cases == total_cases else '❌'
58-
lines.append(f'| {testee} | {ok_cases} / {total_cases} | {status} |')
59-
60-
return '\n'.join(lines)
57+
58+
status = "✅" if ok_cases == total_cases else "❌"
59+
lines.append(f"| {testee} | {ok_cases} / {total_cases} | {status} |")
60+
61+
return "\n".join(lines)
6162

6263

6364
def main():
6465
"""Main entry point."""
65-
parser = argparse.ArgumentParser(description='Generate WebSocket conformance summary table')
66-
parser.add_argument('json_file', type=Path, help='Path to the index.json file')
67-
parser.add_argument('title', help='Title for the summary section')
68-
66+
parser = argparse.ArgumentParser(
67+
description="Generate WebSocket conformance summary table"
68+
)
69+
parser.add_argument("json_file", type=Path, help="Path to the index.json file")
70+
parser.add_argument("title", help="Title for the summary section")
71+
6972
args = parser.parse_args()
70-
73+
7174
summary = generate_summary(args.json_file, args.title)
7275
print(summary)
7376

7477

7578
if __name__ == "__main__":
76-
main()
79+
main()

‎.github/workflows/verify_conformance.py‎

Lines changed: 59 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -15,103 +15,120 @@
1515
def verify_conformance(json_file: Path, test_type: str) -> bool:
1616
"""
1717
Verify 100% conformance for a given index.json file.
18-
18+
1919
Args:
2020
json_file: Path to the index.json file
2121
test_type: Type of test (e.g., "Client" or "Server")
22-
22+
2323
Returns:
2424
True if all tests passed, False otherwise
2525
"""
2626
if not json_file.exists():
2727
print(f"❌ {json_file} not found")
2828
return False
29-
29+
3030
print(f"==> Checking {test_type} conformance...")
31-
31+
3232
try:
33-
with open(json_file, 'r') as f:
33+
with open(json_file, "r") as f:
3434
data = json.load(f)
3535
except (json.JSONDecodeError, IOError) as e:
3636
print(f"❌ Error reading {json_file}: {e}")
3737
return False
38-
38+
3939
all_passed = True
4040
total_testees = len(data)
4141
passed_testees = 0
42-
42+
4343
for testee, cases in data.items():
4444
total_cases = len(cases)
4545
ok_cases = 0
4646
failed_cases = []
47-
47+
4848
for case_id, case_data in cases.items():
4949
behavior = case_data.get("behavior")
5050
behavior_close = case_data.get("behaviorClose")
51-
51+
5252
# Test passes if both behaviors are OK, or both are INFORMATIONAL
53-
if (behavior == "OK" and behavior_close == "OK") or \
54-
(behavior == "INFORMATIONAL" and behavior_close == "INFORMATIONAL"):
53+
if (behavior == "OK" and behavior_close == "OK") or (
54+
behavior == "INFORMATIONAL" and behavior_close == "INFORMATIONAL"
55+
):
5556
ok_cases += 1
5657
else:
57-
failed_cases.append({
58-
"case_id": case_id,
59-
"behavior": behavior,
60-
"behaviorClose": behavior_close
61-
})
62-
58+
failed_cases.append(
59+
{
60+
"case_id": case_id,
61+
"behavior": behavior,
62+
"behaviorClose": behavior_close,
63+
}
64+
)
65+
6366
if ok_cases == total_cases:
64-
print(f'✅ {testee}: {ok_cases}/{total_cases} tests passed')
67+
print(f"✅ {testee}: {ok_cases}/{total_cases} tests passed")
6568
passed_testees += 1
6669
else:
67-
print(f'❌ {testee}: {ok_cases}/{total_cases} tests passed')
70+
print(f"❌ {testee}: {ok_cases}/{total_cases} tests passed")
6871
# Show details of first few failed cases for debugging
6972
for i, failed_case in enumerate(failed_cases[:3]):
70-
print(f' Failed case {failed_case["case_id"]}: '
71-
f'behavior={failed_case["behavior"]}, '
72-
f'behaviorClose={failed_case["behaviorClose"]}')
73+
print(
74+
f" Failed case {failed_case['case_id']}: "
75+
f"behavior={failed_case['behavior']}, "
76+
f"behaviorClose={failed_case['behaviorClose']}"
77+
)
7378
if len(failed_cases) > 3:
74-
print(f' ... and {len(failed_cases) - 3} more failed cases')
79+
print(f" ... and {len(failed_cases) - 3} more failed cases")
7580
all_passed = False
76-
77-
print('')
78-
print(f'{test_type} Summary: {passed_testees}/{total_testees} testees passed all tests')
79-
81+
82+
print("")
83+
print(
84+
f"{test_type} Summary: {passed_testees}/{total_testees} testees passed all tests"
85+
)
86+
8087
if not all_passed:
81-
print(f'❌ {test_type} conformance: FAILED - Not all tests passed')
88+
print(f"❌ {test_type} conformance: FAILED - Not all tests passed")
8289
return False
8390
else:
84-
print(f'✅ {test_type} conformance: PASSED - All tests passed')
91+
print(f"✅ {test_type} conformance: PASSED - All tests passed")
8592
return True
8693

8794

8895
def main():
8996
"""Main entry point."""
90-
parser = argparse.ArgumentParser(description='Verify WebSocket conformance test results')
91-
parser.add_argument('client_json', type=Path, help='Path to client index.json file')
92-
parser.add_argument('server_json', type=Path, help='Path to server index.json file')
93-
97+
parser = argparse.ArgumentParser(
98+
description="Verify WebSocket conformance test results"
99+
)
100+
parser.add_argument("client_json", type=Path, help="Path to client index.json file")
101+
parser.add_argument("server_json", type=Path, help="Path to server index.json file")
102+
94103
args = parser.parse_args()
95-
104+
96105
print("==> Verifying 100% WebSocket conformance...")
97-
106+
98107
client_passed = verify_conformance(args.client_json, "Client")
99108
print("")
100109
server_passed = verify_conformance(args.server_json, "Server")
101-
110+
102111
print("")
103112
print("==> Overall WebSocket Conformance Verification:")
104-
113+
105114
if client_passed and server_passed:
106-
print("✅ PASSED - Both client and server conformance tests achieved 100% pass rate")
115+
print(
116+
"✅ PASSED - Both client and server conformance tests achieved 100% pass rate"
117+
)
107118
sys.exit(0)
108119
else:
109-
print("❌ FAILED - One or more conformance tests did not achieve 100% pass rate")
120+
print(
121+
"❌ FAILED - One or more conformance tests did not achieve 100% pass rate"
122+
)
110123
print("")
111-
print("This means the WebSocket implementation has conformance issues that need to be addressed.")
112-
print("Download the detailed reports from the workflow artifacts to investigate specific failures.")
124+
print(
125+
"This means the WebSocket implementation has conformance issues that need to be addressed."
126+
)
127+
print(
128+
"Download the detailed reports from the workflow artifacts to investigate specific failures."
129+
)
113130
sys.exit(1)
114131

115132

116133
if __name__ == "__main__":
117-
main()
134+
main()

‎.github/workflows/wheels.yml‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -179,8 +179,7 @@ jobs:
179179
with:
180180
path: ${{ env.UV_CACHE_DIR }}
181181
key:
182-
uv-cache-${{ matrix.platform }}-${{ matrix.arch
183-
}}-${{ hashFiles('pyproject.toml') }}
182+
uv-cache-${{ matrix.platform }}-${{ matrix.arch }}-${{ hashFiles('pyproject.toml') }}
184183
restore-keys: |
185184
uv-cache-${{ matrix.platform }}-${{ matrix.arch }}-
186185
uv-cache-${{ matrix.platform }}-

‎.proto‎

Submodule .proto updated 28 files

0 commit comments

Comments
 (0)