Upgrade grpc library to v1.78.1

[yash-ni]

1. Pull request overview

This PR upgrades the gRPC submodule and refactors LVMessage to no longer inherit from google::protobuf::Message, integrating with gRPC via a custom grpc::SerializationTraits specialization and migrating serialization/parsing to public protobuf CodedInputStream/CodedOutputStream APIs.

Changes:

• Removes LVMessage inheritance from google::protobuf::Message and implements custom gRPC SerializationTraits
• Migrates all serialization/deserialization logic from internal protobuf APIs to public CodedInputStream/CodedOutputStream APIs
• Updates protobuf descriptor access to handle API changes (string_view returns, new error collector methods)

2. Architecture Diagrams

2.1 How LVMessage Worked Before (gRPC version <= v1.62.0)

flowchart TB
    subgraph LabVIEW["LabVIEW Application"]
        LV_CLUSTER["LabVIEW Cluster Data"]
    end
    
    subgraph DLL["labview_grpc_server.dll"]
        COPIER["ClusterDataCopier"]
        LV_MSG["LVMessage<br/>(inherits from protobuf::Message)"]
        
        subgraph LVMessage_Internal["LVMessage Internal"]
            VALUES["_values map"]
            METADATA["_metadata"]
            SERIALIZE["_InternalSerialize()"]
            PARSE["_InternalParse()"]
        end
    end
    
    subgraph gRPC_Layer["gRPC Library"]
        GRPC_CALL["BlockingUnaryCall&lt;LVMessage&gt;()"]
        DEFAULT_TRAITS["Default SerializationTraits<br/>(for protobuf::Message)"]
        
        subgraph Protobuf_Old["Protobuf v3.x (old)"]
            MSG_BASE["google::protobuf::Message"]
            GET_CLASS["GetClassData()<br/>✅ Had default impl"]
            CACHED_SIZE["GetCachedSize()"]
            SERIALIZE_PB["SerializeWithCachedSizes()"]
        end
    end
    
    subgraph Network["Network"]
        WIRE["Protobuf Wire Format<br/>(binary bytes)"]
    end
    
    LV_CLUSTER -->|"CopyFromCluster()"| COPIER
    COPIER --> VALUES
    VALUES --> SERIALIZE
    
    GRPC_CALL -->|"Serialize message"| DEFAULT_TRAITS
    DEFAULT_TRAITS -->|"Calls virtual methods"| MSG_BASE
    MSG_BASE --> GET_CLASS
    GET_CLASS -->|"Default impl OK"| CACHED_SIZE
    CACHED_SIZE --> SERIALIZE_PB
    SERIALIZE_PB -->|"Delegates to"| SERIALIZE
    SERIALIZE --> WIRE
    
    LV_MSG -.->|"inherits"| MSG_BASE
    
    style GET_CLASS fill:#90EE90,stroke:#228B22
    style LV_MSG fill:#87CEEB,stroke:#4682B4
    style DEFAULT_TRAITS fill:#DDA0DD,stroke:#8B008B

Loading

How it worked:

1. LabVIEW cluster data copied into LVMessage._values via ClusterDataCopier
2. gRPC calls BlockingUnaryCall<LVMessage>()
3. Default SerializationTraits treats LVMessage as a protobuf::Message
4. Protobuf calls GetClassData() → had a default implementation ✅
5. Eventually calls LVMessage::_InternalSerialize() to write wire format
6. Data sent over network

2.2 What Broke Now (gRPC version > v1.63.0)

flowchart TB
    subgraph LabVIEW["LabVIEW Application"]
        LV_CLUSTER["LabVIEW Cluster Data"]
    end
    
    subgraph DLL["labview_grpc_server.dll"]
        COPIER["ClusterDataCopier"]
        LV_MSG["LVMessage<br/>(inherits from protobuf::Message)"]
        
        subgraph LVMessage_Internal["LVMessage Internal"]
            VALUES["_values map"]
            SERIALIZE["_InternalSerialize()"]
            GET_CLASS_IMPL["GetClassData() override<br/>❌ returns nullptr"]
        end
    end
    
    subgraph gRPC_Layer["gRPC Library"]
        GRPC_CALL["BlockingUnaryCall&lt;LVMessage&gt;()"]
        DEFAULT_TRAITS["Default SerializationTraits<br/>(for protobuf::Message)"]
        
        subgraph Protobuf_New["Protobuf v3.29.0+ (new)"]
            MSG_BASE["google::protobuf::Message"]
            GET_CLASS["GetClassData()<br/>⚠️ NOW PURE VIRTUAL"]
            ACCESS_CACHE["AccessCachedSize()"]
            DEREF["classData->cached_size_offset<br/>💥 CRASH: nullptr dereference"]
        end
    end
    
    subgraph Crash["CRASH"]
        EXCEPTION["❌ Access Violation<br/>GetClassData() returned nullptr"]
    end
    
    LV_CLUSTER -->|"CopyFromCluster()"| COPIER
    COPIER --> VALUES
    
    GRPC_CALL -->|"Serialize message"| DEFAULT_TRAITS
    DEFAULT_TRAITS -->|"Calls virtual methods"| MSG_BASE
    MSG_BASE --> GET_CLASS
    GET_CLASS -->|"Calls our override"| GET_CLASS_IMPL
    GET_CLASS_IMPL -->|"returns nullptr"| ACCESS_CACHE
    ACCESS_CACHE -->|"Tries to use nullptr"| DEREF
    DEREF --> EXCEPTION
    
    LV_MSG -.->|"inherits"| MSG_BASE
    
    style GET_CLASS fill:#FF6B6B,stroke:#8B0000
    style GET_CLASS_IMPL fill:#FF6B6B,stroke:#8B0000
    style DEREF fill:#FF6B6B,stroke:#8B0000
    style EXCEPTION fill:#FF0000,stroke:#8B0000,color:#FFFFFF
    style SERIALIZE fill:#D3D3D3,stroke:#808080

Loading

What breaks:

1. Same flow starts: LabVIEW data → LVMessage._values
2. gRPC calls BlockingUnaryCall<LVMessage>()
3. Default SerializationTraits treats LVMessage as a protobuf::Message
4. Protobuf calls GetClassData() → now pure virtual, requires implementation ⚠️
5. Our GetClassData() returns nullptr (we can't construct valid ClassData)
6. Protobuf tries to access classData->cached_size_offset → 💥 CRASH
7. Additionally, the wire format decoder functions (ReadINT32, ReadUINT32, ReadINT64, ReadSINT32, ReadFIXED32, ReadFLOAT, etc.) have been removed from map_type_handler.h (commit titled "Remove dead code" - file went from 610 to 377 lines). These were internal implementation details for map parsing, never part of the public API. The corresponding encoder functions (WireFormatLite::WriteInt32ToArray, etc.) remain available.

2.3 The New Architecture (Solution with SerializationTraits)

flowchart TB
    subgraph LabVIEW["LabVIEW Application"]
        LV_CLUSTER["LabVIEW Cluster Data"]
    end
    
    subgraph DLL["labview_grpc_server.dll"]
        COPIER["ClusterDataCopier"]
        
        subgraph LVMessage_New["LVMessage (standalone - NO protobuf inheritance)"]
            VALUES["_values map"]
            METADATA["_metadata"]
            SERIALIZE["SerializeToByteBuffer()"]
            PARSE["ParseFromByteBuffer()"]
            CODED["Uses CodedOutputStream<br/>& CodedInputStream"]
        end
        
        subgraph Traits["grpc::SerializationTraits&lt;LVMessage&gt;"]
            TRAIT_SER["Serialize()"]
            TRAIT_DES["Deserialize()"]
        end
    end
    
    subgraph gRPC_Layer["gRPC Library"]
        GRPC_CALL["BlockingUnaryCall&lt;LVMessage&gt;()"]
        CUSTOM_TRAITS["Custom SerializationTraits<br/>✅ Bypasses protobuf::Message"]
    end
    
    subgraph Protobuf_Helpers["Protobuf (helper only)"]
        CODED_STREAM["io::CodedOutputStream<br/>io::CodedInputStream"]
        WIRE_FORMAT["WireFormatLite"]
    end
    
    subgraph Network["Network"]
        WIRE["Protobuf Wire Format<br/>(binary bytes)"]
    end
    
    LV_CLUSTER -->|"CopyFromCluster()"| COPIER
    COPIER --> VALUES
    
    GRPC_CALL -->|"Looks up traits"| CUSTOM_TRAITS
    CUSTOM_TRAITS -->|"Calls our custom"| TRAIT_SER
    TRAIT_SER -->|"Calls"| SERIALIZE
    SERIALIZE -->|"Uses helper APIs"| CODED
    CODED --> CODED_STREAM
    CODED_STREAM --> WIRE_FORMAT
    WIRE_FORMAT --> WIRE
    
    style CUSTOM_TRAITS fill:#90EE90,stroke:#228B22
    style TRAIT_SER fill:#90EE90,stroke:#228B22
    style TRAIT_DES fill:#90EE90,stroke:#228B22
    style LVMessage_New fill:#87CEEB,stroke:#4682B4
    style CODED_STREAM fill:#DDA0DD,stroke:#8B008B

Loading

How the solution works:

1. LabVIEW cluster data → LVMessage._values (unchanged)
2. gRPC calls BlockingUnaryCall<LVMessage>()
3. gRPC finds our custom SerializationTraits<LVMessage> ✅
4. Our traits call LVMessage::SerializeToByteBuffer() directly
5. LVMessage uses CodedOutputStream (public protobuf helper API)
6. No GetClassData() call ever happens - we bypass protobuf::Message entirely
7. Data sent over network in same wire format

3. Summary per file

File	Description
third_party/grpc	Updates gRPC submodule commit to the target version.
tests/unit/lv_message_tests.cc	Adds extensive unit coverage for LVMessage wire round-trips and SerializationTraits integration.
src/unpacked_fields.cc	Uses data() instead of c_str() for binary copies.
src/test_server.cc	Removes includes of internal gRPC core headers (no longer accessible).
src/string_utils.h	Simplifies VerifyUtf8String signature and removes dependency on WireFormatLite::Operation.
src/string_utils.cc	Replaces internal WireFormatLite::VerifyUtf8String usage with utf8_range validation.
src/semaphore.h	Updates header commentary and includes to avoid system <semaphore.h> conflicts (but see suggestion re: rename).
src/proto_parser.cc	Adapts error-collection and descriptor string access for updated protobuf APIs.
src/pointer_manager.h	Updates template constructor declaration.
src/message_value.h	Changes serialization interface to CodedOutputStream and introduces cached-size fields.
src/lv_serialization_traits.h	Adds gRPC SerializationTraits specialization declaration for LVMessage.
src/lv_serialization_traits.cc	Implements SerializationTraits serialize/deserialize via LVMessage ByteBuffer methods.
src/lv_proto_server_reflection_service.h	Adds NOMINMAX for Windows builds.
src/lv_proto_server_reflection_service.cc	Adjusts string_view-returning descriptor APIs by materializing std::string.
src/lv_proto_server_reflection_plugin.h	Adds NOMINMAX for Windows builds.
src/lv_message_value.cc	Reimplements LVMessageValue serialization using CodedOutputStream and public helpers.
src/lv_message_efficient.h	Switches efficient parsing hooks to CodedInputStream-based overrides.
src/lv_message_efficient.cc	Reimplements efficient parsing via CodedInputStream (direct-to-cluster).
src/lv_message.h	Refactors LVMessage away from protobuf inheritance; adds CodedStream parse/serialize APIs.
src/lv_message.cc	Reimplements parsing/unknown-field handling using public CodedInputStream APIs.
src/lv_interop.h	Adds NOMINMAX for Windows builds.
src/grpc_server.h	Ensures SerializationTraits header is included before gRPC headers; updates semaphore include.
src/grpc_client.h	Ensures SerializationTraits header is included before gRPC headers.
src/event_data.cc	Removes internal protobuf namespace usage.
CMakeLists.txt	Moves build to C++20, adds MSVC workaround, and wires up new unit test target.

[jasonmreding]

I would like to review changes again after changes to consolidate some of the code duplication.

[jasonmreding]

This is still using google::protobuf::Message. I think this is further proof this code path is largely untested from an automated test standpoint and is languishing. It would be nice to either get it back into a functioning state or abandon the code and the performance optimizations it can provide. Spending time to maintain and review code that is otherwise unreachable seems like a waste of time.

[yash-ni]

After completing this upgrade, we plan to work on the lv_message_efficient class. Since it is currently disabled, it would be better to address these changes once that work is done.

[jasonmreding]

This is probably another bug in waiting of sorts that has been waiting here, but this class should probably override the Clear method and reset all of these fields. It should probably also reset _LVClusterHandle. However, since that is only assigned during construction, that pretty much makes the instance useless. If we ever want to reuse this instance for streaming calls in the future, we'll probably need to add another API that allows for clearing and updating of the cluster pointer. If you decide this is out of scope for this PR, I'm OK with that.

[yash-ni]

Thanks for pointing this out, will pick this up later.

[jasonmreding]

These tests are a nice addition! Thanks for adding them. I'm still looking through the tests, but the immediate thoughts I had are:

• I didn't immediately see any oneof testing. It would be good to add a few of those.
• This test file is pretty long. Consider breaking it into multiple files for different categories of tests, perhaps by using lv_message_tests_<category>.cc as a naming convention. This is just a suggestion so feel free to ignore if you don't see any value in it.
• Consider using protoc to generate message classes for test messages you define in a proto file. Then use the generated messages to produce serialized data that is parsed from our message class and verify interoperability. Do the same thing for the other direction. At least having a few tests like this would be nice. I'm not sure if I would completely replace the existing round tripping with this approach (risks two wrongs making a right) or just augment it with this approach. I know manually writing a bunch of test messages in a proto file can be pretty tedious compared to programmatically generating message data for testing purposes.

[yash-ni]

We’re planning to add more tests, including LabVIEW tests, sometime next month. The unit tests I added here are mainly to ensure that the recent refactoring didn’t break existing functionality, since the changes were fairly extensive.

I think we can note the additional unit test suggestions you mentioned and incorporate them in a future PR. Let me know what you think.

[jasonmreding]

I think having some interoperability tests that compare against a "trusted source" similar to how we have a Python client communicate with a LV service are a good idea and probably more of a requirement long term. I'm OK if that is deferred to a future PR as long as there is a commitment to revisiting it in the relative near term.

[yash-ni]

I did some code cleanup for the unit tests. The remaining work like breaking down into smaller unit tests can be handled later when we begin dedicated testing.

[bkeryan]

I haven't gotten through all of this but the basic approach makes sense to me.

[jasonmreding]

Call AppendToLVArray here.

[jasonmreding]

Call AppendVectorToLVArray here.

[jasonmreding]

The overall change looks good to me. I would still like to see some minor cleanup from the comments that are still open. The biggest issues for me at this point are:

• Answering the question about Linux RT compatibility.
• Cleaning up the unit tests a bit more to minimize code duplication through parametric tests where appropriate.

When testing this change against LV 2025 and some LV measurement plugins, I encountered crashes on project close that took me a while to figure out. It turns out this is due to existing bugs that I am not going to try and include fixes for as part of this PR. However, they are fairly serious issues so I would like to create a follow on PR after this one is completed and before publishing a new release.

[yash-ni]

@jasonmreding Thanks for identifying the issues with lv_message_efficient. For now, I’m keeping those issues open since its future is still uncertain - we haven’t yet decided whether to enable it again or remove it. If we do decide to re-enable it, I’ll take your comments into account while making the necessary changes.