Skip to content

[WIP] Project#15: GGUF Reader in OpenVINO for direct GGUF Execution#3449

Open
shahkarnav115-beep wants to merge 7 commits intoopenvinotoolkit:masterfrom
shahkarnav115-beep:ggufreaderv2-gsoc26-warmup
Open

[WIP] Project#15: GGUF Reader in OpenVINO for direct GGUF Execution#3449
shahkarnav115-beep wants to merge 7 commits intoopenvinotoolkit:masterfrom
shahkarnav115-beep:ggufreaderv2-gsoc26-warmup

Conversation

@shahkarnav115-beep
Copy link

@shahkarnav115-beep shahkarnav115-beep commented Mar 5, 2026

Description

  • Added gguf_reader_v2.cpp and gguf_reader_v2.hpp to src/cpp/src/gguf_utils/.
  • Added llama.cpp (@ravi9's branch[dev_backend_openvino]) in thirdparty

@ravi9 - Opening this as a WIP to get some feedback on the core C++ implementation and structure. Unit tests (gguf_reader_v2_test.cpp) are currently being finalized locally and will be pushed in a subsequent commit once the base logic is reviewed.

Checklist:

  • This PR follows GenAI Contributing guidelines.
  • Tests have been updated or added to cover the new code.
  • This PR fully addresses the ticket.
  • I have made corresponding changes to the documentation.

@shahkarnav115-beep
Copy link
Author

@ravi9 some of the work for the build is left to be done. Will push the commit shortly.

@as-suvorov as-suvorov marked this pull request as draft March 6, 2026 08:52
@as-suvorov
Copy link
Collaborator

@shahkarnav115-beep Converting PR to draft as description states it's WIP

@shahkarnav115-beep
Copy link
Author

This is the complete integration of llama.cpp ("dev_backend_openvino" - branch) in openvino.genai as a submodule (static library).

I have implemented the minimal architecture (skeleton) of GGUFReaderV2 in gguf_reader_v2.cpp and its .hpp. To verify that it successfully calls the APIs from the submodule, I have also added a minimal C++ regression test in gguf_reader_v2_test.cpp.

Locally, I have built all the required components and tested them, and this minimal scaffold is compiling and working perfectly on my system. As part of my GSoC 2026 preparation, I would be extremely grateful to get your reviews on this initial architectural integration.

cc: @ravi9, @cavusmustafa

@shahkarnav115-beep shahkarnav115-beep marked this pull request as ready for review March 9, 2026 09:53
@shahkarnav115-beep
Copy link
Author

Link to headers file for submodule's (dev_backend_openvino branch): ravi9/llama.cpp#55

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants