Cmake builds libgit2 and --pull_hf_mode for ovms #3105

rasapala · 2025-03-04T15:52:30Z

🛠 Summary

JIRA CVS-163308
Current usage:
bazel-bin/src/ovms --pull_hf_model --source_model OpenVINO/TinyLlama-1.1B-Chat-v1.0-int8-ov --repo_path /download

🧪 Checklist

Unit tests added.
The documentation updated.
Change follows security best practices.
``

…08_libgt2

atobiszei · 2025-03-07T09:48:25Z

src/server.cpp

-        parser.prepare(&serverSettings, &modelsSettings);
+        parser.prepare(&serverSettings, &modelsSettings, &hfDownloader);
+
+        if (hfDownloader.pull_hf_model) {


Shouldn't this be responsibility of ServableModule?

I think we should separate the server mode and hf download mode.

atobiszei · 2025-03-07T09:50:00Z

third_party/libgt2/libgt2_engine.bzl

+import fnmatch
+import sys
+
+def remove_japanese_txt(directory):


We are using this in other repo's already. Calling this should be some kind of bazel procedure loaded from *.bzl file

Need to placed exactly in the repository ctx directory on every build. This script will remove a directory not only files because we cannot filter the files by extension like it was done in the other python script.

…t/model_server into CVS-163308_libgt2

…08_libgt2

atobiszei · 2025-04-15T13:26:36Z

src/capi_frontend/server_settings.hpp

@@ -66,4 +66,10 @@ struct ModelsSettingsImpl {
    std::string configPath;
 };

+struct HFSettingsImpl {


Why separate structure instead of nesting this inside server settings?

This structure will grow and the settings are not about server functionality.

We can nest the new structure in server settings as agreed on sync.

atobiszei · 2025-04-15T13:27:52Z

src/hf_pull_model_module.cpp

+#include "logging.hpp"
+#include "server.hpp"
+#include "status.hpp"
+#include "libgt2/libgt2.hpp"


external dependencies should be before ovms headers.

"libgt2/libgt2.hpp" is our wrapper implementation.

atobiszei · 2025-04-15T13:30:40Z

src/libgt2/libgt2.hpp

+    void setSourceModel(std::string inSourceModel);
+    void setRepositoryPath(std::string inRepoPath);


I assume we set repo path once per start as well as source mode. Why do we need setters here?

We can remove them now.

atobiszei · 2025-04-15T13:31:26Z

src/libgt2/libgt2.hpp

+    int cloneRepository();
+    void setSourceModel(std::string inSourceModel);
+    void setRepositoryPath(std::string inRepoPath);
+    void setPullHfModelMode(bool isOn);


We don't create HfDownloader anyway if we don't pull so why we change mode?

This option can be used in the future. We can remove setter though.

atobiszei · 2025-04-15T13:35:41Z

src/libgt2/libgt2.cpp

+    this->repoUrl = "";
+}
+
+HfDownloader::HfDownloader(const HfDownloader& hfDownloader) {


Do we really need cpy constructor? If we need default then = default; should suffice here.

Not needed anymore.

atobiszei · 2025-04-15T13:47:38Z

src/libgt2/libgt2.cpp

+bool HfDownloader::CheckIfProxySet() {
+    const char* envCred = std::getenv("https_proxy");
+    if (!envCred)
+        return false;
+    return true;
+}
+
+bool HfDownloader::CheckIfTokenSet() {
+    const char* envCred = std::getenv("HF_TOKEN");
+    if (!envCred)
+        return false;
+    return true;
+}


those are not methods but utility functions here

Strictly connected and used in the class. Can we keep them as private?

atobiszei · 2025-04-15T13:48:55Z

src/libgt2/libgt2.cpp

+void HfDownloader::SetHfEndpoint() {
+    const char* envCred = std::getenv("HF_ENDPOINT");
+    this->hfEndpoint = "huggingface.co";
+    if (envCred) {
+        this->hfEndpoint = std::string(envCred);
+    } else {
+        printf("Info: HF_ENDPOINT environment variable not set.\n");
+    }
+
+    if (!endsWith(this->hfEndpoint, "/")) {
+        this->hfEndpoint.append("/");
+    }
+}
+
+void HfDownloader::UpdateRepoUrl() {
+    this->repoUrl = "https://";
+    this->repoUrl += this->hfEndpoint + this->sourceModel;
+}


hfEndpoint is used only to create repo url so this does not need to be a member. We don't need to store it as state after cloning repository. Same is true for repoUrl

atobiszei · 2025-04-15T13:52:46Z

src/libgt2/libgt2.cpp

+    return true;
+}
+
+void HfDownloader::GetRepositoryUrlWithPassword(std::string& passRepoUrl) {


take endpoint & source model as argument. Return passRepoUrl instead of passing ref.

atobiszei · 2025-04-15T13:57:11Z

src/hf_pull_model_module.cpp

+    state = ModuleState::STARTED_INITIALIZE;
+    SPDLOG_INFO("{} starting", HF_MODEL_PULL_MODULE_NAME);
+
+    this->hfDownloader = std::make_unique<HfDownloader>(config.getHfSettings().sourceModel, config.getHfSettings().repoPath, config.getHfSettings().pullHfModelMode);


Maybe put HfSettings as a member of server settings.
Why pass all settings separatelly when we can pass them all at once?

Addiditonally I would consider checking for ENV variables here and passing content below for easier (less intertwined testing). You cannot now verify different behavior with different env variables at the level of module.

Module will have clone methods. This can be tested for env settings.

atobiszei · 2025-04-15T14:01:24Z

third_party/libgt2/libgt2_engine.bzl

+        commit = "338e6fb681369ff0537719095e22ce9dc602dbf0", # Dec 28, 2024 - v1.9.0
+        build_file = "@_libgt2_engine//:BUILD",
+        patch_args = ["-p1"],
+        patches = ["lfs.patch"],


Put commend here why this patch is needed

atobiszei · 2025-04-16T08:52:57Z

src/BUILD

@@ -2712,6 +2715,7 @@ cc_test(
        "test/pipelinedefinitionstatus_test.cpp",
        "test/predict_validation_test.cpp",
        "test/prediction_service_test.cpp",
+        "test/pull_hf_model_test.cpp",


separate test target. check eg openvino_tests/openvino_remote_tensor_tests.

This is standard tests, does not require additional settings nor dependencies.

atobiszei · 2025-04-16T10:01:19Z

src/test/test_utils.cpp

+    while ((server.getModuleState(ovms::HF_MODEL_PULL_MODULE_NAME) != ovms::ModuleState::SHUTDOWN) &&
+           (std::chrono::duration_cast<std::chrono::seconds>(std::chrono::high_resolution_clock::now() - start).count() < timeoutSeconds)) {
+    }
+    ASSERT_EQ(server.getModuleState(ovms::HF_MODEL_PULL_MODULE_NAME), ovms::ModuleState::SHUTDOWN) << "OVMS did not download model in allowed time:" << timeoutSeconds << "s. Check machine load";


Suggested change

ASSERT_EQ(server.getModuleState(ovms::HF_MODEL_PULL_MODULE_NAME), ovms::ModuleState::SHUTDOWN) << "OVMS did not download model in allowed time:" << timeoutSeconds << "s. Check machine load";

ASSERT_EQ(server.getModuleState(ovms::HF_MODEL_PULL_MODULE_NAME), ovms::ModuleState::SHUTDOWN) << "OVMS did not download model in allowed time:" << timeoutSeconds << "s. Check machine & network load";

atobiszei · 2025-04-16T10:03:00Z

src/test/test_utils.cpp

@@ -696,6 +702,30 @@ void EnsureServerStartedWithTimeout(ovms::Server& server, int timeoutSeconds) {
    ASSERT_EQ(server.getModuleState(ovms::SERVABLE_MANAGER_MODULE_NAME), ovms::ModuleState::INITIALIZED) << "OVMS did not fully load until allowed time:" << timeoutSeconds << "s. Check machine load";
 }

+void EnsureServerModelDownloadFinishedWithTimeout(ovms::Server& server, int timeoutSeconds) {
+    auto start = std::chrono::high_resolution_clock::now();
+    while ((server.getModuleState(ovms::HF_MODEL_PULL_MODULE_NAME) != ovms::ModuleState::SHUTDOWN) &&


Shouldn't it be != AVAILABLE?

Module shutdowns after clone() is executed.

atobiszei · 2025-04-16T10:05:09Z

src/hf_pull_model_module.cpp

+    state = ModuleState::INITIALIZED;
+    SPDLOG_INFO("{} started", HF_MODEL_PULL_MODULE_NAME);
+
+    status = this->hfDownloader->cloneRepository();
+    if (!status.ok()) {
+        return status;
+    }


Suggested change

state = ModuleState::INITIALIZED;

SPDLOG_INFO("{} started", HF_MODEL_PULL_MODULE_NAME);

status = this->hfDownloader->cloneRepository();

if (!status.ok()) {

return status;

}

SPDLOG_INFO("{} started", HF_MODEL_PULL_MODULE_NAME);

status = this->hfDownloader->cloneRepository();

if (!status.ok()) {

return status;

}

state = ModuleState::INITIALIZED;

?

rasapala added 12 commits March 4, 2025 16:22

Cmake builds

09ae985

Init missing

66fb2cf

Working with proxy

02727f6

Style

1a4d718

Fix chineses file by removing all files

7fdcd5c

Style

baf7344

Fix so dependency

e561904

Merge branch 'main' into CVS-163308_libgt2

22692c6

License

a580a43

Spell in license

211fc40

Fix redhat

982ab97

Merge remote-tracking branch 'origin/CVS-163308_libgt2' into CVS-1633…

dc673d5

…08_libgt2

atobiszei reviewed Mar 7, 2025

View reviewed changes

rasapala added 13 commits March 7, 2025 14:41

Workong HF_TOKEN

86f9c31

Does not work lfs

a8de843

Working fiters

cb0e77f

Add file name

c0fbfdf

Merge branch 'main' into CVS-163308_libgt2

baaf77c

Linux compile

44c2787

Set workdir on windows

90c26b3

Working linux

8b3d09c

Merge branch 'main' into CVS-163308_libgt2

fe6b425

Add credentails to url

21005f2

Merge branch 'CVS-163308_libgt2' of https://github.com/openvinotoolki…

bb4d800

…t/model_server into CVS-163308_libgt2

WIP

34835af

Merge remote-tracking branch 'origin/CVS-163308_libgt2' into CVS-1633…

fb499e1

…08_libgt2

rasapala changed the title ~~Cmake builds libgit2~~ [WIP] Cmake builds libgit2 Mar 19, 2025

rasapala added 2 commits March 19, 2025 14:30

Pass, hf_endpoint and rewrite

7c5e798

Merge branch 'main' into CVS-163308_libgt2

b1e1790

rasapala added 4 commits April 11, 2025 14:37

Fix srcs

eb4b3ba

Fix install

1583762

Create package

2751ec9

Fix windows

fba3ee9

rasapala changed the title ~~[WIP] Cmake builds libgit2~~ Cmake builds libgit2 and --pull_hf_mode for ovms Apr 11, 2025

rasapala requested a review from dtrawins April 11, 2025 13:12

rasapala added 5 commits April 14, 2025 10:08

Fix so links

3028184

Merge branch 'main' into CVS-163308_libgt2

5d08896

Produce one so

4489a12

Fix win install

9f58ea0

New dev and test

96c0f20

atobiszei reviewed Apr 15, 2025

View reviewed changes

rasapala added 3 commits April 15, 2025 18:27

git2 clone module

5d9c5be

Merge branch 'main' into CVS-163308_libgt2

2520d48

Style

b7190ae

atobiszei reviewed Apr 16, 2025

View reviewed changes

Capi fixes

be8a777

atobiszei reviewed Apr 16, 2025

View reviewed changes

rasapala added 10 commits April 16, 2025 15:32

Works win and lin, refactor 1

f91a655

Working refactor 2

025d6b7

Refactor and review

d5cdfd0

Remove lock

9a6079c

Print info changes

9bc522b

Change to INFO

00d693e

Fix proxy and debug

e825390

No script

b38520f

Take proxy from build configuration

bad937d

Add noproxy

bb4bdd2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cmake builds libgit2 and --pull_hf_mode for ovms #3105

Cmake builds libgit2 and --pull_hf_mode for ovms #3105

rasapala commented Mar 4, 2025 •

edited

Loading

atobiszei Mar 7, 2025

rasapala Mar 19, 2025

atobiszei Mar 7, 2025

rasapala Mar 19, 2025

atobiszei Apr 15, 2025

rasapala Apr 16, 2025

rasapala Apr 16, 2025

atobiszei Apr 15, 2025

rasapala Apr 16, 2025

atobiszei Apr 15, 2025

rasapala Apr 16, 2025

atobiszei Apr 15, 2025

rasapala Apr 16, 2025

atobiszei Apr 15, 2025

rasapala Apr 16, 2025

atobiszei Apr 15, 2025

rasapala Apr 16, 2025

atobiszei Apr 15, 2025

atobiszei Apr 15, 2025

atobiszei Apr 15, 2025

rasapala Apr 16, 2025

atobiszei Apr 15, 2025

atobiszei Apr 16, 2025

rasapala Apr 16, 2025

atobiszei Apr 16, 2025

atobiszei Apr 16, 2025

rasapala Apr 16, 2025

atobiszei Apr 16, 2025

		void setSourceModel(std::string inSourceModel);
		void setRepositoryPath(std::string inRepoPath);

	ASSERT_EQ(server.getModuleState(ovms::HF_MODEL_PULL_MODULE_NAME), ovms::ModuleState::SHUTDOWN) << "OVMS did not download model in allowed time:" << timeoutSeconds << "s. Check machine load";
	ASSERT_EQ(server.getModuleState(ovms::HF_MODEL_PULL_MODULE_NAME), ovms::ModuleState::SHUTDOWN) << "OVMS did not download model in allowed time:" << timeoutSeconds << "s. Check machine & network load";

Cmake builds libgit2 and --pull_hf_mode for ovms #3105

Are you sure you want to change the base?

Cmake builds libgit2 and --pull_hf_mode for ovms #3105

Conversation

rasapala commented Mar 4, 2025 • edited Loading

🛠 Summary

🧪 Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rasapala commented Mar 4, 2025 •

edited

Loading