fix: eliminate redundant RAG model reloading that defeated paralleliz…

[ARYANPATEL-BIT]

Overview

This PR resolves a critical architectural issue in the Government RAG example where redundant model initialization inside a mutex completely negated parallel execution.

---

Problem

The process_query() function instantiated a new GovernmentRAG object for nearly every query. Each instantiation:

• Reloaded the bge-m3 embedding model
• Reinitialized ChromaDB connections

This logic was executed inside a self.gpu_lock, causing:

• Thread serialization: All threads in ThreadPoolExecutor(max_workers=4) were forced to wait for the lock
• Severe performance degradation: Expensive model loading occurred repeatedly per query
• Resource inefficiency: Excessive VRAM usage, memory fragmentation, and unnecessary I/O overhead

As a result, the system behaved like a single-threaded pipeline despite being designed for parallel execution.

---

Solution

This PR aligns the implementation with a standard single-instance retrieval architecture:

• Single initialization

  • GovernmentRAG is initialized once during preprocess()
  • The same instance is reused across all queries

• Optimized lock scope

  • gpu_lock now only protects the vector similarity search (self.rag.query())
  • Removes unnecessary blocking around model setup

• True parallelism for LLM calls

  • get_model_response() is moved outside the lock
  • Enables concurrent API calls and response generation

• Fail-safe initialization

  • Adds a fallback initialization in predict() if preprocess() is skipped

---

Impact

• Restores actual multi-threaded performance
• Eliminates redundant model reloads
• Reduces GPU/VRAM pressure and prevents memory fragmentation
• Significantly improves benchmark execution speed and stability

---

Summary

This change removes a fundamental bottleneck that was silently disabling parallelism, ensuring the benchmarking pipeline performs as intended under concurrent workloads.

##Fixes: #380 381

[kubeedge-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ARYANPATEL-BIT
To complete the pull request process, please assign jaypume after the PR has been reviewed.
You can assign the PR to them by writing /assign @jaypume in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ARYANPATEL-BIT]

/assign @jaypume

[gemini-code-assist]

Code Review

This pull request optimizes the RAG benchmarking process by initializing the model once during preprocessing to improve parallel performance. However, the reviewer noted that this change removes the logic for handling different retrieval scopes (global, local, and other), which breaks the benchmarking comparisons. Additional feedback pointed out a hardcoded absolute path that hinders portability and suggested consolidating redundant initialization logic.

[gemini-code-assist]

The logic for handling different rag_type values ([local], [other]) has been removed. Previously, these types triggered the creation of a GovernmentRAG instance with specific provinces filters. Now, all RAG queries use the same global instance initialized in preprocess(), which ignores the location and rag_type parameters for retrieval. This breaks the benchmarking logic intended to compare different retrieval scopes (global vs local vs other).

[gemini-code-assist]

The model_name parameter is set to a hardcoded absolute path (/home/icyfeather/models/bge-m3). This makes the code non-portable and will cause it to fail on any system where this specific path does not exist. Consider using a configuration parameter or a relative path.

[gemini-code-assist]

The initialization logic for GovernmentRAG is duplicated here. To ensure consistency and avoid repeating hardcoded parameters, you should call self.preprocess() instead.

        if self.rag is None:
            LOGGER.info("RAG not initialized yet, loading now...")
            self.preprocess()