docs: add code structure documentation. #607
Conversation
XuZhang99
changed the title
|
/gemini review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces comprehensive documentation on the core modules and architecture of the xLLM system. The document is well-structured and detailed, providing a valuable technical overview. My review focuses on improving the clarity and accuracy of the information presented. I've pointed out a minor inconsistency in code paths within the summary table and a potential inaccuracy in a method name description for the Worker module. Overall, this is a great addition to the project's documentation.
|
|
||
| | Module Name | Code Path | Core Functionality | Key Responsibilities | | ||
| | :--- | :--- | :--- | :--- | | ||
| | **APIService** | `api_service/` | External Interface Service | Receives external requests (e.g., OpenAI-compatible API calls), translates them into the internal request format, and forwards them to the Master. | |
There was a problem hiding this comment.
For consistency, it's better to use specific file paths in the 'Code Path' column of the table, rather than directory paths. The detailed description for APIService already points to api_service/api_service.cpp as the entry point.
| | **APIService** | `api_service/` | External Interface Service | Receives external requests (e.g., OpenAI-compatible API calls), translates them into the internal request format, and forwards them to the Master. | | |
| | **APIService** | `api_service/api_service.cpp` | External Interface Service | Receives external requests (e.g., OpenAI-compatible API calls), translates them into the internal request format, and forwards them to the Master. | |
|
|
||
| **Code Entry Point**: `core/runtime/llm_worker_impl.cpp` | ||
|
|
||
| * **Local Resource Management**: The Worker is responsible for allocating the local physical memory for the KV Cache (`allocate_kv_cache`) during its initialization phase. |
There was a problem hiding this comment.
The documentation mentions that the Worker is responsible for allocating KV Cache via the allocate_kv_cache method. However, this method doesn't seem to exist in core/runtime/llm_worker_impl.cpp. Please verify if the method name is correct or if it belongs to a base class, and update the documentation accordingly for accuracy.
3 participants
No description provided.