Skip to content

[CI] CI with small size LLM (0.6B) #3516

@DonghakPark

Description

@DonghakPark

Enable CI test LLM Application

<|im_start|>user
 나한테 LLM에 대해서 설명해줘, 한국어로 대답해줘!.<|im_end|>
<|im_start|>assistant

<think>
Alright, the user wants me to explain what an LLM is in Korean. Let me start by recalling the basics. LLM stands for Large Language Model. I know they're a type of AI model, but I should explain it clearly.

First, I should mention that LLMs are trained on huge amounts of text to understand and generate complex human-like results. Then, explain that one of their main applications is in natural language processing. Maybe include something about how they can help with tasks like writing articles or answering questions.

I should keep it simple and conversational. Avoid technical jargon. Maybe add that LLMs can be used in various domains, like healthcare, finance, etc. Also, mention that they are very powerful tools now. Let me make sure the explanation is clear and not too long. Check if there's any additional information needed, but since the user just wants the explanation, keep it to that point.
</think>

**LLLL (Large Language Model)**는 매우 강력한 대량의 텍스트를 기반으로 학습한 AI 모델이며, 자연어 처리, 번역, 문장 생성 등 다양한 인문적 작업을 수행할 수 있는 기능을 가질 수 있습니다. 한 번에 수백만 개의 문장을 훈련하는 데는 매우 빠르고 효율적입니다. 예를 들어, 의사가 맞춤형 보건 정보를 제공하거나, 대국민 언 어 번역을 도와할 수 있습니다. 따라서, LLMs는 다양한 분야에서 활용되고 있습니다.<|im_end|>

=================[ LLM with NNTrainer ]===================
prefill: 27 tokens, 171 ms, 157.895 TPS
generation: 326 tokens, 8236 ms, 39.5823 TPS
==========================================================
Max Resident Set Size: 1291928 KB

i tested 0.6B Model on Local (Linux - i7) with Q40-Q40-FP32(FC - Embedding - LMhead)

  • Enable CI to test LLM Application
  • Download via LFS (?)
  • Compare Result
  • Compare Memory Usage
  • Compare Total Execution Time (?)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions