Implementation of [LFX] Domain-specific large model benchmarks: the edge perspective

[IcyFeather233]

What type of PR is this?

/kind feature

What this PR does / why we need it:

This is the implementation of

CNCF - KubeEdge: Domain-specific large model benchmarks: the edge perspective (2025 Term 1)

This PR is related to #177

[MooreZheng]

The PR looks fine to me so far. A few suggestions:

1. An issue is needed, showing the need to add a module of pre-process into the single task learning scheme and its consequences to community members (i.e., should be fine since the pre-process module could be skipped).
2. Take the 1-2-3-4 dimensions out from the data of each query and make the dimension information as metadata under specific directories, in order to make the dataset simple.
3. Use English after all contented fixed. Besides, there could also be method to ineterate both English and Chinese version for readme (see this link).
4. Try to fix the CI issue

Further reviews could be taken after demonstrations with experiments.

[IcyFeather233]

It seems that this CI / pylink (3.9) (pull_request) is not caused by my code change.

[AryanNanda17]

@IcyFeather233, you can try syncing your fork's main branch with ianvs main branch. PR #213 which is recently merged, corrects the error.

[IcyFeather233]

@MooreZheng @hsj576 Hi please review this PR

[MooreZheng]

Overall it looks good now. Some tiny comments on the routine meeting

1. Update the Kaggle dataset for the embedding
2. Take a look at the preprocess issue and add advices about Sedna version in this PR if needed. If it is about other documents like quick start, advices could be added using another PR.
3. Currently the PR has 9 commits and squash the commits into one.

[IcyFeather233]

1. I have updated the kaggle dataset here, and related information are also added in the PR README file.
2. The preprocess will not influence other examples, to use, just add preprocess function in BaseModel class like this:

@ClassFactory.register(ClassType.GENERAL, alias="gen")
class BaseModel:

    def __init__(self, **kwargs):
        ...

    def preprocess(self, **kwargs):
        # add your preprocess before train
        self.rag = GovernmentRAG(model_name="/path/to/models/bge-m3", device="cuda", persist_directory="./chroma_db")
        LOGGER.info("RAG initialized")


    def train(self, train_data, valid_data=None, **kwargs):
        ...
        
    def save(self, model_path):
        ...

    def predict(self, data, input_shape=None, **kwargs):
        ...

    def load(self, model_url=None):
        ...

    def evaluate(self, data, model_path, **kwargs):
        ...

If it is not needed, do not add this function is ok, because it is written like this:

def _preprocess(self, job):
        if job.preprocess() is None:
            return None
        return job.preprocess()

So it will not influence previous examples.
3. All commits have been merged into one

[MooreZheng]

/lgtm

[MooreZheng]

This is from the LFX mentorship Term 2 and the PR looks good to me now. We need another lgtm from @hsj576 :D

[MooreZheng]

This is from the LFX mentorship Term 2 and the PR looks good to me now. We need another lgtm from @hsj576 :D

@hsj576

[hsj576]

/lgtm

[MooreZheng]

/approve

[kubeedge-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hsj576, IcyFeather233, MooreZheng

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [MooreZheng]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[MooreZheng]

Related issue and PR:
Phase 1 Gov LLM #95 #113 #144
Phase 2 Gov RAG #177 #186 #196