data_process.md

数据处理算子

基类算子

::: lazyllm.tools.data.LazyLLMDataBase members: exclude-members:

演示算子

::: lazyllm.tools.data.operators.demo_ops members: exclude-members:

偏好数据处理算子

::: lazyllm.tools.data.operators.preference_ops members: exclude-members:

工具调用数据处理算子

::: lazyllm.tools.data.operators.tool_use_ops members: exclude-members:

Text2SQL 数据处理算子

::: lazyllm.tools.data.operators.text2sql_ops members: exclude-members:

预训练算子

::: lazyllm.tools.data.operators.pt_op members: exclude-members:

精炼算子

::: lazyllm.tools.data.operators.refine_op members: exclude-members:

过滤算子

::: lazyllm.tools.data.operators.filter_op members: exclude-members:

切块算子

::: lazyllm.tools.data.operators.token_chunker members: exclude-members:

代码生成算子

::: lazyllm.tools.data.operators.codegen_ops members: exclude-members:

Agentic rag

::: lazyllm.tools.data.operators.agentic_rag.agenticrag_atomic_task_generator members: exclude-members:

::: lazyllm.tools.data.operators.agentic_rag.agenticrag_depth_qa_generator members: exclude-members:

::: lazyllm.tools.data.operators.agentic_rag.agenticrag_qaf1_sample_evaluator members: exclude-members:

::: lazyllm.tools.data.operators.agentic_rag.agenticrag_width_qa_generator members: exclude-members:

纯文本生成QA对算子

::: lazyllm.tools.data.operators.text2qa_ops members: exclude-members:

Embedding 数据合成

::: lazyllm.tools.data.operators.embedding_synthesis members: exclude-members:

知识库清洗

::: lazyllm.tools.data.operators.knowledge_cleaning.file_or_url_to_markdown_converter_api members: exclude-members:

::: lazyllm.tools.data.operators.knowledge_cleaning.kbc_chunk_generator_batch members: exclude-members:

::: lazyllm.tools.data.operators.knowledge_cleaning.kbc_chunk_generator members: exclude-members:

::: lazyllm.tools.data.operators.knowledge_cleaning.kbc_multihop_qa_generator_batch members: exclude-members:

::: lazyllm.tools.data.operators.knowledge_cleaning.kbc_text_cleaner_batch members: exclude-members:

::: lazyllm.tools.data.operators.knowledge_cleaning.kbc_text_cleaner members: exclude-members:

::: lazyllm.tools.data.operators.knowledge_cleaning.qa_extract members: exclude-members:

思维链生成算子

::: lazyllm.tools.data.operators.cot_ops members: exclude-members:

多样性强化算子

::: lazyllm.tools.data.operators.enQa_ops members: exclude-members:

数学问题算子

::: lazyllm.tools.data.operators.math_ops members: exclude-members:

Pdf处理算子

::: lazyllm.tools.data.operators.pdf_ops members: exclude-members:

Reranker 数据合成

::: lazyllm.tools.data.operators.reranker_synthesis members: exclude-members:

LLM JSON 算子

::: lazyllm.tools.data.operators.llm_base_ops members: exclude-members:

::: lazyllm.tools.data.operators.llm_json_ops members: exclude-members:

数据处理 Pipeline

演示Pipeline

::: lazyllm.tools.data.pipelines.demo_pipelines members: exclude-members:

文本生成 QA 数据流水线

::: lazyllm.tools.data.pipelines.text_pipelines members: exclude-members:

QA 增强数据流水线

::: lazyllm.tools.data.pipelines.enhance_pipelines members: exclude-members:

思维链（CoT）数据流水线

::: lazyllm.tools.data.pipelines.cot_pipelines members: exclude-members:

图像数据处理流水线

::: lazyllm.tools.data.pipelines.img_pipelines members: exclude-members:

数学数据处理流水线

::: lazyllm.tools.data.pipelines.math_pipelines members: exclude-members:

PDF 数据处理流水线

::: lazyllm.tools.data.pipelines.pdf_pipelines members: exclude-members: