Monthly Round up of new features in DeepFabric

[lukehinds]

Monthly Round up of new features in DeepFabric

Development has been moving fast and realised I have not had time to roundup new features, so here we go

Format datasets directly from Huggingface

It's now possible to pull a deepfabric dataset directly from Hugging Face and format it on the fly! This makes working with different datasets much more agile and adaptive. Simply push your dataset in the generic deepfabric style format, then pull it in and format it at training time.

For example, let's say you have an existing dataset you spent some time creating - you can format it on the fly to train your gpt-oss models in the supported Harmony format from OpenAI:

! deepfabric format --repo "org/dataset-name" --format harmony

train_dataset = load_dataset("json", data_files="./deepfabric.jsonl")
trainer = SFTTrainer(
    model=gpt-oss,
    args=training_args,
    train_dataset=train_dataset,
    processing_class=tokenizer,  # Use tokenizer, not processor
)

Full CoT format

This seems like a long while ago now, but it was just short of a month ago that we shipped full chain-of-thought with reasoning traces

A good example dataset here, showing an agent debugging a kubernetes cluster

XLAM Tool based dataset creation

There is now a built-in XLAM formatter, allowing you to create the XLAM format that recently took top spot on the Berkeley Function Calling Leaderboard (BFCL) V4 , full example here

This can then be piped into training into an SFT pipeline

Support for TRL tools formatting

SFTTrainer now fully supports fine-tuning models with tool calling capabilities.

https://huggingface.co/docs/trl/en/sft_trainer#tool-calling-with-sft

We now have a builtin trl formatter, so you can create full reasoning, agent tool calling datasets which load natively into TRL:

{
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful AI assistant with access to various tools..."
    },
    {
      "role": "user",
      "content": "What's the weather in Paris?"
    },
    {
      "role": "assistant",
      "content": "Let me check the weather for you..."
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get weather information for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "City name"
            },
            "unit": {
              "type": "string",
              "description": "Temperature unit"
            }
          },
          "required": ["location"]
        }
      }
    }
  ]
}

See the docs for more examples

Goodbye LiteLLM

The LiteLLM library was really useful, but we decided to build a leaner native implementation and at the same time bring in the outlines project to make structured output enforcement even more damming on those unruly neural-nets

https://github.com/lukehinds/deepfabric/pull/297

[vdesai-dev]

XLAM is.a great addition, does this work with the xlam models only?

[lukehinds]

It's fairly simple to convert, here is a case of it using Qwen , https://colab.research.google.com/drive/1yy7AZ9mB_RSHq_GPd9fSl5qfbVTJ--xm#scrollTo=ROIKxQWz9E98

with the xlam models though, its native to that format. Likely better to just format from DF into XLAM.