|
| 1 | +<!--Copyright 2025 The HuggingFace Team. All rights reserved. |
| 2 | +
|
| 3 | +Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with |
| 4 | +the License. You may obtain a copy of the License at |
| 5 | +
|
| 6 | +http://www.apache.org/licenses/LICENSE-2.0 |
| 7 | +
|
| 8 | +Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on |
| 9 | +an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the |
| 10 | +specific language governing permissions and limitations under the License. |
| 11 | +
|
| 12 | +⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be |
| 13 | +rendered properly in your Markdown viewer. |
| 14 | +
|
| 15 | +--> |
| 16 | + |
| 17 | +# D-FINE |
| 18 | + |
| 19 | +## Overview |
| 20 | + |
| 21 | +The D-FINE model was proposed in [D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement](https://arxiv.org/abs/2410.13842) by |
| 22 | +Yansong Peng, Hebei Li, Peixi Wu, Yueyi Zhang, Xiaoyan Sun, Feng Wu |
| 23 | + |
| 24 | +The abstract from the paper is the following: |
| 25 | + |
| 26 | +*We introduce D-FINE, a powerful real-time object detector that achieves outstanding localization precision by redefining the bounding box regression task in DETR models. D-FINE comprises two key components: Fine-grained Distribution Refinement (FDR) and Global Optimal Localization Self-Distillation (GO-LSD). |
| 27 | +FDR transforms the regression process from predicting fixed coordinates to iteratively refining probability distributions, providing a fine-grained intermediate representation that significantly enhances localization accuracy. GO-LSD is a bidirectional optimization strategy that transfers localization knowledge from refined distributions to shallower layers through self-distillation, while also simplifying the residual prediction tasks for deeper layers. Additionally, D-FINE incorporates lightweight optimizations in computationally intensive modules and operations, achieving a better balance between speed and accuracy. Specifically, D-FINE-L / X achieves 54.0% / 55.8% AP on the COCO dataset at 124 / 78 FPS on an NVIDIA T4 GPU. When pretrained on Objects365, D-FINE-L / X attains 57.1% / 59.3% AP, surpassing all existing real-time detectors. Furthermore, our method significantly enhances the performance of a wide range of DETR models by up to 5.3% AP with negligible extra parameters and training costs. Our code and pretrained models: this https URL.* |
| 28 | + |
| 29 | +This model was contributed by [VladOS95-cyber](https://github.com/VladOS95-cyber). |
| 30 | +The original code can be found [here](https://github.com/Peterande/D-FINE). |
| 31 | + |
| 32 | +## Usage tips |
| 33 | + |
| 34 | +```python |
| 35 | +>>> import torch |
| 36 | +>>> from transformers.image_utils import load_image |
| 37 | +>>> from transformers import DFineForObjectDetection, AutoImageProcessor |
| 38 | + |
| 39 | +>>> url = 'http://images.cocodataset.org/val2017/000000039769.jpg' |
| 40 | +>>> image = load_image(url) |
| 41 | + |
| 42 | +>>> image_processor = AutoImageProcessor.from_pretrained("ustc-community/dfine_x_coco") |
| 43 | +>>> model = DFineForObjectDetection.from_pretrained("ustc-community/dfine_x_coco") |
| 44 | + |
| 45 | +>>> inputs = image_processor(images=image, return_tensors="pt") |
| 46 | + |
| 47 | +>>> with torch.no_grad(): |
| 48 | +... outputs = model(**inputs) |
| 49 | + |
| 50 | +>>> results = image_processor.post_process_object_detection(outputs, target_sizes=[(image.height, image.width)], threshold=0.5) |
| 51 | + |
| 52 | +>>> for result in results: |
| 53 | +... for score, label_id, box in zip(result["scores"], result["labels"], result["boxes"]): |
| 54 | +... score, label = score.item(), label_id.item() |
| 55 | +... box = [round(i, 2) for i in box.tolist()] |
| 56 | +... print(f"{model.config.id2label[label]}: {score:.2f} {box}") |
| 57 | +cat: 0.96 [344.49, 23.4, 639.84, 374.27] |
| 58 | +cat: 0.96 [11.71, 53.52, 316.64, 472.33] |
| 59 | +remote: 0.95 [40.46, 73.7, 175.62, 117.57] |
| 60 | +sofa: 0.92 [0.59, 1.88, 640.25, 474.74] |
| 61 | +remote: 0.89 [333.48, 77.04, 370.77, 187.3] |
| 62 | +``` |
| 63 | + |
| 64 | +## DFineConfig |
| 65 | + |
| 66 | +[[autodoc]] DFineConfig |
| 67 | + |
| 68 | +## DFineModel |
| 69 | + |
| 70 | +[[autodoc]] DFineModel |
| 71 | + - forward |
| 72 | + |
| 73 | +## DFineForObjectDetection |
| 74 | + |
| 75 | +[[autodoc]] DFineForObjectDetection |
| 76 | + - forward |
0 commit comments