Skip to content

Commit 65a9a2c

Browse files
fengshiwestfengshiwest
authored
[Feature] Semi automated with label studio (#815)
* [doc]:add more social network links * [doc]:add more social network links * [Feature] support semi-automatic annotation base Label-Studio * [fix] change the name and url to mmrotate * [fix] resolve conflict
1 parent 542457c commit 65a9a2c

8 files changed

Lines changed: 660 additions & 1 deletion

File tree

‎docs/en/user_guides/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,3 +24,4 @@ Useful Tools
2424
useful_tools.md
2525
visualization.md
2626
deploy.md
27+
label_studio.md

‎docs/en/user_guides/label_studio.md‎

Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,174 @@
1+
# Semi-automatic Object Detection Annotation with MMRotate and Label-Studio
2+
3+
Annotation data is a time-consuming and laborious task. This article introduces how to perform semi-automatic annotation using the RTMDet-R algorithm in MMRotate in conjunction with Label-Studio software. Specifically, using RTMDet-R to predict image annotations and then refining the annotations with Label-Studio. Community users can refer to this process and methodology and apply it to other fields.
4+
5+
- RTMDet-R: RTMDet-R is a high-precision single-stage object detection algorithm developed by OpenMMLab, open-sourced in the MMRotate rotated object detection toolbox. Its open-source license is Apache 2.0, and it can be used freely without restrictions by industrial users.
6+
7+
- [Label Studio](https://github.com/heartexlabs/label-studio) is an excellent annotation software covering the functionality of dataset annotation in areas such as image classification, object detection, and segmentation.
8+
9+
In this article, we will use [DOTA](https://captain-whu.github.io/DOTA/index.html) images for semi-automatic annotation.
10+
11+
## Environment Configuration
12+
13+
To begin with, you need to create a virtual environment and then install PyTorch and MMCV. In this article, we will specify the versions of PyTorch and MMCV. Next, you can install MMDetection, MMRotate, Label-Studio, and label-studio-ml-backend using the following steps:
14+
15+
Create a virtual environment:
16+
17+
```shell
18+
conda create -n rtmdet python=3.9 -y
19+
conda activate rtmdet
20+
```
21+
22+
Install PyTorch:
23+
24+
```shell
25+
# Linux and Windows CPU only
26+
pip install torch==1.10.1+cpu torchvision==0.11.2+cpu torchaudio==0.10.1 -f https://download.pytorch.org/whl/cpu/torch_stable.html
27+
# Linux and Windows CUDA 11.3
28+
pip install torch==1.10.1+cu113 torchvision==0.11.2+cu113 torchaudio==0.10.1 -f https://download.pytorch.org/whl/cu113/torch_stable.html
29+
# OSX
30+
pip install torch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1
31+
```
32+
33+
Install MMCV:
34+
35+
```shell
36+
pip install -U openmim
37+
mim install "mmcv>=2.0.0rc0"
38+
# Installing mmcv will automatically install mmengine
39+
```
40+
41+
Install MMDetection:
42+
43+
```shell
44+
git clone https://github.com/open-mmlab/mmdetection -b dev-3.x
45+
cd mmdetection
46+
pip install -v -e .
47+
```
48+
49+
Install MMRotate:
50+
51+
```shell
52+
git clone https://github.com/open-mmlab/mmrotate/ -b dev-1.x
53+
cd mmrotate
54+
pip install -v -e .
55+
```
56+
57+
Install Label-Studio and label-studio-ml-backend:
58+
59+
```shell
60+
# Installing Label-Studio may take some time, if the version is not found, please use the official source
61+
pip install label-studio==1.7.2
62+
pip install label-studio-ml==1.0.9
63+
```
64+
65+
Download the RTMDet-R weights:
66+
67+
```shell
68+
cd path/to/mmrotate
69+
mkdir work_dirs
70+
cd work_dirs
71+
wget https://download.openmmlab.com/mmrotate/v1.0/rotated_rtmdet/rotated_rtmdet_m-3x-dota/rotated_rtmdet_m-3x-dota-beeadda6.pth
72+
```
73+
74+
## Start the Service
75+
76+
Start the RTMDet-R backend inference service:
77+
78+
```shell
79+
cd path/to/mmrotate
80+
label-studio-ml start projects/LabelStudio/backend_template --with \
81+
config_file=configs/rotated_rtmdet/rotated_rtmdet_m-3x-dota.py \
82+
checkpoint_file=./work_dirs/rotated_rtmdet_m-3x-dota-beeadda6.pth \
83+
device=cpu \
84+
--port 8003
85+
# Set device=cpu to use CPU inference, and replace cpu with cuda:0 to use GPU inference.
86+
```
87+
88+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/label_studio_ml_start.png)
89+
90+
The RTMDet-R backend inference service has now been started. To configure it in the Label-Studio web system, use http://localhost:8003 as the backend inference service.
91+
92+
Now, start the Label-Studio web service:
93+
94+
```shell
95+
label-studio start
96+
```
97+
98+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/label_studio_start.png)
99+
100+
Open your web browser and go to http://localhost:8080/ to see the Label-Studio interface.
101+
102+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/sign_up.png)
103+
104+
Register a user and then create an RTMDet-R-Semiautomatic-Label project.
105+
106+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/create_project.png)
107+
108+
Download the example DOTA images by following [MMRotate Preparing DOTA Dataset](https://github.com/open-mmlab/mmrotate/blob/main/tools/data/dota/README.md) and import them using the Data Import button.
109+
110+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/data_import.png)
111+
112+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/data_import_from_file.png)
113+
114+
Then, select the Object Detection With Bounding Boxes template.
115+
116+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/templete_select.png)
117+
118+
```shell
119+
plane
120+
ship
121+
storage-tank
122+
baseball-diamond
123+
tennis-court
124+
basketball-court
125+
ground-track-field
126+
harbor
127+
bridge
128+
large-vehicle
129+
small-vehicle
130+
helicopter
131+
roundabout
132+
soccer-ball-field
133+
swimming-pool
134+
```
135+
136+
Then, copy and add the above categories to Label-Studio and click Save.
137+
138+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/add_label.png)
139+
140+
In the Settings, click Add Model to add the RTMDet-R backend inference service.
141+
142+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/add_model.png)
143+
144+
Click Validate and Save, and then click Start Labeling.
145+
146+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/validate_and_save.png)
147+
148+
If you see Connected as shown below, the backend inference service has been successfully added.
149+
150+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/connected.png)
151+
152+
## Start Semi-Automatic Labeling
153+
154+
Click on Label to start labeling.
155+
156+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/click_label.png)
157+
158+
We can see that the RTMDet-R backend inference service has successfully returned the predicted results and displayed them on the image. However, we noticed that there are some missed predicted bounding boxes.
159+
160+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/label_result.png)
161+
162+
We manually add some annotation boxes, adjust the position of the boxes to get the following corrected annotation, and then click Submit to complete the annotation of this image.
163+
164+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/label_result_refined.png)
165+
166+
After submitting all images, click export to export the labeled dataset in COCO format.
167+
168+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/label_export.png)
169+
170+
Use VSCode to open the unzipped folder to see the labeled dataset, which includes the images and the annotation files in JSON format.
171+
172+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/json_show.png)
173+
174+
At this point, the semi-automatic labeling is complete. We can use this dataset to train a more accurate model in MMRotate and then continue semi-automatic labeling on newly collected images with this model. This way, we can iteratively expand the high-quality dataset and improve the accuracy of the model.

‎docs/zh_cn/user_guides/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,3 +24,4 @@ MMRotate 提供了几十个预训练模型 `Model Zoo <https://mmrotate.readthed
2424
useful_tools.md
2525
visualization.md
2626
deploy.md
27+
label_studio.md

‎docs/zh_cn/user_guides/label_studio.md‎

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
# 使用 MMRotate 和 Label-Studio 进行半自动化目标检测标注
2+
3+
标注数据是一个费时费力的任务，本文介绍了如何使用 MMRotate 中的 RTMDet-R 算法联合 Label-Studio 软件进行半自动化标注。具体来说，使用 RTMDet-R 预测图片生成标注，然后使用 Label-Studio 进行微调标注，社区用户可以参考此流程和方法，将其应用到其他领域。
4+
5+
- RTMDet-R：RTMDet-R 是 OpenMMLab 自研的高精度单阶段的目标检测算法，开源于 MMRotate 旋转目标检测工具箱中，其开源协议为 Apache 2.0，工业界的用户可以不受限的免费使用。
6+
- [Label Studio](https://github.com/heartexlabs/label-studio) 是一款优秀的标注软件，覆盖图像分类、目标检测、分割等领域数据集标注的功能。
7+
8+
本文将使用[DOTA数据集](https://captain-whu.github.io/DOTA/index.html)的图片，进行半自动化标注。
9+
10+
## 环境配置
11+
12+
首先需要创建一个虚拟环境，然后安装 PyTorch 和 MMCV。在本文中，我们将指定 PyTorch 和 MMCV 的版本。接下来安装 MMDetection、MMRotate、Label-Studio 和 label-studio-ml-backend，具体步骤如下：
13+
14+
创建虚拟环境：
15+
16+
```shell
17+
conda create -n rtmdet python=3.9 -y
18+
conda activate rtmdet
19+
```
20+
21+
安装 PyTorch
22+
23+
```shell
24+
# Linux and Windows CPU only
25+
pip install torch==1.10.1+cpu torchvision==0.11.2+cpu torchaudio==0.10.1 -f https://download.pytorch.org/whl/cpu/torch_stable.html
26+
# Linux and Windows CUDA 11.3
27+
pip install torch==1.10.1+cu113 torchvision==0.11.2+cu113 torchaudio==0.10.1 -f https://download.pytorch.org/whl/cu113/torch_stable.html
28+
# OSX
29+
pip install torch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1
30+
```
31+
32+
安装 MMCV
33+
34+
```shell
35+
pip install -U openmim
36+
mim install "mmcv>=2.0.0rc0"
37+
# 安装 mmcv 的过程中会自动安装 mmengine
38+
```
39+
40+
安装 MMDetection
41+
42+
```shell
43+
git clone https://github.com/open-mmlab/mmdetection -b dev-3.x
44+
cd mmdetection
45+
pip install -v -e .
46+
```
47+
48+
安装 MMRotate
49+
50+
```shell
51+
git clone https://github.com/open-mmlab/mmrotate/ -b dev-1.x
52+
cd mmrotate
53+
pip install -v -e .
54+
```
55+
56+
安装 Label-Studio 和 label-studio-ml-backend
57+
58+
```shell
59+
# 安装 label-studio 需要一段时间,如果找不到版本请使用官方源
60+
pip install label-studio==1.7.2
61+
pip install label-studio-ml==1.0.9
62+
```
63+
64+
下载 RTMDet-R 权重
65+
66+
```shell
67+
cd path/to/mmrotate
68+
mkdir work_dirs
69+
cd work_dirs
70+
wget https://download.openmmlab.com/mmrotate/v1.0/rotated_rtmdet/rotated_rtmdet_m-3x-dota/rotated_rtmdet_m-3x-dota-beeadda6.pth
71+
```
72+
73+
## 启动服务
74+
75+
启动 RTMDet-R 后端推理服务：
76+
77+
```shell
78+
cd path/to/mmrotate
79+
label-studio-ml start projects/LabelStudio/backend_template --with \
80+
config_file=configs/rotated_rtmdet/rotated_rtmdet_m-3x-dota.py \
81+
checkpoint_file=./work_dirs/rotated_rtmdet_m-3x-dota-beeadda6.pth \
82+
device=cpu \
83+
--port 8003
84+
# device=cpu 为使用 CPU 推理，如果使用 GPU 推理，将 cpu 替换为 cuda:0
85+
```
86+
87+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/label_studio_ml_start.png)
88+
89+
此时，RTMDet-R 后端推理服务已经启动，后续在 Label-Studio Web 系统中配置 http://localhost:8003 后端推理服务即可。
90+
91+
现在启动 Label-Studio 网页服务：
92+
93+
```shell
94+
label-studio start
95+
```
96+
97+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/label_studio_start.png)
98+
99+
打开浏览器访问 [http://localhost:8080/](http://localhost:8080/) 即可看到 Label-Studio 的界面。
100+
101+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/sign_up.png)
102+
103+
我们注册一个用户，然后创建一个 RTMDet-R-Semiautomatic-Label 项目。
104+
105+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/create_project.png)
106+
107+
我们按照 [MMRotate Preparing DOTA Dataset](https://github.com/open-mmlab/mmrotate/blob/main/tools/data/dota/README.md) 下载好示例的DOTA图片，点击 Data Import 导入需要标注的图片。
108+
109+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/data_import.png)
110+
111+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/data_import_from_file.png)
112+
113+
然后选择 Object Detection With Bounding Boxes 模板。
114+
115+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/templete_select.png)
116+
117+
```shell
118+
plane
119+
ship
120+
storage-tank
121+
baseball-diamond
122+
tennis-court
123+
basketball-court
124+
ground-track-field
125+
harbor
126+
bridge
127+
large-vehicle
128+
small-vehicle
129+
helicopter
130+
roundabout
131+
soccer-ball-field
132+
swimming-pool
133+
```
134+
135+
然后将上述类别复制添加到 Label-Studio，然后点击 Save。
136+
137+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/add_label.png)
138+
139+
然后在设置中点击 Add Model 添加 RTMDet-R 后端推理服务。
140+
141+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/add_model.png)
142+
143+
点击 Validate and Save，然后点击 Start Labeling。
144+
145+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/validate_and_save.png)
146+
147+
看到如下 Connected 就说明后端推理服务添加成功。
148+
149+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/connected.png)
150+
151+
## 开始半自动化标注
152+
153+
点击 Label 开始标注
154+
155+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/click_label.png)
156+
157+
我们可以看到 RTMDet-R 后端推理服务已经成功返回了预测结果并显示在图片上，我们可以发现存在一些漏检的情况。
158+
159+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/label_result.png)
160+
161+
我们手工添加一些标注框，并修正一下框的位置，得到以下修正过后的标注，然后点击 Submit，本张图片就标注完毕了。
162+
163+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/label_result_refined.png)
164+
165+
我们 submit 完毕所有图片后，点击 export 导出 COCO 格式的数据集，就能把标注好的数据集的压缩包导出来了。
166+
167+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/label_export.png)
168+
169+
用 VSCode 打开解压后的文件夹，可以看到标注好的数据集，包含了图片和 json 格式的标注文件。
170+
171+
![](https://github.com/fengshiwest/mmrotate/blob/add_image/projects/LabelStudio/images/json_show.png)
172+
173+
到此半自动化标注就完成了，我们可以用这个数据集在 MMRotate 训练精度更高的模型了，训练出更好的模型，然后再用这个模型继续半自动化标注新采集的图片，这样就可以不断迭代，扩充高质量数据集，提高模型的精度。

0 commit comments

Comments
 (0)