Bidirectional Collaborative Mentoring Network for Marine Organism Detection and Beyond

Authors: Jinguang Cheng, Zongwei Wu, Shuo Wang, Cédric Demonceaux, and Qiuping Jiang.

1. Preface

• This repository provides code for "Bidirectional Collaborative Mentoring Network for Marine Organism Detection and Beyond" TCSVT-2023. (paper)

2. Overview

2.1. Introduction

Organism detection plays a vital role in marine resource exploitation and the marine economy. How to accurately locate the target object within the camouflaged and dark light oceanic scene has recently drawn great attention in the research community. Existing learning-based works usually leverage local texture details within a neighboring area, with few methods explicitly exploring the usage of contextualized awareness for accurate object detection. From a novel perspective, we present a Bidirectional Collaborative Mentoring Network (BCMNet) which fully explores both texture and context clues during the encoding and decoding stages, making the cross-paradigm interaction bidirectional and improving the scene understanding at all stages. Specifically, we first extract texture and context features through a dual branch encoder and attentively fuse them through our adjacent feature fusion (AFF) block. Then, we propose a structure-aware module (SAM) and a detailenhanced module (DEM) to form our two-stage decoding pipeline. On the one hand, our SAM leverages both local and global clues to preserve morphological integrity and generate an initial prediction of the target object. On the other hand, the DEM explicitly explores long-range dependencies to refine the initially predicted object mask further. The combination of SAM and DEM enables better extracting, preserving, and enhancing the object morphology, making it easier to segment the target object from the camouflaged background with sharp contour. Extensive experiments on three benchmark datasets show that our proposed BCMNet performs favorably over state-of-the-art models.

2.2. Framework Overview

Figure 1: Architecture Overview. Our BCMNet consists of a dual-branch encoder (Sec. III-B), feature fusion module AFF (Sec. III-C), an initial decoder based on SAM (Sec. III-D), and a refiner based on DEM (Sec. III-E). During feature extraction, each encoder has its specific purposes, i.e., the texture encoder analyzes local details based on colour information, while the context encoder leverages the long-range dependencies for contextualized awareness. To fuse the encoded features, we propose an AFF module to aggregate features from different paradigms and scales. Then we introduce the SAM to partially decode the feature and generate the initial prediction based on the object's external shape. Finally, the partial mask is further refined by enhancing the awareness of local fine-grained details through DEM. The initial and final predictions are both supervised by the ground truth mask, making our network end-to-end trainable.

2.3. Qualitative Results

Figure 2: Qualitative Results.

3. Proposed Baseline

3.1. Training/Testing

The training and testing experiments are conducted using PyTorch with double 2080Ti GPU of 48 GB Memory.

1. Configuring your environment (Prerequisites):

   • Creating a virtual environment in terminal: conda create -n C2FNet python=3.8.

   • Installing necessary packages: pip install -r requirements.txt.

2. Downloading necessary data:

   • downloading training/testing dataset and move it into ./data/, which can be found in this (Google Drive) or (BaiduNetdisk) (Extracted code：BCMN).

   • downloading pretrained weights and move it into ./checkpoints/BCMNet.pth, which can be found in this (Google Drive) or (BaiduNetdisk) (Extracted code：BCMN).

   • downloading ResNet weights and move it into ./models/res2net50_v1b_26w_4s-3cf99910.pth(Google Drive) or (BaiduNetdisk) (Extracted code：BCMN).

3. Training Configuration:

   • Assigning your costumed path, like --train_save and --train_path in MyTrain.py.
   • I modify the total epochs and the learning rate decay method (lib/utils.py has been updated), so there are differences from the training setup reported in the paper. Under the new settings, the training performance is more stable.

4. Testing Configuration:

   • After you download all the pre-trained model and testing dataset, just run MyTest.py to generate the final prediction map: replace your trained model directory (--pth_path).

3.2 Evaluating your trained model:

One-key evaluation is written in python code (revised from link)

4. Citation

⬆ back to top