BST: Badminton Stroke-type Transformer for Skeleton-based Action Recognition in Racket Sports

Poster

Inference

I've provided an example of inferring code in stroke_classification/main_on_shuttleset/bst_infer.py.

Caution

Make sure your input data is normalized in the same way as in stroke_classification/preparing_data/prepare_train_on_<dataset_name>.py.

• normalize_shuttlecock
  • Normalized by the video resolution.
  • Each element is in the range [0, 1].
• normalize_joints
  • Use relative coordinates within the player's bounding box, with its top-left corner as the origin.
  • Normalized by the diagonal distance of the player's bounding box by default.
  • center_align default is True. $\Rightarrow$ Make the center of the bounding box to be the new origin.
  • Each element is in the range [-0.X, 0.X].
• normalize_position
  • Convert the points of feet from camera coordinates to court coordinates.
  • Calculate the midpoint between them.
  • Normalized by court boundary (in court coordinate, which is a rectangle).
  • Each element is in the range [0, 1] on a badminton dataset. For a tennis dataset, it may slightly out of this range.

Weights

• Weights trained on ShuttleSet
• Weights trained on BadmintonDB
• Weights trained on TenniSet

Tip

How to choose a great weight file? Please see results_and_images/result_table.xlsx.

Reproducing the Results

There are three main steps:

1. Generating Stroke Clips
2. Data Preprocessing
3. Training Classification Models

Since the process before training is time-consuming, I have provided the non-collated .npy files (use the last step in stroke_classification/preparing_data/prepare_train_on_<dataset_name>.py to collate these .npy files) in stroke_classification/preparing_data folders:

• ShuttleSet_data_merged (25 classes)
  • (seq_len=30) dataset_npy
  • (seq_len=30) dataset_3d_npy
  • (seq_len=100) dataset_npy_between_2_hits_with_max_limits
• ShuttleSet_data (35 classes)
  • (seq_len=30) dataset_npy
  • (seq_len=30) dataset_3d_npy
  • (seq_len=100) dataset_npy_between_2_hits_with_max_limits
• BadmintonDB_data
  • (seq_len=72) dataset_npy
• TenniSet_data
  • (seq_len=100) dataset_npy

Generating Stroke Clips

ShuttleSet (35 classes)

(In ShuttleSet folder)

1. Download the videos from links in set/match.csv and put them into raw_video folder.
   • flaw_shot_records.csv shows some errors in the original ShuttleSet, so please use the modified version in this repository.
   • Append each video ID to the front of its name, such as <ID> - <name>.mp4.
2. Modify my_raw_video_resolution.csv to the resolutions you downloaded.
3. Run gen_my_dataset.py 6 (2 players x 3 splitted sets) times, and please check the following settings in the code each time:
   • out_root_dir: output root directory.
   • player: Top / Bottom.
   • set_name: train / val / test.
   • type_ls: each element should be the same class name as in ShuttleSet.
   • strategy: please check if it matches the out_root_dir or not.
4. Remove all the clips labeled "removed" in flaw_shot_records.csv.

Important

Before the next step, please make a copy of Top_小平球, Bottom_小平球 and 未知球種 folders to another place, if you want to generate ShuttleSet (25 classes) later.

5. Move all clips in Top_小平球 or Bottom_小平球 to 未知球種, which means "none" type, and delete these empty folders.
6. Make sure the total number of the clips each type is the same as that shown in class_total.xlsx.

ShuttleSet (25 classes)

(In ShuttleSet_merged folder)

Tip

We don't need to generate the clips again if we have done the generating process for ShuttleSet (35 classes).

Go check the total number of the clips each merged type in class_total.xlsx.

BadmintonDB (18 classes)

(In BadmintonDB folder)

1. Download the videos from links in their repository BadmintonDB and put them into raw_video folder.
2. Modify my_raw_video_resolution.csv to the resolutions you downloaded.
3. Run gen_my_dataset.py.

TenniSet (6 classes)

(In TenniSet folder)

1. Download the videos from their repository TenniSet and put them into videos folder.
2. Run gen_my_dataset.py.

Note

Since they provide video files directly, the downloaded video resolutions must be the same. We don't need my_raw_video_resolution.csv here.

Data Preprocessing

1. (Only TenniSet needs) Make sure your TennisCourtDetector works well.

2. Make sure your shuttlecock tracking model works well, you can choose the one you like:

   • TrackNetV3 (using attension) $\Leftarrow$ used in this paper.
   • TrackNetV3 (with rectification module).

3. Make sure your MMPose works well.

   • Set your venv_prepare_train virtual environment first for running Human Pose Estimation (HPE) successfully.
   • See prepare_train_env.txt as an example based on my setup.

4. Enter stroke_classification folder, and activate your venv_prepare_train virtual environment.

   cd stroke_classification
   preparing_data/venv_prepare_train/Scripts/activate
   

5. Run 3 or 4 steps in prepare_train_on_<dataset_name>.py individually.

   python preparing_data/prepare_train_on_<dataset_name>.py
   

   1. Run court detection (only for TenniSet).
   2. Run ball tracking model.
      • Check the parameters are being passed into your tracking model.
   3. Run HPE model and save 3 kinds of inputs for classification model from each clip to .npy files.
      • I don't recommend using 3D poses because there are still some bugs using MMPose 3D pose API.
   4. Collate these .npy files into several big .npy files.
      • Set seq_len properly for the specific dataset.

6. Deactivate your venv_prepare_train virtual environment.

   deactivate
   

Thus, we have collated .npy files now.

Training Classification Models

(In stroke_classification folder)

Run each <architecture_name>_main.py for training, validating, testing each model, and please check hyper-parameters:

python main_on_<dataset_name>/<architecture_name>_main.py

• hyp.n_classes: should be the same as in your dataset.
• hyp.seq_len: should be the same as in your dataset.
• hyp.pose_style
  • J_only: modality J in this paper.
  • JnB_bone: modality J+B in this paper.
• use_3d_pose: False for 2D, True for 3D.
• train_partial: 1 for the whole training set.
• model_name
  • BST_0: BST-0 in this paper.
  • BST_CG: BST-CG in this paper.
  • BST_AP: BST-AP in this paper.
  • BST_CG_AP: BST-CG-AP in this paper.
• additional_model_info: to distinguish your model weights.
• serial_no: to distinguish your model weights when trying different initial weights.

Note

BST-CG is identical to BST-1 from the arXiv preprint v2.
BST-AP is identical to BST-2 from the arXiv preprint v2.
BST-CG-AP is identical to BST-3 from the arXiv preprint v2.

Citation

@InProceedings{Chang_2026_CVPR,
    author    = {Chang, Jing-Yuan},
    title     = {BST: Badminton Stroke-type Transformer for Skeleton-based Action Recognition in Racket Sports},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2026},
    pages     = {9889-9898}
}

References

• Shuttlecock Trajectory
  • TrackNetV3 (using attention)
  • TrackNetV3 (with rectification module)
• Human Pose Estimation
  • MMPose
• Court Detection (if the dataset does not contain court information)
  • MonoTrack
  • TennisCourtDetector
• Classification Models
  • ST-GCN
  • BlockGCN
  • SkateFormer
  • ProtoGCN
  • TemPose
• Dataset
  • ShuttleSet
  • BadmintonDB
  • TenniSet