Skip to content

Commit 071e870

Browse files
committed
update readme
1 parent 82b8bdc commit 071e870

1 file changed

Lines changed: 67 additions & 5 deletions

File tree

  • src/sailsprep/tracking_pose_model_testing

src/sailsprep/tracking_pose_model_testing/README.md

Lines changed: 67 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -16,20 +16,37 @@ bytetrack.py YOLO-Pose + Ultralytics ByteTrack
1616
entitysam.py EntitySAM-based video segmentation
1717
mediapipe_holistic.py MediaPipe Holistic (pose + face + hands)
1818
face_mediapipe.py YOLO person detection + MediaPipe Face Mesh
19+
movenet.py MoveNet (TF-Hub) single-person pose w/ adaptive cropping
20+
poseformer.py PoseFormer 3D pose (delegates to an external poseformer_demo repo)
21+
openpose_video.py OpenPose (pyopenpose) body + face + hands, CSV-batch
22+
openpifpaf.py OpenPifPaf multi-person pose estimation
23+
rtmlib.py RTMLib Wholebody pose (ONNX/OpenVINO, no mm-stack)
24+
rtmpose.py RTMPose wholebody pose (RTMDet + mmpose)
25+
sam2_yolov8.py YOLOv8 detection + SAM2 video segmentation/tracking
26+
deva.py DEVA + Grounding DINO text-prompted segmentation/tracking
27+
efficientpose.py EfficientPose (Keras/TF/TFLite/PyTorch) pose estimation
1928
```
2029

2130
## Single-video demo scripts
2231

23-
`deepsort.py`, `deepsort_reid.py`, and `bytetrack.py` hardcode
24-
`video_path = "video.mp4"` (and `output_folder`/`output_path`) at the top of
25-
their `main()`. Edit that path and run directly:
32+
`deepsort.py`, `deepsort_reid.py`, `bytetrack.py`, `movenet.py`, and
33+
`poseformer.py` hardcode a single input video path (`video_path` /
34+
`video_name`, and `output_folder`/`output_path`) near the top of the file.
35+
Edit that path and run directly:
2636

2737
```bash
2838
python deepsort.py
2939
python deepsort_reid.py
3040
python bytetrack.py
41+
python movenet.py
42+
python poseformer.py
3143
```
3244

45+
`poseformer.py` only trims the input to 10s with ffmpeg and copies the
46+
result out of an external `poseformer_demo` checkout (`/content/poseformer_demo`
47+
in the script) — that repo's own demo pipeline does the actual 3D pose
48+
inference and must be run separately.
49+
3350
## CSV-batch scripts
3451

3552
`yolo_pose.py`, `mediapipe_holistic.py`, and `face_mediapipe.py` read a
@@ -42,6 +59,35 @@ python mediapipe_holistic.py
4259
python face_mediapipe.py
4360
```
4461

62+
`openpose_video.py` follows the same CSV/`BidsProcessed` pattern but also
63+
accepts optional positional overrides for `csv_path`, `output_dir`, and
64+
`model_folder`:
65+
66+
```bash
67+
python openpose_video.py [csv_path] [output_dir] [model_folder]
68+
```
69+
70+
## Folder-batch scripts
71+
72+
`openpifpaf.py`, `rtmlib.py`, `rtmpose.py`, `sam2_yolov8.py`, `deva.py`, and
73+
`efficientpose.py` hardcode an input folder and output folder near the top
74+
of the file and process every video (or, for `efficientpose.py`, every
75+
`.mp4`) found there:
76+
77+
```bash
78+
python openpifpaf.py
79+
python rtmlib.py
80+
python rtmpose.py
81+
python sam2_yolov8.py
82+
python deva.py
83+
python efficientpose.py
84+
```
85+
86+
`sam2_yolov8.py` uses YOLOv8 to detect people on one anchor frame per video,
87+
then propagates SAM2 video segmentation from those boxes across the clip.
88+
`deva.py` uses Grounding DINO with a fixed `'person'` text prompt to drive
89+
DEVA's text-conditioned segmentation/tracking.
90+
4591
## SLURM array batch scripts
4692

4793
`hrnet.py` and `vit_pose.py` share the same interface — a split CSV plus an
@@ -85,7 +131,23 @@ Corresponding SLURM job scripts for all nine methods live under
85131

86132
Covered by a mix of Poetry groups depending on the method: `tracking`
87133
(ultralytics, deep-sort-realtime, mediapipe), `pose-estimation` /
88-
`clip_tracker` (mmcv/mmdet/mmpose, used by `hrnet.py`), `vitpose`
89-
(transformers-based ViTPose), and `entity-sam` (SAM2 + panopticapi).
134+
`clip_tracker` (mmcv/mmdet/mmpose, used by `hrnet.py` and `rtmpose.py`),
135+
`vitpose` (transformers-based ViTPose), and `entity-sam` (SAM2 +
136+
panopticapi, used by `entitysam.py` and `sam2_yolov8.py`).
90137
`deepsort_reid.py` additionally needs `torchreid`, which is not covered by
91138
any Poetry group and must be installed separately.
139+
140+
None of the remaining new scripts are covered by a Poetry group; each
141+
depends on a separate external install:
142+
143+
- `movenet.py``tensorflow`, `tensorflow_hub`, `tensorflow_docs`, `imageio`
144+
- `openpifpaf.py``openpifpaf`
145+
- `rtmlib.py``rtmlib` (ONNX/OpenVINO runtime, no mm-stack needed)
146+
- `deva.py` — a `Tracking-Anything-with-DEVA` checkout plus Grounding DINO
147+
(or `GroundingDINO`) and its SAM checkpoints
148+
- `openpose_video.py` — a built `openpose` install exposing `pyopenpose`,
149+
plus `ffmpeg` on `PATH` for H.264 encoding (falls back to OpenCV X264/mp4v)
150+
- `poseformer.py` — an external `poseformer_demo` checkout and `ffmpeg`
151+
- `efficientpose.py``pymediainfo`, `scikit-video` (`skvideo`), and
152+
whichever of `tensorflow`/`torch` matches the chosen framework, plus the
153+
corresponding EfficientPose model weights under `models/`

0 commit comments

Comments
 (0)