Skip to content

Commit ec8bf72

Browse files
committed
release video benchmarks' models
1 parent f09ce01 commit ec8bf72

File tree

3 files changed

+190
-189
lines changed

3 files changed

+190
-189
lines changed

docs/en/model_zoos/traffic_benchmarks.md

Lines changed: 28 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Traffic Prediction Benchmarks
22

3-
**We provide benchmark results of spatiotemporal prediction learning (STL) methods on popular traffic prediction datasets. More STL methods will be supported in the future. Issues and PRs are welcome!** Currently, we only provide benchmark results, trained models and logs will be released soon (you can contact us if you require these files).
3+
**We provide benchmark results of spatiotemporal prediction learning (STL) methods on popular traffic prediction datasets. More STL methods will be supported in the future. Issues and PRs are welcome!** Currently, we only provide benchmark results, trained models and logs will be released soon (you can contact us if you require these files). You can download model files from [Baidu Cloud (3t2t)](https://pan.baidu.com/s/1dH3gS9pyl3SQP8mL2FBgoA?pwd=3t2t).
44

55
<details open>
66
<summary>Currently supported spatiotemporal prediction methods</summary>
@@ -43,45 +43,46 @@
4343

4444
## TaxiBJ Benchmarks
4545

46-
We provide traffic benchmark results on the popular [TaxiBJ](https://arxiv.org/abs/1610.00081) dataset using $4\rightarrow 4$ frames prediction setting. Metrics (MSE, MAE, SSIM, pSNR) of the the best models are reported in three trials. Parameters (M), FLOPs (G), and V100 inference FPS (s) are also reported for all methods. All methods are trained by Adam optimizer with Cosine Annealing scheduler (5 epochs warmup and min lr is 1e-6) and **single GPU**.
46+
We provide traffic benchmark results on the popular [TaxiBJ](https://arxiv.org/abs/1610.00081) dataset using $4\rightarrow 4$ frames prediction setting. Metrics (MSE, MAE, SSIM, pSNR) of the best models are reported in three trials. Parameters (M), FLOPs (G), and V100 inference FPS (s) are also reported for all methods. All methods are trained by Adam optimizer with Cosine Annealing scheduler (5 epochs warmup and min lr is 1e-6) and **single GPU**.
4747

4848
### **STL Benchmarks on TaxiBJ**
4949

5050
For a fair comparison of different methods, we report the best results when models are trained to convergence. We provide config files in [configs/taxibj](https://github.com/chengtan9907/OpenSTL/configs/taxibj).
5151

5252
| Method | Setting | Params | FLOPs | FPS | MSE | MAE | SSIM | PSNR | Download |
5353
|--------------|:--------:|:------:|:------:|:----:|:------:|:-----:|:------:|:-----:|:------------:|
54-
| ConvLSTM-S | 50 epoch | 14.98M | 20.74G | 815 | 0.3358 | 15.32 | 0.9836 | 39.45 | model \| log |
55-
| E3D-LSTM\* | 50 epoch | 50.99M | 98.19G | 60 | 0.3427 | 14.98 | 0.9842 | 39.64 | model \| log |
56-
| PhyDNet | 50 epoch | 3.09M | 5.60G | 982 | 0.3622 | 15.53 | 0.9828 | 39.46 | model \| log |
57-
| PredNet | 50 epoch | 12.5M | 0.85G | 5031 | 0.3516 | 15.91 | 0.9828 | 39.29 | model \| log |
58-
| PredRNN | 50 epoch | 23.66M | 42.40G | 416 | 0.3194 | 15.31 | 0.9838 | 39.51 | model \| log |
59-
| MIM | 50 epoch | 37.86M | 64.10G | 275 | 0.3110 | 14.96 | 0.9847 | 39.65 | model \| log |
60-
| MAU | 50 epoch | 4.41M | 6.02G | 540 | 0.3268 | 15.26 | 0.9834 | 39.52 | model \| log |
61-
| PredRNN++ | 50 epoch | 38.40M | 62.95G | 301 | 0.3348 | 15.37 | 0.9834 | 39.47 | model \| log |
62-
| PredRNN.V2 | 50 epoch | 23.67M | 42.63G | 378 | 0.3834 | 15.55 | 0.9826 | 39.49 | model \| log |
63-
| DMVFN | 50 epoch | 3.54M | 0.057G | 6347 | 0.3517 | 15.72 | 0.9833 | 39.33 | model \| log |
64-
| SimVP+IncepU | 50 epoch | 13.79M | 3.61G | 533 | 0.3282 | 15.45 | 0.9835 | 39.45 | model \| log |
65-
| SimVP+gSTA-S | 50 epoch | 9.96M | 2.62G | 1217 | 0.3246 | 15.03 | 0.9844 | 39.71 | model \| log |
66-
| TAU | 50 epoch | 9.55M | 2.49G | 1268 | 0.3108 | 14.93 | 0.9848 | 39.74 | model \| log |
54+
| ConvLSTM-S | 50 epoch | 14.98M | 20.74G | 815 | 0.3358 | 15.32 | 0.9836 | 39.45 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_convlstm_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_convlstm_cos_ep50.log) |
55+
| E3D-LSTM\* | 50 epoch | 50.99M | 98.19G | 60 | 0.3427 | 14.98 | 0.9842 | 39.64 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_e3dlstm_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_e3dlstm_cos_ep50.log) |
56+
| PhyDNet | 50 epoch | 3.09M | 5.60G | 982 | 0.3622 | 15.53 | 0.9828 | 39.46 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_phydnet_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_phydnet_cos_ep50.log) |
57+
| PredNet | 50 epoch | 12.5M | 0.85G | 5031 | 0.3516 | 15.91 | 0.9828 | 39.29 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_prednet_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_prednet_cos_ep50.log) |
58+
| PredRNN | 50 epoch | 23.66M | 42.40G | 416 | 0.3194 | 15.31 | 0.9838 | 39.51 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_predrnn_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_predrnn_cos_ep50.log) |
59+
| MIM | 50 epoch | 37.86M | 64.10G | 275 | 0.3110 | 14.96 | 0.9847 | 39.65 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_mim_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_mim_cos_ep50.log) |
60+
| MAU | 50 epoch | 4.41M | 6.02G | 540 | 0.3268 | 15.26 | 0.9834 | 39.52 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_mau_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_mau_cos_ep50.log) |
61+
| PredRNN++ | 50 epoch | 38.40M | 62.95G | 301 | 0.3348 | 15.37 | 0.9834 | 39.47 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_predrnnpp_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_predrnnpp_cos_ep50.log) |
62+
| PredRNN.V2 | 50 epoch | 23.67M | 42.63G | 378 | 0.3834 | 15.55 | 0.9826 | 39.49 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_predrnnv2_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_predrnnv2_cos_ep50.log) |
63+
| DMVFN | 50 epoch | 3.54M | 0.057G | 6347 | 0.3517 | 15.72 | 0.9833 | 39.33 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_dmvfn_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_dmvfn_cos_ep50.log) |
64+
| SimVP+IncepU | 50 epoch | 13.79M | 3.61G | 533 | 0.3282 | 15.45 | 0.9835 | 39.45 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_incepu_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_incepu_cos_ep50.log) |
65+
| SimVP+gSTA-S | 50 epoch | 9.96M | 2.62G | 1217 | 0.3246 | 15.03 | 0.9844 | 39.71 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_gsta_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_gsta_cos_ep50.log) |
66+
| TAU | 50 epoch | 9.55M | 2.49G | 1268 | 0.3108 | 14.93 | 0.9848 | 39.74 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_tau_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_tau_cos_ep50.log) |
6767

6868
### **Benchmark of MetaFormers on SimVP (MetaVP)**
6969

7070
Similar to [Moving MNIST Benchmarks](#moving-mnist-benchmarks), we benchmark popular Metaformer architectures on [SimVP](https://arxiv.org/abs/2211.12509) with training times of 50-epoch. We provide config files in [configs/taxibj/simvp](https://github.com/chengtan9907/OpenSTL/configs/taxibj/simvp/).
7171

7272
| MetaFormer | Setting | Params | FLOPs | FPS | MSE | MAE | SSIM | PSNR | Download |
7373
|------------------|:--------:|:------:|:-----:|:----:|:------:|:-----:|:------:|:-----:|:------------:|
74-
| SimVP+IncepU | 50 epoch | 13.79M | 3.61G | 533 | 0.3282 | 15.45 | 0.9835 | 39.45 | model \| log |
75-
| SimVP+gSTA-S | 50 epoch | 9.96M | 2.62G | 1217 | 0.3246 | 15.03 | 0.9844 | 39.71 | model \| log |
76-
| ViT | 50 epoch | 9.66M | 2.80G | 1301 | 0.3171 | 15.15 | 0.9841 | 39.64 | model \| log |
77-
| Swin Transformer | 50 epoch | 9.66M | 2.56G | 1506 | 0.3128 | 15.07 | 0.9847 | 39.65 | model \| log |
78-
| Uniformer | 50 epoch | 9.52M | 2.71G | 1333 | 0.3268 | 15.16 | 0.9844 | 39.64 | model \| log |
79-
| MLP-Mixer | 50 epoch | 8.24M | 2.18G | 1974 | 0.3206 | 15.37 | 0.9841 | 39.49 | model \| log |
80-
| ConvMixer | 50 epoch | 0.84M | 0.23G | 4793 | 0.3634 | 15.63 | 0.9831 | 39.41 | model \| log |
81-
| Poolformer | 50 epoch | 7.75M | 2.06G | 1827 | 0.3273 | 15.39 | 0.9840 | 39.46 | model \| log |
82-
| ConvNeXt | 50 epoch | 7.84M | 2.08G | 1918 | 0.3106 | 14.90 | 0.9845 | 39.76 | model \| log |
83-
| VAN | 50 epoch | 9.48M | 2.49G | 1273 | 0.3125 | 14.96 | 0.9848 | 39.72 | model \| log |
84-
| HorNet | 50 epoch | 9.68M | 2.54G | 1350 | 0.3186 | 15.01 | 0.9843 | 39.66 | model \| log |
85-
| MogaNet | 50 epoch | 9.96M | 2.61G | 1005 | 0.3114 | 15.06 | 0.9847 | 39.70 | model \| log |
74+
| SimVP+IncepU | 50 epoch | 13.79M | 3.61G | 533 | 0.3282 | 15.45 | 0.9835 | 39.45 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_incepu_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_incepu_cos_ep50.log) |
75+
| SimVP+gSTA-S | 50 epoch | 9.96M | 2.62G | 1217 | 0.3246 | 15.03 | 0.9844 | 39.71 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_gsta_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_gsta_cos_ep50.log) |
76+
| ViT | 50 epoch | 9.66M | 2.80G | 1301 | 0.3171 | 15.15 | 0.9841 | 39.64 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_vit_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_vit_cos_ep50.log) |
77+
| Swin Transformer | 50 epoch | 9.66M | 2.56G | 1506 | 0.3128 | 15.07 | 0.9847 | 39.65 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_swin_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_swin_cos_ep50.log) |
78+
| Uniformer | 50 epoch | 9.52M | 2.71G | 1333 | 0.3268 | 15.16 | 0.9844 | 39.64 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_uniformer_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_uniformer_cos_ep50.log) |
79+
| MLP-Mixer | 50 epoch | 8.24M | 2.18G | 1974 | 0.3206 | 15.37 | 0.9841 | 39.49 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_mlpmixer_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_mlpmixer_cos_ep50.log) |
80+
| ConvMixer | 50 epoch | 0.84M | 0.23G | 4793 | 0.3634 | 15.63 | 0.9831 | 39.41 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_convmixer_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_convmixer_cos_ep50.log) |
81+
| Poolformer | 50 epoch | 7.75M | 2.06G | 1827 | 0.3273 | 15.39 | 0.9840 | 39.46 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_poolformer_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_poolformer_cos_ep50.log) |
82+
| ConvNeXt | 50 epoch | 7.84M | 2.08G | 1918 | 0.3106 | 14.90 | 0.9845 | 39.76 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_convnext_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_convnext_cos_ep50.log) |
83+
| VAN | 50 epoch | 9.48M | 2.49G | 1273 | 0.3125 | 14.96 | 0.9848 | 39.72 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_van_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_van_cos_ep50.log) |
84+
| HorNet | 50 epoch | 9.68M | 2.54G | 1350 | 0.3186 | 15.01 | 0.9843 | 39.66 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_hornet_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_hornet_cos_ep50.log) |
85+
| MogaNet | 50 epoch | 9.96M | 2.61G | 1005 | 0.3114 | 15.06 | 0.9847 | 39.70 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_moganet_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_simvp_moganet_cos_ep50.log) |
86+
| TAU | 50 epoch | 9.55M | 2.49G | 1268 | 0.3108 | 14.93 | 0.9848 | 39.74 | [model](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_tau_cos_ep50.pth) \| [log](https://github.com/chengtan9907/OpenSTL/releases/download/taxibj-weights/taxibj_tau_cos_ep50.log) |
8687

8788
<p align="right">(<a href="#top">back to top</a>)</p>

0 commit comments

Comments
 (0)