Is YoloV8 + SAHI capable of real-time?

[whittenator]

Is SAHI + YoloV8 capable of real-time? Using just YoloV8 pytorch I am able to run inference at 50ms per frame(Could be much faster if I converted to TensorRT or ONNX). Running the code below I am seeing 358ms per frame with SAHI:

from sahi import AutoDetectionModel
from sahi.utils.cv import read_image
from sahi.utils.file import download_from_url
from sahi.predict import get_prediction, get_sliced_prediction, predict

yolov8_model_path = "/home/ubuntu/yolov8-training/XOD/xod-yolov8-anthony-89541-800x800-UFO3/weights/best.pt"

detection_model = AutoDetectionModel.from_pretrained(
    model_type='yolov8',
    model_path=yolov8_model_path,
    confidence_threshold=0.3,
    device="cuda:0", # or 'cpu'
)

video_path = "/home/ubuntu/yolov8-training/test_videos/IRSmallDrone.mp4"

result = predict(detection_model=detection_model, source=video_path, project="XOD", name="SAHI-Test", view_video=False, novisual=False, postprocess_type="NMS", image_size=800, model_device="cuda:0")

The video is only 30 seconds long, yet it's taking well over 10 minutes to process through with an Nvidia T4 GPU. If I run YoloV8 alone I am getting 50ms per frame, which is much closer to real-time. Any ideas?

[BabyHuy17]

How do you use SAHI with converted tensorrt or onnx file?