Classify the object that crossed the line

[Xkim2000]

Hello,
Before all congratulations on your project.
I want to know what type of object that crossed the line in that moment. How can i do that ?
My Code:

import cv2
from ultralytics import YOLO
import supervision as sv
import numpy as np

START = sv.Point(320,0)
END = sv.Point(320,480)



def main():
    model = YOLO("yolov8l.pt") 
    model.to('cuda')

    line_counter = sv.LineZone(start=START, end=END)
    line_annotator = sv.LineZoneAnnotator(
        thickness=2, 
        text_thickness=1, 
        text_scale=0.5
    )

    CLASS_NAMES_DICT = model.model.names

    box_annotator = sv.BoxAnnotator(
        thickness=2,
        text_thickness=1,
        text_scale=0.5
    )

    out_count = 0
    in_count = 0


    for result in model.track(source=0, show=True, stream=True):
        frame = result.orig_img
        detections = sv.Detections.from_yolov8(result)

        if result.boxes.id is not None:
            detections.tracker_id = result.boxes.id.cpu().numpy().astype(int)
           
        
        # Only keep detections with class_id 0, 1, 3
        detections = detections[np.isin(detections.class_id, [0, 1, 3])]

        labels = [
            f"#{tracker_id} {CLASS_NAMES_DICT[class_id]} {confidence:0.2f}"
            for _, _, confidence, class_id, tracker_id
            in detections
        ]

        frame = box_annotator.annotate(scene=frame, detections=detections, labels=labels)
        line_counter.trigger(detections=detections)
        line_annotator.annotate(frame=frame, line_counter=line_counter)

        out_count = line_counter.out_count
        in_count = line_counter.in_count

        cv2.imshow("yolov8", frame)

        if(cv2.waitKey(30) == 27):
            break
    
    print("OUT: " + str(out_count))
    print("IN: " + str(in_count))

if __name__ == "__main__":
    main()

My output:

...
0: 480x640 1 person, 35.0ms
0: 480x640 1 person, 35.4ms
0: 480x640 1 person, 36.8ms
0: 480x640 1 person, 35.8ms
0: 480x640 1 person, 33.9ms
0: 480x640 1 person, 33.8ms
0: 480x640 1 person, 34.4ms
0: 480x640 1 person, 34.9ms
OUT: 1
IN: 1

Thanks for you time

[SkalskiP]

I want to know what type of object that crossed the line in that moment.

By what type of object that crossed the line you mean what is the object class?

[Xkim2000]

Yes, pls

[AlainPilon]

I guess you also have a classified model already trained. So you just have to crop the current frame to the dimension provided by the detection then send the crop to the classifier.

I would strongly suggest to perform the classification async because if it is blocking, you will lose a lot of FPS.

[Xkim2000]

I'm using a trained model already. And i'm using a live feed from a camera. And I only want to get the class of the object when it crosses the line.

[Xkim2000]

I'm doing this :

        if old_in_counter != in_count:
            array_in.append(detections.class_id)

        if old_out_counter != out_count:
            array_out.append(detections.class_id)


        old_in_counter = in_count
        old_out_counter = out_count

But it adds all the classes in detections and not only what crossed the line. Any thoughts ?

[Matthias1982]

Hi
I made some test and modified the class LineZone in line_counter the following way:

`class LineZone:
def init(self, start: Point, end: Point):
self.vector = Vector(start=start, end=end)
self.tracker_state: Dict[str, bool] = {}
self.in_count: int = 0
self.out_count: int = 0
self.in_count_car: int = 0
self.out_count_car: int = 0
self.in_count_truck: int = 0
self.out_count_truck: int = 0

def trigger(self, detections: Detections) -> Tuple[np.ndarray, np.ndarray]:
    crossed_in = np.full(len(detections), False)
    crossed_out = np.full(len(detections), False)

    for i, detection in enumerate(detections):
        xyxy, _, confidence, class_id, tracker_id = detection  # Dummy-Zuweisung, ersetzen Sie durch Ihren eigenen Code

        if tracker_id is None:
            continue

        x1, y1, x2, y2 = xyxy  # Dummy-Zuweisung, ersetzen Sie durch Ihren eigenen Code
        anchors = [Point(x=x1, y=y1), Point(x=x1, y=y2), Point(x=x2, y=y1), Point(x=x2, y=y2)]
        triggers = [self.vector.is_in(point=anchor) for anchor in anchors]

        if len(set(triggers)) == 2:
            continue

        # Hier wird tracker_state definiert
        tracker_state = triggers[0]

        if tracker_id not in self.tracker_state:
            self.tracker_state[tracker_id] = tracker_state
            continue

        if self.tracker_state.get(tracker_id) == tracker_state:
            continue

        self.tracker_state[tracker_id] = tracker_state

        # Update der Zählervariablen je nach Objekttyp
        if class_id == 2:  # Car
            if tracker_state:
                self.in_count_car += 1
                crossed_in[i] = True
            else:
                self.out_count_car += 1
                crossed_out[i] = True

        elif class_id == 7:  # Truck
            if tracker_state:
                self.in_count_truck += 1
                crossed_in[i] = True
            else:
                self.out_count_truck += 1
                crossed_out[i] = True
        # Fügen Sie mehr Bedingungen für weitere Objekte hinzu

    return crossed_in, crossed_out

`

the class can be updated with more class_id's. There is most probably a nicer ways to do this, however i am a beginner in programing.

[Xkim2000]

That was very heplful.
Thank's for your contribution.

[AlainPilon]

In the current version the LineZone class is pretty lacking and it is hard to use. There is a PR for the next release which should improve things.

But one thing for sure, you cannot use the detections as they include EVERYTHING that was detected in the frame. LineZone as a tracker_state dict which you can use. It stores the tracking_id and state (relative to the line).

[Xkim2000]

Ok, I will look into that. Thanks for the advice.

[Matthias1982]

Hi
Does somebody know if or when LineZone class is improved? I need the same behavior like Xkim. I like to count different vehicles (Bike, Truck, Bus, Car and Motorbike) for each class (like car).

[AlainPilon]

Next release should contain a refactor of the trigger method, which will return a tupple of 2 arrays: one of items crossing the line in one direction and the second for items crossing in the other directions. You will then just have to look at the array which interests you and check their class and act on it.