Error while providing stream to pipeless

[Ashwini4869]

Describe the bug

I have installed pipeless successfully following the instruction (https://www.pipeless.ai/docs/v1/getting-started/installation) on how to install locally . I created the minimal project as instructed in https://www.pipeless.ai/docs/docs/v1/examples/cats . When I try to run the last step, providing the stream, it throws the following error:
[2024-08-28T05:08:56Z WARN pipeless_ai::config::adapters::rest] Restart policy not specified for stream, defaulting to 'never' [2024-08-28T05:08:56Z INFO pipeless_ai::dispatcher] New stream entry detected, creating pipeline Opening in BLOCKING MODE [2024-08-28T05:08:59Z INFO pipeless_ai::input::pipeline] Using SystemMemory [2024-08-28T05:08:59Z WARN pipeless_ai::input::pipeline] Videoconvert pad already linked, skipping link. NvMMLiteOpen : Block : BlockType = 261 NvMMLiteBlockCreate : Block : BlockType = 261 [2024-08-28T05:08:59Z ERROR pipeless_ai::input::pipeline] Error in input gst pipeline from element souphttpsrc0. Pipeline id: 4440eb3f-65eb-46b4-be84-5c372481b90d. Error: Internal data stream error. [2024-08-28T05:08:59Z ERROR pipeless_ai::pipeline] Stopping stream for pipeline: 4440eb3f-65eb-46b4-be84-5c372481b90d due to input stream error: Internal data stream error. [2024-08-28T05:08:59Z INFO pipeless_ai::dispatcher] Stream config entry removed. Stopping associated pipeline [2024-08-28T05:08:59Z INFO pipeless_ai::events] Stream loop stopped
What I have tried:

• Considering this as a GStreamer issue, I ran GStreamer alone and it works without any problem.
• The GStreamer version is the same as mentioned in the installation instructions.

Desktop (please complete the following information):

• OS: Ubuntu
• Version: 22.04.4 LTS
• Processor arch: arm aarch64
• Device : Jetpack 6

Any help to solve this problem would be much appreciated. Thank you.

[miguelaeh]

Hi @Ashwini4869 ,

Please provide run it again using GST_DEBUG=3 to see if it outputs some hint

[Ashwini4869]

Hi @miguelaeh ,
Thanks for the quick response.
I set GST_DEBUG=3 and tried again. It is still giving me the same log as earlier.
Can you provide any other solution?

[dresiwela]

Hi @Ashwini4869 ,

I am having the same error, did you find a solution?

[2024-08-30T12:47:02Z INFO  pipeless_ai::config::adapters::rest] REST adapter running
[2024-08-30T12:47:02Z INFO  warp::server] Server::run; addr=0.0.0.0:3030
[2024-08-30T12:47:02Z INFO  warp::server] listening on http://0.0.0.0:3030
[2024-08-30T12:47:16Z DEBUG hyper::proto::h1::io] parsed 4 headers
[2024-08-30T12:47:16Z DEBUG hyper::proto::h1::conn] incoming body is content-length (164 bytes)
[2024-08-30T12:47:16Z DEBUG hyper::proto::h1::conn] incoming body completed
[2024-08-30T12:47:16Z WARN  pipeless_ai::config::adapters::rest] Restart policy not specified for stream, defaulting to 'never'
[2024-08-30T12:47:16Z DEBUG hyper::proto::h1::io] flushed 391 bytes
[2024-08-30T12:47:16Z INFO  pipeless_ai::dispatcher] New stream entry detected, creating pipeline
[2024-08-30T12:47:16Z DEBUG hyper::proto::h1::conn] read eof
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: Playing
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: Paused
0:00:20.468468564 27925 0xffff74027f50 WARN                 basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<filesrc0> pad not activated yet
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: Paused
0:00:20.468818314 27925 0xffff74027f50 WARN                 basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<filesrc0> pad not activated yet
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: Paused
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: Paused
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
0:00:20.483353118 27925 0xffff50076d20 WARN                 qtdemux qtdemux_types.c:249:qtdemux_type_get: unknown QuickTime node type sgpd
0:00:20.483395848 27925 0xffff50076d20 WARN                 qtdemux qtdemux_types.c:249:qtdemux_type_get: unknown QuickTime node type sbgp
0:00:20.483428720 27925 0xffff50076d20 INFO                 qtdemux qtdemux.c:13794:qtdemux_parse_tree:<qtdemux0> timescale: 1000
0:00:20.483446869 27925 0xffff50076d20 INFO                 qtdemux qtdemux.c:13795:qtdemux_parse_tree:<qtdemux0> duration: 56550
0:00:20.483518694 27925 0xffff50076d20 WARN                 qtdemux qtdemux.c:3121:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 1
0:00:20.483625984 27925 0xffff50076d20 INFO                 qtdemux qtdemux.c:12146:qtdemux_parse_trak:<qtdemux0> type avc1 caps video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)4, profile=(string)high, codec_data=(buffer)01640028ffe1001d67640028acd940780227e59a80808120000003002000000501e30632c001000668ebe3cb22c0fdf8f800
0:00:20.483718199 27925 0xffff50076d20 WARN                 qtdemux qtdemux.c:3121:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 2
0:00:20.483801387 27925 0xffff50076d20 INFO                 qtdemux qtdemux.c:12971:qtdemux_parse_trak:<qtdemux0> type mp4a caps audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)118856e500
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
0:00:20.489710474 27925 0xffff50076d20 FIXME             decodebin3 gstdecodebin3.c:1123:update_requested_selection:<decodebin3-0> Implement EXPOSE_ALL_MODE
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
0:00:20.492264952 27925 0xffff50076d20 FIXME             decodebin3 gstdecodebin3-parse.c:433:unblock_pending_input:<decodebin3-0> Re-use existing input streams if/when possible
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
0:00:20.493149904 27925 0xffff4803ec60 FIXME             decodebin3 gstdecodebin3.c:1621:get_output_for_slot:<decodebin3-0> emit autoplug-continue
0:00:20.493288017 27925 0xffff4803ec60 FIXME             decodebin3 gstdecodebin3.c:1624:get_output_for_slot:<decodebin3-0> Handle EXPOSE_ALL_MODE
Opening in BLOCKING MODE 
0:00:20.581211672 27925 0xffff4803ec60 WARN                    v4l2 gstv4l2object.c:4671:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z INFO  pipeless_ai::input::pipeline] Using SystemMemory
[2024-08-30T12:47:16Z WARN  pipeless_ai::input::pipeline] Videoconvert pad already linked, skipping link.
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
0:00:20.581729366 27925 0xffff4803ecc0 FIXME             decodebin3 gstdecodebin3.c:1621:get_output_for_slot:<decodebin3-0> emit autoplug-continue
0:00:20.581762718 27925 0xffff4803ecc0 FIXME             decodebin3 gstdecodebin3.c:1624:get_output_for_slot:<decodebin3-0> Handle EXPOSE_ALL_MODE
NvMMLiteOpen : Block : BlockType = 261 
NvMMLiteBlockCreate : Block : BlockType = 261 
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Null. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z INFO  pipeless_ai::input::pipeline] Using SystemMemory
[2024-08-30T12:47:16Z WARN  pipeless_ai::input::pipeline] Videoconvert pad already linked, skipping link.
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Paused. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
0:00:20.732957783 27925 0xffff4803ec60 WARN                    v4l2 gstv4l2object.c:4671:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:20.733574381 27925 0xffff4803ec60 WARN            v4l2videodec gstv4l2videodec.c:2311:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder0> Duration invalid, not setting latency
0:00:20.734210056 27925 0xffff4803ec60 WARN          v4l2bufferpool gstv4l2bufferpool.c:1116:gst_v4l2_buffer_pool_start:<nvv4l2decoder0:pool:src> Uncertain or not enough buffers, enabling copy threshold
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
0:00:20.764199696 27925 0xffff4803f120 WARN          v4l2bufferpool gstv4l2bufferpool.c:1567:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:src> Driver should never set v4l2_buffer.field to ANY
0:00:20.768529743 27925 0xffff4803ec60 WARN            videodecoder gstvideodecoder.c:3158:gst_video_decoder_prepare_finish_frame:<nvv4l2decoder0> decreasing timestamp (0:00:00.850000000 < 0:00:00.900000000)
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
0:00:20.768625734 27925 0xffff4803ec60 WARN            videodecoder gstvideodecoder.c:3158:gst_video_decoder_prepare_finish_frame:<nvv4l2decoder0> decreasing timestamp (0:00:00.250000000 < 0:00:00.900000000)
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
0:00:20.768746692 27925 0xffff4803ec60 WARN            videodecoder gstvideodecoder.c:3158:gst_video_decoder_prepare_finish_frame:<nvv4l2decoder0> decreasing timestamp (0:00:00.300000000 < 0:00:00.900000000)
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
0:00:20.768817973 27925 0xffff50076d20 WARN                 qtdemux qtdemux.c:6749:gst_qtdemux_loop:<qtdemux0> error: Internal data stream error.
0:00:20.768871042 27925 0xffff50076d20 WARN                 qtdemux qtdemux.c:6749:gst_qtdemux_loop:<qtdemux0> error: streaming stopped, reason not-linked (-1)
0:00:20.768836473 27925 0xffff4803ec60 WARN            videodecoder gstvideodecoder.c:3158:gst_video_decoder_prepare_finish_frame:<nvv4l2decoder0> decreasing timestamp (0:00:00.350000000 < 0:00:00.900000000)
0:00:20.768984574 27925 0xffff4803ec60 WARN            videodecoder gstvideodecoder.c:3158:gst_video_decoder_prepare_finish_frame:<nvv4l2decoder0> decreasing timestamp (0:00:00.400000000 < 0:00:00.900000000)
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
0:00:20.769055951 27925 0xffff4803ec60 WARN            videodecoder gstvideodecoder.c:3158:gst_video_decoder_prepare_finish_frame:<nvv4l2decoder0> decreasing timestamp (0:00:00.450000000 < 0:00:00.900000000)
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Debug info for the following error: ../gst/isomp4/qtdemux.c(6749): gst_qtdemux_loop (): /GstPipeline:pipeline0/GstBin:bin0/GstURIDecodeBin3:source/GstDecodebin3:decodebin3-0/GstParseBin:parsebin0/GstQTDemux:qtdemux0:
    streaming stopped, reason not-linked (-1)
0:00:20.769116478 27925 0xffff4803ec60 WARN            videodecoder gstvideodecoder.c:3158:gst_video_decoder_prepare_finish_frame:<nvv4l2decoder0> decreasing timestamp (0:00:00.500000000 < 0:00:00.900000000)
[2024-08-30T12:47:16Z ERROR pipeless_ai::input::pipeline] Error in input gst pipeline from element qtdemux0.
                    Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa. Error: Internal data stream error.
0:00:20.769184334 27925 0xffff50076d20 FIXME             decodebin3 gstdecodebin3-parse.c:148:check_all_streams_for_eos:<multiqueue0:sink_0> Remove input stream
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
0:00:20.769228953 27925 0xffff50076d20 FIXME             decodebin3 gstdecodebin3-parse.c:148:check_all_streams_for_eos:<multiqueue0:sink_1> Remove input stream
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z ERROR pipeless_ai::pipeline] Stopping stream for pipeline: 460b6a58-8931-4b4b-bc3b-f07450e16baa due to input stream error: Internal data stream error.
[2024-08-30T12:47:16Z INFO  pipeless_ai::events] Stream loop stopped
[2024-08-30T12:47:16Z INFO  pipeless_ai::dispatcher] Stream config entry removed. Stopping associated pipeline
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Playing. Current state: Paused. Pending state: Ready
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Ready. Pending state: Null
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Null. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] 
                Unhandled message on input gst pipeline bus.
                Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Null. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Null. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Null. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Null. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Null. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Paused. Current state: Ready. Pending state: Null
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Null. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG pipeless_ai::input::pipeline] Input gst pipeline state change. Pipeline id: 460b6a58-8931-4b4b-bc3b-f07450e16baa.
                    Old state: Ready. Current state: Null. Pending state: VoidPending
[2024-08-30T12:47:16Z DEBUG sled::pagecache::iterator] ordering before clearing tears: {0: 0}, max_header_stable_lsn: 0
[2024-08-30T12:47:16Z DEBUG sled::pagecache::iterator] in clean_tail_tears, found missing item in tail: None and we'll scan segments {0: 0} above lowest lsn 0
[2024-08-30T12:47:16Z DEBUG sled::pagecache::iterator] filtering out segments after detected tear at (lsn, lid) 95
[2024-08-30T12:47:16Z DEBUG sled::pagecache::iterator] hit max_lsn 95 in iterator, stopping
[2024-08-30T12:47:16Z DEBUG sled::pagecache::snapshot] zeroing the end of the recovered segment at lsn 0 between lids 96 and 524287
[2024-08-30T12:47:16Z DEBUG sled::pagecache::blob_io] gc_blobs removing any blob with an lsn above 96
[2024-08-30T12:47:16Z DEBUG sled::pagecache::segment] SA starting with tip 524288 stable -1 free {}
[2024-08-30T12:47:16Z DEBUG sled::pagecache::iobuf] starting log at recovered active offset 96, recovered lsn 96
[2024-08-30T12:47:16Z DEBUG sled::pagecache::iobuf] starting IoBufs with next_lsn: 96 next_lid: 96
[2024-08-30T12:47:16Z DEBUG sled::pagecache] load_snapshot loading pages from 0..4

[dresiwela]

@miguelaeh I have been looking into it further this is what I have found, when attempting to process a VP9 video in a WebM/Matroska container, Pipeless fails with an "Internal data stream error" originating from the matroskademux element. The error message indicates a "not-linked" condition, suggesting an issue with the GStreamer pipeline construction.

Steps to reproduce:

1. Use a VP9 encoded video in a WebM/Matroska container
2. Attempt to process the video using Pipeless

Error message:
[ERROR pipeless_ai::input::pipeline] Error in input gst pipeline from element matroskademux0. Pipeline id: [ID]. Error: Internal data stream error.

Suspected cause:
The GStreamer pipeline in Pipeless may not be correctly configured for VP9 video in WebM/Matroska containers. It appears that the matroskademux element is not properly linked to the subsequent elements required for VP9 decoding, I also get a similar issue for mp4, mkv and avi.

Steps I have taken:

1. Reviewed the pipeline construction code, especially handling of VP9 and WebM/Matroska containers
2. Implemented dynamic pipeline construction based on input file format and codec
3. Ensure proper linking of elements, especially after the demuxer

Additional context:

• GStreamer version: 1.20.3
• Pipeless is able to detect the VP9 codec, but fails to process it
• Audio (Opus) decoding appears to work until the video stream error occurs

Let me know if you need any additional information or logs to diagnose this issue

[miguelaeh]

Steps I have taken:

1. Reviewed the pipeline construction code, especially handling of VP9 and WebM/Matroska containers
2. Implemented dynamic pipeline construction based on input file format and codec
3. Ensure proper linking of elements, especially after the demuxer

So, did you fix the error?

[dresiwela]

@miguelaeh No, those were the steps I tried but I couldnt manage to get it working.

[Ashwini4869]

Hi @miguelaeh ,
Have you found any solution for this issue? GStreamer by itself works fine. It is only when providing stream to pipeless that it causes the error.
Thanks.

[Ashwini4869]

I started pipeless using GST_DEBUG=3 and got the error in detail:

[2024-11-21T12:41:56Z INFO warp::server] Server::run; addr=0.0.0.0:3030 [2024-11-21T12:41:56Z INFO warp::server] listening on http://0.0.0.0:3030 [2024-11-21T12:42:05Z WARN pipeless_ai::config::adapters::rest] Restart policy not specified for stream, defaulting to 'never' [2024-11-21T12:42:05Z INFO pipeless_ai::dispatcher] New stream entry detected, creating pipeline 0:00:12.353927771 46430 0xffff58028750 WARN basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<filesrc0> pad not activated yet 0:00:12.354273554 46430 0xffff58028750 WARN basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<filesrc0> pad not activated yet 0:00:12.366447850 46430 0xffff18079520 WARN qtdemux qtdemux.c:3121:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 1 0:00:12.369306692 46430 0xffff18079520 FIXME decodebin3 gstdecodebin3.c:1123:update_requested_selection:<decodebin3-0> Implement EXPOSE_ALL_MODE 0:00:12.369940308 46430 0xffff18079520 FIXME decodebin3 gstdecodebin3-parse.c:433:unblock_pending_input:<decodebin3-0> Re-use existing input streams if/when possible 0:00:12.370259117 46430 0xffff18079f00 FIXME decodebin3 gstdecodebin3.c:1621:get_output_for_slot:<decodebin3-0> emit autoplug-continue 0:00:12.370283340 46430 0xffff18079f00 FIXME decodebin3 gstdecodebin3.c:1624:get_output_for_slot:<decodebin3-0> Handle EXPOSE_ALL_MODE Opening in BLOCKING MODE 0:00:12.392968739 46430 0xffff18079f00 WARN v4l2 gstv4l2object.c:4682:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1 [2024-11-21T12:42:05Z INFO pipeless_ai::input::pipeline] Using SystemMemory [2024-11-21T12:42:05Z WARN pipeless_ai::input::pipeline] Videoconvert pad already linked, skipping link. NvMMLiteOpen : Block : BlockType = 261 NvMMLiteBlockCreate : Block : BlockType = 261 0:00:12.494976269 46430 0xffff18079f00 WARN v4l2 gstv4l2object.c:4682:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1 0:00:12.495530399 46430 0xffff18079f00 WARN v4l2videodec gstv4l2videodec.c:2297:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder0> Duration invalid, not setting latency 0:00:12.514077755 46430 0xffff18079f00 WARN v4l2bufferpool gstv4l2bufferpool.c:1130:gst_v4l2_buffer_pool_start:<nvv4l2decoder0:pool:src> Uncertain or not enough buffers, enabling copy threshold 0:00:12.532860433 46430 0xffff14017520 WARN v4l2bufferpool gstv4l2bufferpool.c:1607:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:src> Driver should never set v4l2_buffer.field to ANY 0:00:12.533659742 46430 0xffff18079520 WARN qtdemux qtdemux.c:6749:gst_qtdemux_loop:<qtdemux0> error: Internal data stream error. 0:00:12.533678909 46430 0xffff18079f00 WARN videodecoder gstvideodecoder.c:3158:gst_video_decoder_prepare_finish_frame:<nvv4l2decoder0> decreasing timestamp (0:00:00.800000000 < 0:00:00.880000000) 0:00:12.533707517 46430 0xffff18079520 WARN qtdemux qtdemux.c:6749:gst_qtdemux_loop:<qtdemux0> error: streaming stopped, reason not-linked (-1) 0:00:12.533828506 46430 0xffff18079f00 WARN videodecoder gstvideodecoder.c:3158:gst_video_decoder_prepare_finish_frame:<nvv4l2decoder0> decreasing timestamp (0:00:00.240000000 < 0:00:00.880000000) 0:00:12.533918232 46430 0xffff18079f00 WARN videodecoder gstvideodecoder.c:3158:gst_video_decoder_prepare_finish_frame:<nvv4l2decoder0> decreasing timestamp (0:00:00.280000000 < 0:00:00.880000000) 0:00:12.533976342 46430 0xffff18079f00 WARN videodecoder gstvideodecoder.c:3158:gst_video_decoder_prepare_finish_frame:<nvv4l2decoder0> decreasing timestamp (0:00:00.320000000 < 0:00:00.880000000) 0:00:12.534032373 46430 0xffff18079f00 WARN videodecoder gstvideodecoder.c:3158:gst_video_decoder_prepare_finish_frame:<nvv4l2decoder0> decreasing timestamp (0:00:00.360000000 < 0:00:00.880000000) 0:00:12.534082964 46430 0xffff18079f00 WARN videodecoder gstvideodecoder.c:3158:gst_video_decoder_prepare_finish_frame:<nvv4l2decoder0> decreasing timestamp (0:00:00.400000000 < 0:00:00.880000000) 0:00:12.534128627 46430 0xffff18079f00 WARN videodecoder gstvideodecoder.c:3158:gst_video_decoder_prepare_finish_frame:<nvv4l2decoder0> decreasing timestamp (0:00:00.440000000 < 0:00:00.880000000) 0:00:12.534148850 46430 0xffff18079520 FIXME decodebin3 gstdecodebin3-parse.c:148:check_all_streams_for_eos:<multiqueue0:sink_0> Remove input stream 0:00:12.534176209 46430 0xffff18079f00 FIXME decodebin3 gstdecodebin3.c:1931:multiqueue_src_probe:<multiqueue0:src_0> EOS on multiqueue source pad. input:0xffff140546e0 0:00:12.534245200 46430 0xffff18079f00 FIXME decodebin3 gstdecodebin3.c:1931:multiqueue_src_probe:<multiqueue0:src_0> EOS on multiqueue source pad. input:0xffff140546e0 [2024-11-21T12:42:05Z ERROR pipeless_ai::input::pipeline] Error in input gst pipeline from element qtdemux0. Pipeline id: d9748046-ae65-49e0-af7a-7455b62146cd. Error: Internal data stream error. [2024-11-21T12:42:05Z ERROR pipeless_ai::pipeline] Stopping stream for pipeline: d9748046-ae65-49e0-af7a-7455b62146cd due to input stream error: Internal data stream error. [2024-11-21T12:42:05Z INFO pipeless_ai::events] Stream loop stopped [2024-11-21T12:42:05Z INFO pipeless_ai::dispatcher] Stream config entry removed. Stopping associated pipeline

Also, when I run
gst-launch-1.0 filesrc location=<video_file> ! qtdemux ! h264parse ! avdec_h264 ! autovideosink

it works flawlessly.
Any kind of help would be very much appreciated. Thank you.