blank output -- cuGraphicsGLRegisterBuffer failed with error(219) gst_eglglessink_cuda_init texture = 1

[saket424]

I am trying to follow all the steps documented in the README.md and i am getting the following error on an 3070ti amd ryzen9 laptop

root@mgear1:/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-retail-analytics# nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Jun__8_16:49:14_PDT_2022
Cuda compilation tools, release 11.7, V11.7.99
Build cuda_11.7.r11.7/compiler.31442593_0
root@mgear1:/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-retail-analytics# export CUDA_VER=11.7
root@mgear1:/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-retail-analytics# make -B
cc -c -o retail_iva.o -I../../../includes -I /usr/local/cuda-11.7/include -pthread -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include retail_iva.c
cc -o ds-retail-iva retail_iva.o -lgstreamer-1.0 -lgobject-2.0 -lglib-2.0 -L/opt/nvidia/deepstream/deepstream-6.1/lib/ -lnvdsgst_meta -lnvds_meta -lrt -L/usr/local/cuda-11.7/lib64/ -lcudart -lnvds_yml_parser -lcuda -Wl,-rpath,/opt/nvidia/deepstream/deepstream-6.1/lib/
root@mgear1:/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-retail-analytics# echo $DISPLAY
:1
root@mgear1:/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-retail-analytics#


root@mgear1:/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-retail-analytics# ./ds-retail-iva configs/retail_iva.yml 
Unknown key 'threshold' for group [class-attrs-all]
Using file configs/retail_iva.yml 
0:00:00.104451490   968 0x55fb9a3cd0f0 INFO                 nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<secondary-inference-engine> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1923> [UID = 2]: Trying to create engine from model files
WARNING: [TRT]: The implicit batch dimension mode has been deprecated. Please create the network with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag whenever possible.
0:00:07.916146035   968 0x55fb9a3cd0f0 INFO                 nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<secondary-inference-engine> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1955> [UID = 2]: serialize cuda engine to file: /opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-retail-analytics/files/basketClassifier.etlt_b1_gpu0_fp32.engine successfully
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 2
0   INPUT  kFLOAT input_1         3x224x224       
1   OUTPUT kFLOAT predictions/Softmax 2x1x1           

0:00:07.977693130   968 0x55fb9a3cd0f0 INFO                 nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus:<secondary-inference-engine> [UID 2]: Load new model:configs/basket_classifier.yml sucessfully
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-6.1/lib/libnvds_nvmultiobjecttracker.so
~~ CLOG[include/modules/NvMultiObjectTracker/NvTrackerParams.hpp, getConfigRoot() @line 52]: [NvTrackerParams::getConfigRoot()] !!![WARNING] Invalid low-level config file caused an exception, but will go ahead with the default config values
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is OFF
~~ CLOG[include/modules/NvMultiObjectTracker/NvTrackerParams.hpp, getConfigRoot() @line 52]: [NvTrackerParams::getConfigRoot()] !!![WARNING] Invalid low-level config file caused an exception, but will go ahead with the default config values
[NvMultiObjectTracker] Initialized
0:00:07.995750500   968 0x55fb9a3cd0f0 INFO                 nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<primary-inference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1923> [UID = 1]: Trying to create engine from model files
WARNING: [TRT]: The implicit batch dimension mode has been deprecated. Please create the network with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag whenever possible.
0:00:27.551765138   968 0x55fb9a3cd0f0 INFO                 nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<primary-inference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1955> [UID = 1]: serialize cuda engine to file: /opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-retail-analytics/files/resnet34_peoplenet_pruned.etlt_b1_gpu0_fp32.engine successfully
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 3
0   INPUT  kFLOAT input_1         3x544x960       
1   OUTPUT kFLOAT output_bbox/BiasAdd 12x34x60        
2   OUTPUT kFLOAT output_cov/Sigmoid 3x34x60         

0:00:27.614072376   968 0x55fb9a3cd0f0 INFO                 nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus:<primary-inference-engine> [UID 1]: Load new model:configs/pgie_config_peoplenet.yml sucessfully
Running ...
Frame Number = 0 Person Count = 0
Frame Number = 1 Person Count = 0
Frame Number = 2 Person Count = 0
Frame Number = 3 Person Count = 0
cuGraphicsGLRegisterBuffer failed with error(219) gst_eglglessink_cuda_init texture = 1

[saket424]

It turns out that even though nvidia-smi was working the laptop was using the integrated gpu instead of the discrete nvidia gpu
To force it to use the discrete nvidia gpu, use the following command and reboot

sudo prime-select nvidia
sudo reboot

Confirm that the directive to use the nvidia gpu is in fact heeded

anand@l5i:~$ nvidia-settings -g | grep OpenGL
  OpenGL vendor string: NVIDIA Corporation
  OpenGL renderer string: NVIDIA GeForce RTX 3070 Ti Laptop GPU/PCIe/SSE2
  OpenGL version string: 4.6.0 NVIDIA 525.60.11
  OpenGL extensions:
anand@l5i:~$

Now the deepstream-retail-analytics sample should work

root@l5i:/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstre
am-retail-analytics# ./ds-retail-iva configs/retail_iva.yml 
Unknown key 'threshold' for group [class-attrs-all]
Using file configs/retail_iva.yml 
0:00:00.104544513   243 0x5610d577ea10 INFO                 nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<secondary-inference-engine> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1914> [UID = 2]: Trying to create engine from model files
0:00:09.163562086   243 0x5610d577ea10 INFO                 nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<secondary-inference-engine> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1946> [UID = 2]: serialize cuda engine to file: /opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-retail-analytics/files/basketClassifier.etlt_b1_gpu0_fp32.engine successfully
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 2
0   INPUT  kFLOAT input_1         3x224x224       
1   OUTPUT kFLOAT predictions/Softmax 2x1x1           

0:00:09.178372125   243 0x5610d577ea10 INFO                 nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus:<secondary-inference-engine> [UID 2]: Load new model:configs/basket_classifier.yml sucessfully
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-6.1/lib/libnvds_nvmultiobjecttracker.so
~~ CLOG[include/modules/NvMultiObjectTracker/NvTrackerParams.hpp, getConfigRoot() @line 52]: [NvTrackerParams::getConfigRoot()] !!![WARNING] Invalid low-level config file caused an exception, but will go ahead with the default config values
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is OFF
~~ CLOG[include/modules/NvMultiObjectTracker/NvTrackerParams.hpp, getConfigRoot() @line 52]: [NvTrackerParams::getConfigRoot()] !!![WARNING] Invalid low-level config file caused an exception, but will go ahead with the default config values
[NvMultiObjectTracker] Initialized
0:00:09.197505384   243 0x5610d577ea10 INFO                 nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<primary-inference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1914> [UID = 1]: Trying to create engine from model files
0:00:30.563587155   243 0x5610d577ea10 INFO                 nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<primary-inference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1946> [UID = 1]: serialize cuda engine to file: /opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-retail-analytics/files/resnet34_peoplenet_pruned.etlt_b1_gpu0_fp32.engine successfully
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 3
0   INPUT  kFLOAT input_1         3x544x960       
1   OUTPUT kFLOAT output_bbox/BiasAdd 12x34x60        
2   OUTPUT kFLOAT output_cov/Sigmoid 3x34x60         

0:00:30.579839650   243 0x5610d577ea10 INFO                 nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus:<primary-inference-engine> [UID 1]: Load new model:configs/pgie_config_peoplenet.yml sucessfully
Running ...
Frame Number = 0 Person Count = 0
Frame Number = 1 Person Count = 0
Frame Number = 2 Person Count = 0
Frame Number = 3 Person Count = 0
Frame Number = 4 Person Count = 0
Frame Number = 5 Person Count = 3
Frame Number = 6 Person Count = 3
Frame Number = 7 Person Count = 3
Frame Number = 8 Person Count = 3
Frame Number = 9 Person Count = 3
Frame Number = 10 Person Count = 3
Frame Number = 11 Person Count = 4
Frame Number = 12 Person Count = 5
Frame Number = 13 Person Count = 5
Frame Number = 14 Person Count = 5

[saket424]

Thanks for the blogpost https://developer.nvid2e9bac788bc3ia.com/blog/building-an-end-to-end-retail-analytics-application-with-nvidia-deepstream-and-nvidia-tao-toolkit/

[ckhire]

Thanks @saket424 . It was really helful. !

[Abhi-shekes]

Thanks @saket424 . you saved me