opencv-python-inference-engine

This is Unofficial pre-built OpenCV with the inference engine part of dldt module package for Python.

Installing from pip3

Remove previously installed versions of cv2

pip3 install opencv-python-inference-engine

Why

I needed an ability to fast deploy a small package that able to run models from Intel's model zoo and use Movidius NCS. Wellr-known opencv-python can't do this. The official way is to use OpenVINO, but it is big and clumsy (just try to use it with python venv or fast download it on cloud instance).

Description

Limitations

• Package comes without contrib modules.
• You need to add udev rules if you want working MYRIAD plugin.
• It was tested on Ubuntu 18.04, Ubuntu 18.10 as Windows 10 Subsystem and Gentoo.
• It will not work for Ubuntu 16.04 and below (except v4.1.0.4).
• I had not made builds for Windows or MacOS.
• It built with ffmpeg and v4l support (ffmpeg libs included).
• No GTK/QT support -- use matplotlib for plotting your results.
• It is 64 bit.

Main differences from opencv-python-headless

• Usage of AVX2 instructions
• No JPEG 2000, WEBP, OpenEXR support
• TBB used as a parallel framework
• Inference Engine with MYRIAD plugin

Main differences from OpenVINO

• No model-optimizer
• OpenBLAS instead of MKL (details)
• No ITT
• No IPP
• No Intel Media SDK
• No OpenVINO IE API
• No python2 support (it is dead)
• No Gstreamer (use ffmpeg)
• No GTK (+16 MB and a lot of problems and extra work to compile Qt\GTK libs from sources.)

For additional info read cv2.getBuildInformation() output.

Versioning

The first 3 letters are the version of OpenCV, the last one -- package version. E.g, 4.1.0.2 -- 2nd version of based on 4.1.0 OpenCV package. Package versions are not continuously numbered -- each new OpenCV version starts its own numbering.

Downloading intel models

The official way is clumsy because you need to git clone the whole model_zoo (details)

Better to find a model description here and download manually from here

Compiling from source

You will need ~6GB RAM and ~10GB disk space

I am using Ubuntu 18.04 multipass instance: multipass launch -c 6 -d 10G -m 7G.

Requirements

From opencv, dldt, ffmpeg, openBLAS and ngraph

• build-essential
• >=cmake-3.11
• autoconf (for ngraph)
• libtool-bin (for ngraph)
• git
• pkg-config
• python3-dev
• virtualenv
• chrpath
• libusb-1.0-0-dev (for MYRIAD plugin)
• nasm (for ffmpeg)

# We need newer `cmake` for dldt (commands from  <https://apt.kitware.com/>)
wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | sudo apt-key add -
sudo apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic main'

sudo apt-get update
sudo apt install build-essential cmake git pkg-config python3-dev nasm python3 virtualenv libusb-1.0-0-dev chrpath autoconf libtool-bin

# for ngraph
# the `dldt/_deps/ext_onnx-src/onnx/gen_proto.py` has `#!/usr/bin/env python` string and will throw an error otherwise
sudo ln -s  /usr/bin/python3 /usr/bin/python

Preparing

1. git clone https://github.com/banderlog/opencv-python-inference-engine
2. cd opencv-python-inference-engine
3. run download_all_stuff.sh (refer for script code for details)

Compilation

cd build/openblas
./openblas_setup.sh &&
make -j8 &&
make install

cd ../ffmpeg
./ffmpeg_setup.sh &&
./ffmpeg_premake.sh &&
make -j8 &&
make install

cd ../dldt
./dldt_setup.sh &&
make -j8

cd ../opencv
./opencv_setup.sh &&
make -j8

Wheel creation

# get all compiled libs together
cd ../../
cp build/opencv/lib/python3/cv2.cpython*.so create_wheel/cv2/cv2.so

cp dldt/bin/intel64/Release/lib/*.so create_wheel/cv2/
cp dldt/bin/intel64/Release/lib/*.mvcmd create_wheel/cv2/
cp dldt/bin/intel64/Release/lib/plugins.xml create_wheel/cv2/
cp dldt/inference-engine/temp/tbb/lib/libtbb.so.2 create_wheel/cv2/

cp build/ffmpeg/binaries/lib/*.so create_wheel/cv2/

cp build/openblas/lib/libopenblas.so.0 create_wheel/cv2/

# change RPATH
cd create_wheel
for i in  cv2/*.so; do chrpath -r '$ORIGIN' $i; done

# final .whl will be in /create_wheel/dist/
../venv/bin/python3 setup.py bdist_wheel

Optional things to play with

• dldt build instruction
• dldt cmake flags
• opencv cmake flags

NB: removing QUIET from find_package() in project Cmake files, could help to solve some problems -- сmake will start to log them.

GTK2

Make next changes in opencv-python-inference-engine/build/opencv/opencv_setup.sh:

1. change string -D WITH_GTK=OFF \ to -D WITH_GTK=ON \
2. export PKG_CONFIG_PATH=$ABS_PORTION/build/ffmpeg/binaries/lib/pkgconfig:$PKG_CONFIG_PATH -- you will need to add absolute paths to .pc files. On Ubuntu 18.04 they here: /usr/lib/x86_64-linux-gnu/pkgconfig/:/usr/share/pkgconfig/:/usr/local/lib/pkgconfig/:/usr/lib/pkgconfig/

Exporting PKG_CONFIG_PATH for ffmpeg somehow messes with default values.

IPP

Just set -D WITH_IPP=ON in opencv_setup.sh.

It will give +30MB to the final cv2.so size. And it will boost some opencv functions.

(Image from Learning OpenCV 3 by Gary Bradski, Adrian Kaehler)

Official Intel's IPP benchmarks (may ask for registration)

MKL

You need to download MKL-DNN release and set two flags:-D GEMM=MKL , -D MKLROOT (details)

OpenVino comes with 30MB libmkl_tiny_tbb.so, but you will not be able to compile it, because it made from proprietary MKL.

Our opensource MKL-DNN experiment will end with 125MB libmklml_gnu.so and inference speed compatible with 5MB openblas (details).

OpenBLAS

Please refer here for details: OpenMathLib/OpenBLAS#2528

• OpenBLAS Installation guide
• OpenBLAS User Manual

If you compile it with make FC=gfortran, you'll need to put libgfortran.so.4 and libquadmath.so.0 to wheel and set them rpath via patchelf --set-rpath \$ORIGIN *.so

CUDA

I did not try it.

• Compile OpenCV’s ‘dnn’ module with NVIDIA GPU support
• Use OpenCV’s ‘dnn’ module with NVIDIA GPUs, CUDA, and cuDNN

Build ffmpeg with tbb

Both dldt and opencv are compiled with tbb support, and ffmpeg compiled without it -- this does not feel right. There is some unproved solution for how to compile ffmpeg with tbb support: https://stackoverflow.com/questions/6049798/ffmpeg-mt-and-tbb

OpenMP

It is possible to compile OpenBLAS, dldt and OpenCV with OpenMP. I am not sure that the result would be better than now, but who knows.

Use opencv for NLP

Presumably, you could also use speech2text model now -- source