Investigate GPUs on ARM boards

[lissyx]

We have ARM and ARM64 (#1305) arriving on RPi3B+ and LePotato boards. We should try and see if we can get GPU acceleration there:

• OpenCL can work not too bad on Intel GPUs (cf ccpp) using ComputeCpp
• LePotato has a MALI-450 GPU, no idea yet how much we can expect of OpenCL there
• RPi3B (and + model) now have OpenCL 1.2 Embedded Profile compliant https://github.com/doe300/VC4CL#opencl-support and ComputeCpp has ARMv7 binaries targetting Ubuntu 14.04, so we might be able to get stuff to work there.

[edubois]

The Rock64Pro-AI has a mali-t860, I would consider this option.

[lissyx]

@edubois You're welcome to explore that, but I'll stick to what I have right now :)

[edubois]

Will do,

[renepeinl]

You state ARM boards but then mention Intel GPUs. Therefore, I would offer to do some testing on the UP board which is powered by a 4-core Atom x5-Z8350, has 4 GB of RAM and incorporates a Cherry Trail HD graphics with 12 execution units.
We are using that, since it is compatible with the Matrix Voice, our hardware for sound capturing.

We are also looking at the Intel Movidius Neural Compute Stick, which is compatible with Tensor Flow and the Raspberry Pi 3, since the combination would be more cost-effective.

The Rock64Pro-AI looks interesting as well and would be even cheaper. I'm curious on the results of @edubois

[lissyx]

@renepeinl I have actually been experimenting for quite some time on Intel GPUs on my laptop, debugging and checking performances (thanks to CodePlay people and Intel people), so I know we can get it working with the "Neo" driver, which is far from being released yet, sadly. The Compute Stick is useless in our case, because of RNNs. The previous driver, Beignet, was a dead-end: not working with ComputeCpp (layer used by TensorFlow for OpenCL), and not being actively developped anymore by intel.

People who want to experiment should use the ccpp branch of our TensorFlow and DeepSpeech repo, but be aware it's hack in progress :)

[renepeinl]

Thanks for these information. Could you provide some hints on compiler flags for building the software as well? I'm not sure how much influence they have and we are mainly Java developers with no deeper knowledge about C++.

[lissyx]

@renepeinl it should be pretty simple, if you follow the docs we have in place and TensorFlow's building doc. For ComputeCpp branch, you'll need to download ComputeCpp matching version. Since it's hack in progress, I have not documented that, but you can look the tc-*.sh shell scripts in our TensorFlow's repo, it should contain everything. Basically, Bazel v0.10.0, ComputeCpp 0.5.1 (I think?) and proper ./configure flags (check tc-vars.sh mostly). On the DeepSpeech build side, it should not change OpenCL or not.

[lissyx]

@renepeinl If you run into issues, you can join us on IRC (#machinelearning on irc.mozilla.org) or on Discourse: https://discourse.mozilla.org/c/deep-speech

[edubois]

@renepeinl, the chip is not yet available, will start when I get one, probably in Sept.

[lissyx]

Good first milestone on RPi3:

• I have llvm-spirv cross-built
• I have been able to cross-compile vc4c, vc4clstdlib and vc4cl bits
• Debian packages properly installs on Raspbian
• ComputeCpp 0.7.0 for Ubuntu 14.04 / ARM32 shows VC4 GPU
• Currently running the VC4C testsuite, there are failures, but there are successes as well, meaning the basics of the infra is there and working

[lissyx]

$ sudo ComputeCpp-CE-0.7.0-Ubuntu-14.04-ARM_32/bin/computecpp_info --verbose --use-spirv 
********************************************************************************

ComputeCpp Info (CE 0.7.0)

********************************************************************************

Toolchain information:

GLIBC version: 2.24
GLIBCXX: 20150426
This version of libstdc++ is supported.

********************************************************************************


Device Info:

Discovered 1 devices matching:
  platform    : <any>
  device type : <any>

--------------------------------------------------------------------------------
Device 0:

  Device is supported                     : NO - Device does not support SPIR
  CL_DEVICE_NAME                          : VideoCore IV GPU
  CL_DEVICE_VENDOR                        : Broadcom
  CL_DRIVER_VERSION                       : 0.4
  CL_DEVICE_TYPE                          : CL_DEVICE_TYPE_GPU 
  CL_DEVICE_VERSION                       : OpenCL 1.2 VC4CL 0.4
  CL_DEVICE_PROFILE                       : EMBEDDED_PROFILE
  CL_DEVICE_MAX_COMPUTE_UNITS             : 1
  CL_DEVICE_MAX_WORK_ITEM_DIMENSIONS      : 3
  CL_DEVICE_MAX_WORK_ITEM_SIZES           : 12 / 12 / 12
  CL_DEVICE_MAX_WORK_GROUP_SIZE           : 12
  CL_DEVICE_MAX_CLOCK_FREQUENCY           : 300 MHz
  CL_DEVICE_ADDRESS_BITS                  : 32
  CL_DEVICE_HOST_UNIFIED_MEMORY           : YES
  CL_DEVICE_MAX_MEM_ALLOC_SIZE            : 76 MByte
  CL_DEVICE_GLOBAL_MEM_SIZE               : 76 MByte
  CL_DEVICE_ERROR_CORRECTION_SUPPORT      : NO
  CL_DEVICE_LOCAL_MEM_TYPE                : global
  CL_DEVICE_LOCAL_MEM_SIZE                : 77824 KByte
  CL_DEVICE_MAX_CONSTANT_BUFFER_SIZE      : 77824 KByte
  CL_DEVICE_QUEUE_PROPERTIES              : CL_QUEUE_PROFILING_ENABLE
  CL_DEVICE_IMAGE_SUPPORT                 : NO
  CL_DEVICE_MAX_READ_IMAGE_ARGS           : 64
  CL_DEVICE_MAX_WRITE_IMAGE_ARGS          : 64
  CL_DEVICE_IMAGE2D_MAX_WIDTH             : 2048
  CL_DEVICE_IMAGE2D_MAX_HEIGHT            : 2048
  CL_DEVICE_IMAGE3D_MAX_WIDTH             : 2048
  CL_DEVICE_IMAGE3D_MAX_HEIGHT            : 2048
  CL_DEVICE_IMAGE3D_MAX_DEPTH             : 2048
  CL_DEVICE_PREFERRED_VECTOR_WIDTH        : CHAR 16 SHORT 16 INT 16 LONG 0 FLOAT 16 DOUBLE 0 
  CL_DEVICE_EXTENSIONS                    : cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_nv_pragma_unroll cl_arm_core_id cl_ext_atomic_counters_32 cl_khr_initialize_memory


If you encounter problems when using any of these OpenCL devices, please consult
this website for known issues:
https://computecpp.codeplay.com/releases/v0.7.0/platform-support-notes

********************************************************************************