Sub-project 2: Support for Intel AVX512

[larryxiao]

• Get to know ISA
• Prepare development environment: Intel Toolchain, SDE
• Study programming interface
• Prototype float 16
• Compile
• Testing
• Code to support all size

[larryxiao]

• Reference
  • AVX-512 - Wikipedia, the free encyclopedia
  • Intel® Architecture Instruction Set Extensions Programming Reference
  • Additional AVX-512 instructions | Intel® Developer Zone
  • AVX-512 instructions | Intel® Developer Zone

[larryxiao]

AVX512 Toolchain

• Intel® Software Development Emulator | Intel® Developer Zone
• In separate stages

# compile
../Software/gcc_install_5.0.0-mpx-r214719/bin/x86_64-pc-linux-gnu-c++ -mavx512f -B ../Software/binutils-gdb_install_2.24.51.20140422/bin -c avx512.cpp -o avx512.o
# link
g++ -L /usr/lib/x86_64-linux-gnu/ avx512.o
# running
../Software/sde-external-7.21.0-2015-04-01-lin/sde -mpx-mode -- ./a.out

• In one stage
  • gcc - Compiling problems: cannot find crt1.o - Stack Overflow (solves ../Software/binutils-gdb_install_2.24.51.20140422/bin/ld: cannot find crt1.o) see Multiarch/LibraryPathOverview - Debian Wiki

# compile
../Software/gcc_install_5.0.0-mpx-r214719/bin/x86_64-pc-linux-gnu-c++ -mavx512f -B ../Software/binutils-gdb_install_2.24.51.20140422/bin -L /usr/lib/x86_64-linux-gnu/ avx512.cpp
# run
../Software/sde-external-7.21.0-2015-04-01-lin/sde -mpx-mode -- ./a.out

Source

#include "stdio.h"
#include "immintrin.h"
//#include "zmmintrin.h" not providedm immintrin.h includes avx512* headers

int main(int argc, const char *argv[])
{
    double a[4] = {1.0, 2.0, 3.0, 4.0};
    double b[4] = {8.0, 7.0, 6.0, 5.0};
    double e[4] = {0.0};
    __m256d test0 = _mm256_set_pd(1.0, 2.0, 3.0, 4.0);
    __m256d val0 = _mm256_load_pd(a);
    __m256d val1 = _mm256_load_pd(b);
    __m256d val2 = _mm256_add_pd(val0, val1);
    _mm256_store_pd(e, val2);
    printf("avx256 %f %f %f %f\n", e[0], e[1], e[2], e[3]);

    double c[8] = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0};
    double d[8] = {8.0, 7.0, 6.0, 5.0, 4.0, 3.0, 2.0, 1.0};
    double f[8] = {0.0};
    __m512d test1 = _mm512_set_pd(1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0);
    __m512d val3 = _mm512_load_pd(c);
    __m512d val4 = _mm512_load_pd(d);
    __m512d val5 = _mm512_add_pd(val3, val4);
    _mm512_store_pd(f, val5);
    printf("avx512 %f %f %f %f %f %f %f %f\n", f[0], f[1], f[2], f[3],
            f[4], f[5], f[6], f[7]);

    printf("test\n");
    return 0;
}

[larryxiao]

Debug

• error: inlining failed in call to always_inline ‘__m512d _mm512_add_pd(__m512d, __m512d)’: target specific option mismatch
  • c++ - Determine what intrinsic flag is activated - Stack Overflow
• The Operating System configuration prevents Pin from using the default (parent) injection mode.
  • Pin: Pin 2.10 User Guide
• learn how to compile, and run sde
  • Using Intel® MPX with the Intel® Software Development Emulator | Intel® Developer Zone
• AVX512 ISA
• SDE ERROR: TID: 0 executed instruction with an unaligned memory reference to address

Boost

CMake Error at /usr/share/cmake-3.2/Modules/FindBoost.cmake:1202 (message):
  Unable to find the requested Boost libraries.

  Boost version: 1.54.0

  Boost include path: /usr/include

  Could not find the following Boost libraries:

          boost_date_time
          boost_filesystem
          boost_system

  No Boost libraries were found.  You may need to set BOOST_LIBRARYDIR to the
  directory containing Boost libraries or BOOST_ROOT to the location of
  Boost.

Set BOOST_INCLUDEDIR and BOOST_LIBRARYDIR

Compile

$ make VERBOSE=1
[ 12%] Building CXX object test/CMakeFiles/aligned_allocator_test.dir/aligned_allocator_test.cpp.o
cd /home/walnut/Developer/libflatarray/build/Linux-x86_64/test && /home/walnut/Developer/libflatarray/Software/gcc_install_5.0.0-mpx-r214719/bin/x86_64-pc-linux-gnu-c++    -D __AVX512__ -O3 -DNDEBUG -I/home/walnut/Developer/libflatarray/src    -o CMakeFiles/aligned_allocator_test.dir/aligned_allocator_test.cpp.o -c /home/walnut/Developer/libflatarray/test/aligned_allocator_test.cpp
In file included from /home/walnut/Developer/libflatarray/Software/gcc_install_5.0.0-mpx-r214719/include/c++/5.0.0/bits/localefwd.h:40:0,
                 from /home/walnut/Developer/libflatarray/Software/gcc_install_5.0.0-mpx-r214719/include/c++/5.0.0/ios:41,
                 from /home/walnut/Developer/libflatarray/Software/gcc_install_5.0.0-mpx-r214719/include/c++/5.0.0/ostream:38,
                 from /home/walnut/Developer/libflatarray/Software/gcc_install_5.0.0-mpx-r214719/include/c++/5.0.0/iostream:39,
                 from /usr/include/boost/detail/lightweight_test.hpp:66,
                 from /home/walnut/Developer/libflatarray/test/aligned_allocator_test.cpp:8:
/home/walnut/Developer/libflatarray/Software/gcc_install_5.0.0-mpx-r214719/include/c++/5.0.0/x86_64-pc-linux-gnu/bits/c++locale.h:52:23: error: ‘uselocale’ was not declared in this scope
   extern "C" __typeof(uselocale) __uselocale;
                       ^
/home/walnut/Developer/libflatarray/Software/gcc_install_5.0.0-mpx-r214719/include/c++/5.0.0/x86_64-pc-linux-gnu/bits/c++locale.h: In function ‘int std::__convert_from_v(__locale_struct* const&, char*, int, const char*, ...)’:
/home/walnut/Developer/libflatarray/Software/gcc_install_5.0.0-mpx-r214719/include/c++/5.0.0/x86_64-pc-linux-gnu/bits/c++locale.h:75:53: error: ‘__gnu_cxx::__uselocale’ cannot be used as a function
     __c_locale __old = __gnu_cxx::__uselocale(__cloc);
                                                     ^
/home/walnut/Developer/libflatarray/Software/gcc_install_5.0.0-mpx-r214719/include/c++/5.0.0/x86_64-pc-linux-gnu/bits/c++locale.h:100:33: error: ‘__gnu_cxx::__uselocale’ cannot be used as a function
     __gnu_cxx::__uselocale(__old);
                                 ^
In file included from /usr/include/stdio.h:934:0,
                 from /usr/include/malloc.h:24,
                 from /home/walnut/Developer/libflatarray/src/libflatarray/aligned_allocator.hpp:15,
                 from /home/walnut/Developer/libflatarray/test/aligned_allocator_test.cpp:9:
/usr/include/bits/stdio.h: In function ‘__ssize_t getline(char**, size_t*, FILE*)’:
/usr/include/bits/stdio.h:117:52: error: ‘__getdelim’ was not declared in this scope
   return __getdelim (__lineptr, __n, '\n', __stream);
                                                    ^
make[2]: *** [test/CMakeFiles/aligned_allocator_test.dir/aligned_allocator_test.cpp.o] Error 1
make[2]: Leaving directory `/home/walnut/Developer/libflatarray/build/Linux-x86_64'
make[1]: *** [test/CMakeFiles/aligned_allocator_test.dir/all] Error 2
make[1]: Leaving directory `/home/walnut/Developer/libflatarray/build/Linux-x86_64'
make: *** [all] Error 2

• Nick Stokes - g++ cross distro compilation problem

  necessary pieces for the C++ runtime library to be configured with
  --enable-clocale=gnu (it will be used automatically if configure
  detects it is supported)
  Apparently on the system where it's installed something is missing.
  Probably something in glibc, as Ian suggests.

  • Jonathan Wakely - Re: g++ cross distro compilation problem

  You need to find why uselocale is not defined here:

  /opt/gcc/4.4.3/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.4.3/../../../../include/c++/4.4.3/x86_64-unknown-linux-gnu/bits/c++locale.h:52:
  error: 'uselocale' was not declared in this scope

  Try using the -E flag and examining the preprocessed output. Do that on both mismatchines and look for differences in the path or contents of the locale.h header. To minimise the output it should be sufficient to do:
  echo '#include ' | g++ -E -x c++ -

  • Nick Stokes - Re: g++ cross distro compilation problem

  Great! This indeed revealed it. In /usr/include/locale.h (same location, line 133, in both distros actually) there is #ifdef __USE_GNU on CentOS version, which is #ifdef __USE_XOPEN2K8 in SUSE's version. So, in fact if I define `__USE_XOPEN2K8' while compiling on SUSE, it works. Hmm, go figure.. This can not be the right way to do this. What am I missing?

• trick for debugging make VERBOSE=1 ref

• Info

$ libflatarray/Software/gcc_install_5.0.0-mpx-r214719/bin/x86_64-pc-linux-gnu-c++ -v
Using built-in specs.
COLLECT_GCC=/home/walnut/Developer/libflatarray/Software/gcc_install_5.0.0-mpx-r214719/bin/x86_64-pc-linux-gnu-c++
COLLECT_LTO_WRAPPER=/home/walnut/Developer/libflatarray/Software/gcc_install_5.0.0-mpx-r214719/bin/../libexec/gcc/x86_64-pc-linux-gnu/5.0.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /iclmnt/msticlxl6_sharedws/users/saguriev/gcc/T/gcc_5.0.0-mpx-r214719/configure --enable-bootstrap=no --enable-languages=c,c++,fortran,objc,lto --enable-plugin --enable-shared --build=x86_64-pc-linux-gnu --target=x86_64-pc-linux-gnu --with-gmp=/iclmnt/msticlxl6_sharedws/users/saguriev/gcc/util --with-mpc=/iclmnt/msticlxl6_sharedws/users/saguriev/gcc/util --with-mfpr=/iclmnt/msticlxl6_sharedws/users/saguriev/gcc/util --prefix=/iclmnt/msticlxl6_sharedws/users/saguriev/gcc/T/gcc_install_5.0.0-mpx-r214719 --enable-multilib --enable-initfini-array --enable-cloog-backend=isl --with-fpmath=sse --with-as=/iclmnt/msticlxl6_sharedws/users/saguriev/binutils/T/binutils-gdb_install_2.24.51.20140422/bin/as CC=/iclmnt/msticlxl6_sharedws/users/saguriev/gcc/bin/gcc CXX=/iclmnt/msticlxl6_sharedws/users/saguriev/gcc/bin/g++
Thread model: posix
gcc version 5.0.0 20140925 (experimental) (GCC)