Linker fails on ARM64

[sagitter]

Hi all.

PPSSPP-1.10.3 cannot be compiled against pre-built PPSSPP-FFmpeg's libraries on ARM64 architectures (see also hrydgard/ppsspp#13849):

/usr/bin/ld: ../ffmpeg/linux/aarch64/lib/libavformat.a(avienc.o): relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `__stack_chk_guard@@GLIBC_2.17' which may bind externally can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: ../ffmpeg/linux/aarch64/lib/libavformat.a(avienc.o)(.text+0xc): unresolvable R_AARCH64_ADR_PREL_PG_HI21 relocation against symbol `__stack_chk_guard@@GLIBC_2.17'
/usr/bin/ld: final link failed: bad value

Full build log: here
Fedora 34 (devel branch) - aarch64
GCC-11.0.0
PPSSPP-1.10.3
PPSSPP-ffmpeg commit 0b28335

[unknownbrackets]

Does it work to recompile the code in this repo and link? Or is more needed than that?

-[Unknown]

[0x647262]

@unknownbrackets I'll take a whack at this since I ran into this on: https://gitlab.com/fdroid/fdroiddata/-/merge_requests/3083

[unknownbrackets]

In latest master, you can likely use system FFmpeg.

The only (known) limitation of newer FFmpeg versions right now is that all videos are forced to 29.97 FPS. I'm not aware of any games that use videos with other frame rates, but it's theoretically possible.

Unfortunately, FFmpeg doesn't handle framerate properly anymore when you add a stream it didn't autodetect. The trouble is, the PSP API exposed to games allows explicitly adding streams not autodetected, and games expect it to work. So we're living with the 29.97 problem for now.

FFmpeg v3.0.x can handle this properly, so if you can just recompile that (we have a script in that repo with minimal FFmpeg configure needs) and statically link it, you'll get full functionality.

The old video/audio formats PSP games use haven't really seen much excitement in later FFmpeg versions, so you're not missing much using the older version.

-[Unknown]

[0x647262]

Passing:

NDK=/opt/android-sdk/ndk/21.4.7075529
NDK_PREBUILT=$NDK/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64
NDK_PREBUILTLLVM=$NDK/toolchains/llvm/prebuilt/linux-x86_64

to android_arm64-v8a.sh, I receive the following error(s):

In file included from libavformat/apetag.c:23:
/opt/android-sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/lib64/clang/9.0.9/include/inttypes.h:21:15: fatal error: 'inttypes.h' file not found
#include_next <inttypes.h>
              ^~~~~~~~~~~~
In file included from libavformat/avidec.c:22:
/opt/android-sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/lib64/clang/9.0.9/include/inttypes.h:21:15: fatal error: 'inttypes.h' file not found
#include_next <inttypes.h>
              ^~~~~~~~~~~~
In file included from libavformat/allformats.c:22:
libavformat/avformat.h:316:10: fatal error: 'time.h' file not found
#include <time.h>
         ^~~~~~~~
In file included from libavformat/aacdec.c:24:
libavformat/avformat.h:316:10: fatal error: 'time.h' file not found
#include <time.h>
         ^~~~~~~~
1 error generated.
1 error generated.
make: *** [common.mak:60: libavformat/apetag.o] Error 1
make: *** Waiting for unfinished jobs....
make: *** [common.mak:60: libavformat/aacdec.o] Error 1
1 error generated.
make: *** [common.mak:60: libavformat/allformats.o] Error 1
1 error generated.
make: *** [common.mak:60: libavformat/avidec.o] Error 1

Looking at the directory structure of my NDK, I'm unsure what's out of place.

21:31:50 drb@aether:/home/drb
 + fd inttypes.h /opt/android-sdk/ndk/21.4.7075529/
/opt/android-sdk/ndk/21.4.7075529/sources/android/support/include/inttypes.h
/opt/android-sdk/ndk/21.4.7075529/sources/cxx-stl/llvm-libc++/include/inttypes.h
/opt/android-sdk/ndk/21.4.7075529/sysroot/usr/include/inttypes.h
/opt/android-sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/lib64/clang/9.0.9/include/inttypes.h
/opt/android-sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/c++/v1/inttypes.h
/opt/android-sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/inttypes.h
/opt/android-sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/local/include/inttypes.h
/opt/android-sdk/ndk/21.4.7075529/toolchains/renderscript/prebuilt/linux-x86_64/clang-include/inttypes.h

My C-foo isn't as great as it once was, so I'm not entirely sure why #include_next <inttypes.h> is causing the build to fail (as the files are seemingly there).

It looks like time.h is also present both in the NDK directory, and the project itself:

21:42:32 drb@aether:/home/drb
 + fd ^time.h /opt/android-sdk/ndk/21.4.7075529/
/opt/android-sdk/ndk/21.4.7075529/sysroot/usr/include/linux/time.h
/opt/android-sdk/ndk/21.4.7075529/sysroot/usr/include/sys/time.h
/opt/android-sdk/ndk/21.4.7075529/sysroot/usr/include/time.h
/opt/android-sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/linux/time.h
/opt/android-sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/sys/time.h
/opt/android-sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/time.h
21:43:44 drb@aether:/home/drb
 + fd ^time.h ~/Git/ppsspp/ffmpeg/android/arm64/
/home/drb/Git/ppsspp/ffmpeg/android/arm64/include/libavutil/time.h

Is there something obvious I'm missing here? I've done a bit of searching, but have come up empty handed...

[unknownbrackets]

I'd assume #include <time.h> is meant for the system header, but I'm not sure why it's not finding it. As for #include_next, it's some way of including the next header with exactly the same name (as a way of inheritance.) It's coming from the clang headers...

https://yesimroy.gitbooks.io/android-note/content/compile_ffmpeg_for_android.html

Not sure what's going wrong here. It's like it's not using the other include paths. Maybe using the target needs to be changed to aarch64-linux-android21 now? See:
https://developer.android.com/ndk/guides/other_build_systems

https://github.com/hrydgard/ppsspp-ffmpeg/blob/master/android_arm64-v8a.sh#L93-L96

-[Unknown]

[0x647262]

With the help of: https://stackoverflow.com/a/57707863, https://proandroiddev.com/a-story-about-ffmpeg-in-android-part-i-compilation-898e4a249422 and a locally modified: https://github.com/Javernaut/ffmpeg-android-maker, I was able to get this working.

My fork: https://github.com/0x647262/ffmpeg-android-maker

Is there any special procedure to get these files into this repository or can I just open an MR for them?

EDIT: Got a little too ahead of myself... I'll need to trim the libs down a bit before they're ready for a merge... (a whopping 115M of disk space 😱)

[0x647262]

Copied over this repository's ./configure options and now the built files are within a few MB of the originals (and seem to work fine while testing the debug builds I built).

Is there any special procedure to get the updated files into this repository or can I just open an MR for them?

[unknownbrackets]

You can just PR, although I worry this means it's build configuration specific and will just fail for someone else - we currently ship Android builds using the current ones.

For Linux/Android, I'd really prefer to move to always building it and not tracking the compiled versions at all. Hopefully macOS too, although there have been some tricky bits with iOS iirc.

Windows might be harder because we want to avoid introducing dependencies (i.e. yasm, but especially msys2) as a barrier to people being able to build. This is less of an issue elsewhere where yasm is a single package manager install command away and sometimes already installed with basic build tools anyway.

-[Unknown]

[0x647262]

I've been testing most of the night, and a handful of cutscenes in my game library are mangled (vertically "smeared" across the screen) when building with the new libraries (even on the old 3.0.2 release)... Definitely not comfortable pushing broken libraries into master.

For Linux/Android, I'd really prefer to move to always building it and not tracking the compiled versions at all.

How difficult of a task would this be (even if only for Android and Linux for now)?

EDIT: I fixed the new builds... I had set a ./configure option before --disable-everything, so I ended up shooting myself in the foot. :/ I'll get an MR created, and assuming the scope isn't too crazy I wouldn't mind taking a swing at moving away from this submodule for Android/Linux.

[mrcmunir]

For linux aarch64 you only needed rebuild with --enable-pic

[unknownbrackets]

I may have fixed this - the latest NDK has problems if you pass --sysroot. You also need to define NDK_PLATFORM. See here:
hrydgard/ppsspp#9112 (comment)

-[Unknown]