when using PortAudio:ERROR: InitError: Could not load pa_shim library

[lhxhaxan]

julia> using PortAudio
INFO: Recompiling stale cache file C:\Users\lihan.julia\lib\v0.6\RingBuffers.ji for module RingBuffers.
INFO: Recompiling stale cache file C:\Users\lihan.julia\lib\v0.6\PortAudio.ji for module PortAudio.
ERROR: InitError: Could not load pa_shim library, please file an issue at https://github.com/JuliaAudio/RingBuffers.jl/issues with your versioninfo() output
Stacktrace:
[1] init_pa_shim() at C:\Users\lihan.julia\v0.6\PortAudio\src\pa_shim.jl:25
[2] init() at C:\Users\lihan.julia\v0.6\PortAudio\src\PortAudio.jl:20
[3] _include_from_serialized(::String) at .\loading.jl:157
[4] _require_from_serialized(::Int64, ::Symbol, ::String, ::Bool) at .\loading.jl:200
[5] _require(::Symbol) at .\loading.jl:464
[6] require(::Symbol) at .\loading.jl:405
during initialization of module PortAudio

===================================================================

julia> versioninfo()
Julia Version 0.6.4
Commit 9d11f62bcb* (2018-07-09 19:09 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
WORD_SIZE: 64
BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell MAX_THREADS=16)
LAPACK: libopenblas64_
LIBM: libopenlibm
LLVM: libLLVM-3.9.1 (ORCJIT, broadwell)

[ssfrr]

Yeah, there are some known issues with how the PortAudio and RingBuffers code find each other. I'm working on a new BinaryBuilder-based system for both of them that should alleviate the issue.

[mikelserralde]

[ Info: Precompiling PortAudio [80ea8bcb-4634-5cb3-8ee8-a132660d1d2d]
ERROR: LoadError: Could not load pa_shim library, please file an issue at https://github.com/JuliaAudio/RingBuffers.jl/issues with your `versioninfo()` output
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] find_pa_shim() at C:\Users\Mikel\.julia\packages\PortAudio\5va8g\src\pa_shim.jl:25
 [3] top-level scope at none:0
 [4] include at .\boot.jl:326 [inlined]
 [5] include_relative(::Module, ::String) at .\loading.jl:1038
 [6] include(::Module, ::String) at .\sysimg.jl:29
 [7] top-level scope at none:2
 [8] eval at .\boot.jl:328 [inlined]
 [9] eval(::Expr) at .\client.jl:404
 [10] top-level scope at .\none:3
in expression starting at C:\Users\Mikel\.julia\packages\PortAudio\5va8g\src\PortAudio.jl:395
ERROR: Failed to precompile PortAudio [80ea8bcb-4634-5cb3-8ee8-a132660d1d2d] to C:\Users\Mikel\.julia\compiled\v1.1\PortAudio\Q9ZEe.ji.
Stacktrace:
 [1] compilecache(::Base.PkgId, ::String) at .\loading.jl:1197
 [2] _require(::Base.PkgId) at .\loading.jl:960
 [3] require(::Base.PkgId) at .\loading.jl:858
 [4] require(::Module, ::Symbol) at .\loading.jl:853

julia> versioninfo()
Julia Version 1.1.1
Commit 55e36cc308 (2019-05-16 04:10 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i5-4310U CPU @ 2.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, haswell)
Environment:
  JULIA_EDITOR = "C:\Users\Mikel\AppData\Local\atom\app-1.38.2\atom.exe"  -a
  JULIA_NUM_THREADS = 2

I'm getting a similar issue. Not sure how to fix this.

[ssfrr]

does it help if you run build RingBuffers and build PortAudio from the pkg> prompt?

[mikelserralde]

Yeah, it seemingly builds everything, but it throws the same error.

(v1.1) pkg> build RingBuffers
  Building RingBuffers → `C:\Users\Mikel\.julia\packages\RingBuffers\P7bLa\deps\build.log`

(v1.1) pkg> build PortAudio
  Building LibCURL ─────────→ `C:\Users\Mikel\.julia\packages\LibCURL\teHPG\deps\build.log`
  Building WinRPM ──────────→ `C:\Users\Mikel\.julia\packages\WinRPM\Y9QdZ\deps\build.log`
  Building RingBuffers ─────→ `C:\Users\Mikel\.julia\packages\RingBuffers\P7bLa\deps\build.log`
  Building SpecialFunctions → `C:\Users\Mikel\.julia\packages\SpecialFunctions\fvheQ\deps\build.log`
  Building Conda ───────────→ `C:\Users\Mikel\.julia\packages\Conda\kLXeC\deps\build.log`
  Building FFTW ────────────→ `C:\Users\Mikel\.julia\packages\FFTW\p7sLQ\deps\build.log`
  Building Homebrew ────────→ `C:\Users\Mikel\.julia\packages\Homebrew\s09IX\deps\build.log`
  Building PortAudio ───────→ `C:\Users\Mikel\.julia\packages\PortAudio\5va8g\deps\build.log`

julia> using PortAudio
[ Info: Precompiling PortAudio [80ea8bcb-4634-5cb3-8ee8-a132660d1d2d]
ERROR: LoadError: Could not load pa_shim library, please file an issue at https://github.com/JuliaAudio/RingBuffers.jl/issues with your `versioninfo()` output
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] find_pa_shim() at C:\Users\Mikel\.julia\packages\PortAudio\5va8g\src\pa_shim.jl:25
 [3] top-level scope at none:0
 [4] include at .\boot.jl:326 [inlined]
 [5] include_relative(::Module, ::String) at .\loading.jl:1038
 [6] include(::Module, ::String) at .\sysimg.jl:29
 [7] top-level scope at none:2
 [8] eval at .\boot.jl:328 [inlined]
 [9] eval(::Expr) at .\client.jl:404
 [10] top-level scope at .\none:3
in expression starting at C:\Users\Mikel\.julia\packages\PortAudio\5va8g\src\PortAudio.jl:395
ERROR: Failed to precompile PortAudio [80ea8bcb-4634-5cb3-8ee8-a132660d1d2d] to C:\Users\Mikel\.julia\compiled\v1.1\PortAudio\Q9ZEe.ji.
Stacktrace:
 [1] compilecache(::Base.PkgId, ::String) at .\loading.jl:1197
 [2] _require(::Base.PkgId) at .\loading.jl:960
 [3] require(::Base.PkgId) at .\loading.jl:858
 [4] require(::Module, ::Symbol) at .\loading.jl:853

julia>

[ssfrr]

what versions of PortAudio and RingBuffers are you using? Does it work if you try the master branch of RingBuffers and the julia1 branch of PortAudio?

[mikelserralde]

I've been running the latest version of RingBuffers and the #julia1 branch of PortAudio. It still has been throwing the same errors.

    Status `C:\Users\Mikel\.julia\environments\v1.1\Project.toml`
  [9e28174c] BinDeps v0.8.10
  [34da2185] Compat v2.1.0
  [80ea8bcb] PortAudio v1.0.0+ #julia1 (https://github.com/JuliaAudio/PortAudio.jl.git)
  [f6d8bcc6] RingBuffers v1.1.3
  [bd7594eb] SampledSignals v2.0.0

[ssfrr]

Hmm, I'm not sure what's going on there then. I'm not on my windows setup very often so I'm not sure I'll get a chance to troubleshoot this in the near future.

[mikelserralde]

Thank you. I'll keep on trying and update if any progress is made.

[kyokke]

Hi I got same error as @mikelserralde but,
After I call some functions defined in RingBuffers, using PortAudio seems work.

I have not confirm whether all PortAudio.jl functions works correctly yet but
I hope it would be helpful for you.

julia> versioninfo()                                                                                                    
Julia Version 1.2.0-rc2.0                                                                                               
Commit 9248bf7687 (2019-07-08 19:42 UTC)                                                                                
Platform Info:                                                                                                            
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz 
WORD_SIZE: 64                                                                                                          
LIBM: libopenlibm                                                                                                      
LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

julia> using RingBuffers

julia> PaUtilRingBuffer(1024,256)
PaUtilRingBuffer(256, 0, 0, 511, 255, 1024, Ptr{Int8} @0x00000000259a0930)

julia> using PortAudio

julia> stream = PortAudioStream(2,2)
PortAudioStream{Float32}
  Samplerate: 44100.0Hz
  Buffer Size: 4096 frames
  2 channel sink: "�X�s�[�J�[ / �w�b�h�t�H�� (Real"
  2 channel source: "�}�C�N�z�� (Realtek High Defini"

[ssfrr]

Thanks, that's a useful clue. The pa_shim C library depends on the functions in the pa_ringbuffer library that ships with RingBuffers.jl. I thought it would have already been loaded because using PortAudio will internally call using RingBuffers, but maybe for some reason that's not happening. Does it work if you just call using RingBuffers first, or do you actually need to call functions from it?

[kyokke]

Without calling actual function, the same error occurs.
The actual path to userfolder is replaced by "%USERFOLDER%"

Directry load PortAudio

julia> using PortAudio
ERROR: InitError: could not load library "%USERFOLDER%.julia\packages\PortAudio\5va8g\src\..\deps\usr\lib\pa_shim_x86_64-w64-mingw32"
The specified module could not be found.

Stacktrace:
 [1] #dlopen#3(::Bool, ::typeof(Libdl.dlopen), ::String, ::UInt32) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Libdl\src\Libdl.jl:109
 [2] dlopen at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [3] set_global_callbacks() at %USERFOLDER%\.julia\packages\PortAudio\5va8g\src\PortAudio.jl:415
 [4] __init__() at %USERFOLDER%\.julia\packages\PortAudio\5va8g\src\PortAudio.jl:441
 [5] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:685
 [6] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:765
 [7] _require(::Base.PkgId) at .\loading.jl:990
 [8] require(::Base.PkgId) at .\loading.jl:911
 [9] require(::Module, ::Symbol) at .\loading.jl:906
during initialization of module PortAudio

Load RingBuffers and then PortAudio (without call function)

julia> using RingBuffers

julia> using PortAudio
ERROR: InitError: could not load library "%USERFOLDER%\.julia\packages\PortAudio\5va8g\src\..\deps\usr\lib\pa_shim_x86_64-w64-mingw32"
The specified module could not be found.

Stacktrace:
 [1] #dlopen#3(::Bool, ::typeof(Libdl.dlopen), ::String, ::UInt32) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Libdl\src\Libdl.jl:109
 [2] dlopen at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [3] set_global_callbacks() at %USERFOLDER%\.julia\packages\PortAudio\5va8g\src\PortAudio.jl:415
 [4] __init__() at %USERFOLDER%\.julia\packages\PortAudio\5va8g\src\PortAudio.jl:441
 [5] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:685
 [6] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:765
 [7] _require(::Base.PkgId) at .\loading.jl:990
 [8] require(::Base.PkgId) at .\loading.jl:911
 [9] require(::Module, ::Symbol) at .\loading.jl:906
during initialization of module PortAudio

I have one thing to apology.Originally I got the same error as @mikelserralde but when I send the previous comment, my error was different than @mikelserralde.

I will clean my Julia environment and make clear how and when different error was generated.

[kyokke]

I could regenerate the problem from clean julia environment as follows

My Environment and version info:

julia> versioninfo()
Julia Version 1.2.0-rc2.0
Commit 9248bf7687 (2019-07-08 19:42 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

At first, I installed packages and build them

pkg> add RingBuffers
pkg> add SampledSignals
pkg> PortAudio#julia1
pkg> build RingBuffers
pkg> build PortAudio
pkg> using Libdl

now you can regenerate the problem same as @mikelserralde

julia> using PortAudio
[ Info: Precompiling PortAudio [80ea8bcb-4634-5cb3-8ee8-a132660d1d2d]
ERROR: LoadError: Could not load pa_shim library, please file an issue at https://github.com/JuliaAudio/RingBuffers.jl/issues with your `versioninfo()` output
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] find_pa_shim() at %USER_FOLDER%.julia\packages\PortAudio\5va8g\src\pa_shim.jl:25
 [3] top-level scope at %USER_FOLDER%.julia\packages\PortAudio\5va8g\src\PortAudio.jl:395
 [4] include at .\boot.jl:328 [inlined]
 [5] include_relative(::Module, ::String) at .\loading.jl:1094
 [6] include(::Module, ::String) at .\Base.jl:31
 [7] top-level scope at none:2
 [8] eval at .\boot.jl:330 [inlined]
 [9] eval(::Expr) at .\client.jl:432
 [10] top-level scope at .\none:3
in expression starting at %USER_FOLDER%.julia\packages\PortAudio\5va8g\src\PortAudio.jl:395
ERROR: Failed to precompile PortAudio [80ea8bcb-4634-5cb3-8ee8-a132660d1d2d] to %USER_FOLDER%.julia\compiled\v1.2\PortAudio\Q9ZEe.ji.
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at .\loading.jl:1253
 [3] _require(::Base.PkgId) at .\loading.jl:1013
 [4] require(::Base.PkgId) at .\loading.jl:911
 [5] require(::Module, ::Symbol) at .\loading.jl:906

I don't know why but Libdl.find_library cannot locate the dll so ,
I forced pa_shim() function to returns the full path of the dll on my PC.
restart julia and then:

julia> using Libdl
julia> Libdl.dlopen("%USER_FOLDER%.julia/packages/RingBuffers/P7bLa/deps/usr/lib/pa_ringbuffer_x86_64-w64-mingw32.dll")
julia> using PortAudio
julia> PortAudio.Pa_GetDefaultInputDevice()
1

I could compile the PortAudio and it seems work! Please note that I had to dlopen pa_ringbuffer dll to compile PortAudio.

when you restart the julia, you get InitError by

julia> using PortAudio

ERROR: InitError: could not load library "%USER_FOLDER%/.julia/packages/PortAudio/5va8g/deps/usr/lib/pa_shim_x86_64-w64-mingw32.dll"
The specified module could not be found.

Stacktrace:
 [1] #dlopen#3(::Bool, ::typeof(Libdl.dlopen), ::String, ::UInt32) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Libdl\src\Libdl.jl:109
 [2] dlopen at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [3] set_global_callbacks() at %USER_FOLDER%\.julia\packages\PortAudio\5va8g\src\PortAudio.jl:415
 [4] __init__() at %USER_FOLDER%\.julia\packages\PortAudio\5va8g\src\PortAudio.jl:441
 [5] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:685
 [6] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:765
 [7] _require(::Base.PkgId) at .\loading.jl:990
 [8] require(::Base.PkgId) at .\loading.jl:911
 [9] require(::Module, ::Symbol) at .\loading.jl:906
during initialization of module PortAudio

# if you retry InitError doesn't occur, but it doesn't seem work correctly. 
julia> using PortAudio

julia> PortAudio.Pa_GetDefaultInputDevice()
-1

As I commented yesterday, you can avoid InitError by restarting julia and It seems O.K.

julia> using RingBuffers
julia> PaUtilRingBuffer(1024,256)
PaUtilRingBuffer(256, 0, 0, 511, 255, 1024, Ptr{Int8} @0x0000000024a4b240)

julia> using PortAudio
julia> PortAudio.Pa_GetDefaultInputDevice()
1

Without call of PaUtilRingBuffer, you failed.

julia> using RingBuffers
julia> using PortAudio

# -> the same InitError occurs

QUESTION: RingBuffer v1.1.3 and SampledSignals v2.0.0 are suitable version for PortAudio on the branch named "julia1" ?

[ssfrr]

RingBuffers should be at master - does that help?

[kyokke]

@ssfrr thank you for reply.

For the combination of RingBuffers#master, PortAudio#julia1 and Julia v1.1.1/v1.2.0-rc2,
recompile of PortAudio was failed in init of PortAudio:

julia> using Libdl
julia> Libdl.dlopen("%USER_FOLDER%.julia/packages/RingBuffers/P7bLa/deps/usr/lib/pa_ringbuffer.dll")
julia> using PortAudio
ERROR: InitError: could not load library "%USERFOLDER%.julia\packages\PortAudio\5va8g\src\..\deps\usr\lib\pa_shim_x86_64-w64-mingw32"
The specified module could not be found.

Stacktrace:
 [1] #dlopen#3(::Bool, ::typeof(Libdl.dlopen), ::String, ::UInt32) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Libdl\src\Libdl.jl:109
 [2] dlopen at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [3] set_global_callbacks() at %USERFOLDER%\.julia\packages\PortAudio\5va8g\src\PortAudio.jl:415
 [4] __init__() at %USERFOLDER%\.julia\packages\PortAudio\5va8g\src\PortAudio.jl:441
 [5] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:685
 [6] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:765
 [7] _require(::Base.PkgId) at .\loading.jl:990
 [8] require(::Base.PkgId) at .\loading.jl:911
 [9] require(::Module, ::Symbol) at .\loading.jl:906
during initialization of module PortAudio

I have another observation.
For pa_ringbuffer.dll and libportaudio-2.dll, Libdl.dlopen, Libdl.find_library and ccall seems work correctly but for pa_shim*.dll , Libdl.dlopen and Libdl.find_library was failed.

According to the a series of observation, trouble would be pa_shim and integration of DLLs of RingBuffers and pa_shim.

I will look into this problem and send report to you but It will take a while because I am completely new to Julia and have to learn how to make the packages and modules, how to debug and other related items.

It would be helpful for me if you can give me the information about version combination of PortAudio.jl and its dependencies which is well tested on Windows.

[kyokke]

I could specify one of the causes of this issue.

pa_ringbuffer*.dll , which is dependencies of pa_shim*.dll, cannot be found from dll search paths,
when pa_shim*.dll is loaded. Loading pa_shim*.dll is failed if its dependencies cannot be loaded.

I have confirmed that I could import PortAudio packages successfully with the following workarounds:

• if you use RingBuffers#master, rename pa_ringbuffer.dll to pa_ringbuffer_x86_64-w64-mingw32.dll
• copy pa_ringbuffer_x86_64-w64-mingw32.dll (renamed dll from RingBuffers#maseter or original dll from RingBuffers#v1.1.3) into the same folder as pa_shim*.dll

So This issue in Windows could be solved by ...

1. adding path of pa_ringbuffer*.dll before you load pa_shim*.dll
2. update pa_shim*.dll to match dll name of RingBuffers.jl ( It might be one option to modfy PortAudio.jl to use BinaryBuilder.jl )

Even afeter the above workarounds,
I have "concurrency violation detected" Error during write() and read() function. I think this error should be discussed in other ticket.