Replies: 1 comment
-
Hello. Before anything else, could you please re-format your comment to use a Code Block (see https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#quoting-code) |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I rewrote an R function in Julia to make it faster. I will call this function many times, and I thought about further parallelizing it in R and call the Julia function parallel.
When I do parallel run using “foreach/future”, the parallel loop fails, because the Julia function is not sourced to the threads / environments, I guess. If you add the sourcing to the loop, it gets really slow, obviously. Interestingly, after the foreach loop failed, if you run it again, the function works, but still kind of slow.
I am wondering if there is a proper way to source the function to the threads, and still make sure its fast, like i would execute it sequentially.
Some sample code:
Packages
Generate 210-by-3 matrix
R function to shuffle (randomize) each row of a matrix
Julia function to shuffle (randomize) each row of a matrix
function to load Julia and execute
Benchmark (Julia is 10-20x faster)
Call function 100 times in for loop (Julia faster)
Julia gets really slow with loading the function
Parallel execution of R on 4 cores
This does not work, since function is not sources to parallel threads / environments
UndefVarError:
myFuncJL
not definedWorks but slow because we source function each time in loop
Now works since function was loaded previously? Still kind of slow though
Beta Was this translation helpful? Give feedback.
All reactions