Skip to content

Running a simple remote cluster

Jump to bottom
Blake Griffith edited this page May 12, 2014 · 2 revisions

Here we describe how to run a set of ipython engines and an ipython controller on a remote machine, and connect to the with a Client from your local machine.

  1. On the remote machine make sure you have a ipython profile with parallel configuration files. You can create one with the command ipython profile create --parallel (this creates a profile_default directory). Then on the remote machine start the cluster using dacluster start or ipcluster --n=4 --engines=MPI.

  2. scp the ipython profile you are using (default in this case) to the local machine.

    

    3. 



scp -r 10.7.0.121:~/.ipython/profile_default ~/profile_default
```


    

  1. Edit the ipcontroller-client.json file. Change the "ssh": "" line to "ssh": "<remote host>".

    

    2. 



sed -i 's/"ssh": "",/"ssh": "",/g' ~/profile_default/security/ipcontroller-client.json
```


    

  1. Move this profile to ~/.ipython/profile_default/ so that IPython uses it by default.

    

    2. 



mv ~/profile_default ~/.ipython
```


Now when you create a client object with IPython.parallel.Client() it should automatically connect to the controller on the remote machine.


You can run the following as a bash script to transfer, edit, and move the profile to the proper location (steps 2, 3, and 4).


#!/usr/bin/env bash
scp -r <remote host>:~/.ipython/profile_default ~/profile_default

sed -i 's/"ssh": "",/"ssh": "<remote host>",/g' ~/profile_default/security/ipcontroller-client.json

rm -rf ~/.ipython/profile_default.bak

mv ~/.ipython/profile_default ~/.ipython/profile_default.bak

mv ~/profile_default ~/.ipython


              


          



  
          

            


            
Clone this wiki locally