Skip to content

robotology superbuild handbook

Jump to bottom Edit New page
Luca Muratore edited this page Nov 25, 2015 · 22 revisions

This handbook contains a guide for installing the robotology build system.

The single ''super'' build (the robotology build) pulls code from all the repositories and compiles everything. This mechanisms work similar to a binary distribution, but it provides access to the source code and ''forces'' everybody to get used to the toolchain. Contributions are favored by the fact that everybody has access to the sources and can compile all the modules.

The modules need ot be versioned using the GIT source revision tool. Repositories can be hosted on different servers depending on the choice of the group that maintains it. For example open source code can be hosted on Github.

Requirements

To simplify integration the software is developed and should be tested run on a single well-defined Linux distribution. At the moment this is Linux Ubuntu 14.04. This can change in the future when new releases are made available and contains important updates.

Installation

The first step to begin the procedure is to install git

sudo apt-get install git

and configure it.

git config --global user.name "John Doe"
git config --global user.email "[email protected]"

Then you need to create your ssh keys

To generate a new SSH key just open your terminal and use code below, after substituting the "[email protected]" string with the email you are using (you should use the same both for github and gitlab)

ssh-keygen -t rsa -C "[email protected]"
# Creates a new ssh key using the provided email
# Generating public/private rsa key pair...

Next just use code below

cat ~/.ssh/id_rsa.pub
# ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6eNtGpNGwstc....

and add it to your Github ssh keys If you wish you can add color in git, and notification of the current branch while navigating folders. See the Additional Steps section for more info.

After configuring git you can download the robotology build system

git clone [email protected]:robotology-playground/robotology-superbuild.git

and proceed with the setup, from inside the robotology-superbuild folder

./setup.sh

choosing to install the superbuild with the simulation or robot installation. If you are not on the robot embedded pc (e.g. PC 104 or COM Express) you should choose the simulation installation.

Super Build Workflow

After the first installation you need to refreshing your terminal sourcing your .bashrc

source ~/.bashrc

Multiple robotology development workflows are possible. To begin with the easiest, after building the superbuild, you will have in the build folder a set of out-of-source build folders for your modules. For example, for module_a, you will have a folder $ROBOTOLOGY_ROOT/build/modules/module_a

To build and install it you can proceed in two ways:

  • Run:
cd $ROBOTOLOGY_ROOT/build/modules/module_a
make
make install

which has the advantage of being easily integrated inside the qtcreator/kdevelop workflow, by selecting the right build folder when opening $ROBOTOLOGY_ROOT/modules/module_a/CMakeLists.txt and adding a build step for installation to the project

  • Run:
cd $ROBOTOLOGY_ROOT/build/
make module_a

A note of advice

warning running make module_a from the main build folder (e.g. $ROBOTOLOGY_ROOT/build) build first all the dependency of module_a and it might reset your "custom" CMake variables related to module_a.

Use Cases

Following is a list a cases with relative recommended workflows

Changing Configuration File

Bob is a new team member. He wants to get up to speed, poke around, and learn a bit about what is available before starting to develop. He has a blank-slate computer plus access to robots which are ready to go. He needs to pull a tagged demo named tag_a, run it, modify some configuration file parameters in the module_a module and install them to test its functionalities with the new parameters.

git clone [email protected]:arocchi/robotology.git
git checkout tag_a
sudo scripts/get_dependencies.sh
scripts/bootstrap.sh
cd $ROBOTOLOGY_ROOT/modules/module_a/app/conf
gedit initial_configuration.ini
cd $ROBOTOLOGY_ROOT/build/modules/module_a/
make
make install

If Bob likes the changes, he can commit them

cd $ROBOTOLOGY_ROOT/modules/module_a/app/conf
git add initial_configuration.ini
git commit -m "New configuration improves speed by 50%"
git push

Developing a new module

Joe is starting a new module. He has the superbuild built. He needs to create a new repo, modify a YCM template CMakeFiles.txt and add his dependencies. A little coding later he's ready for his first build and test... Finally he must make his code available to others via the superbuild.

By following the Creating a new Project section in this page, Joe creates its module in Github, and starts developing on it. After seeing that everything works, he follows the same guide to add it to the superbuild.

Updating dependencies

Dave is doing standard development. He pulls the robotology tag that he needs (tag_a) and then goes to his development repositories and branches, codes, tests, and merges his source as needed. Midway through the day Mark pushes changes to a dependency (dep_a) and tells Dave about this. He updates the dependency and continues working. At the end of the day he pushes his updated repositories. Colleagues should now be able to use the new features he created.

While Dave is working, Mark pushes some updated dependency. There are two cases now. If Mark is working on the same branch Dave is using of dep_a (it means they are collaborating for a project deadline!) then Dave needs to update it as soon as Mark pushes the changes online.

The easy way is to run

cd $ROBOTOLOGY_ROOT/build
make update_all
make dep_a

which will make sure the dependency is updated. Note however, that if you are using the alternative workflow, you risk losing your work. Of course, if you are an expert you can always solve that manually, that is

cd $ROBOTOLOGY_ROOT/modules/dep_a
git pull
cd $ROBOTOLOGY_ROOT/build/modules/dep_a
make; make install

In our case, Dave is using the recommended worflow so everything goes smoothly ;)

Superbuild dependencies

Sam is developing and fine tuning control code on the robot hardware. He makes some adjustments to the code and configuration on his laptop (the first of many) and is ready to test it on the robot. He pushes directly to the git repo of the robot, opens an ssh terminal to the robot to compile and run his new code. The robot has the last stable build of all dependencies. Sam has been using the testing version of two dependencies; one will generate a compiler error because of a API change, the other had a bugfix dealing with a math error that causes non-obvious errors which is the behavior of the robot which holds the pre-bugfix code. How does Sam detect and resolve each of these dependency problems? (Said another way, the robotology tag for the laptop and the robotology tag for the robot are different.)

Since the robot is using the branch master of the superbuild, every module and dependency is downloaded from their respective branch (usually master) and repo. Sam on the other side uses a test branch of robotology, which uses test branches for many modules and dependencies. Before going on the robot, Sam should merge the changes from his module from the test branch to the master branch, then test it using a local installation of the robotology build, master version.

cd my_workspace/dep_b;
git checkout tag-robot-is-using
git merge tag-of-latest-bugfix
git push
cd my_workspace
git clone [email protected]:arocchi/robotology.git
mv robotology robotology-robot
cd robotology-robot
git checkout tag-robot-is-using
scripts/bootstrap.sh

When his changes work on the local installation of the robotology build which has the same tag as the robot version, then he can proceed to compile everything on the robot.

In the case when there are API inconsistencies, Sam could decide to build a branch of the robot superbuild by modifying the cmake/ProjectsTags.cmake file. In our case, since dep_a is incompatible with the latest version of YARP, he will do the following on his laptop

cd robotology-robot
git branch new-tag-to-solve-dependencies-conflict
git checkout new-tag-to-solve-dependencies-conflict
cd external/YARP
git log

he will proceed to copy paste the commit hashtag of the desired version of YARP, cc8bc62, then write

echo "set(YARP_TAG cc8bc62)" >> cmake/ProjectsTags.cmake

git add cmake/ProjectsTags.cmake
git push origin new-tag-to-solve-dependencies-conflict
cd build
make update-all
make

After testing the API conflict has been solved, he can go on the robot and run

git clone [email protected]:arocchi/robotology.git
mv robotology robotology-new-tag
cd robotology-new-tag
git checkout new-tag-to-solve-dependencies-conflict
scripts/bootstrap.sh

Integrating issues

Bob, Joe, and Sam are integrating everything. They are each working with some combination of 3 superbuild directories, each superbuild is configured for a particular computer (C2, Motor, Vision). All non-present developers have tagged their repos with the version that can be considered stable or otherwise communicated the version for the integration. Bob, Joe and Sam will spend time tweaking code and configuration to produce a working demo. Then the integration should result in a demo that can be retrieved at any time in the future.

Let's suppose the version for the integration of the previous modules is not always the master version, in particular for mod_a and mod_b we need to use the tags ca8bc61 and 4010df1. Bob takes care of creating a new superbuild branch for the demo, by issuing

git clone [email protected]:arocchi/robotology.git
mv robotology robotology-demo-1
cd robotology-demo-1
cd $ROBOTOLOGY_ROOT/build
cmake ..; make update-all;
cd $ROBOTOLOGY_ROOT/modules/mod_a
git checkout ca8bc61
cd $ROBOTOLOGY_ROOT/modules/mod_b
git checkout 4010df1

scripts/admin/branch-all.sh demo-1
git push origin demo-1

now, everyone can check everything works and continue integrating in the demo superbuild, and issue on their laptops and on the robot computer the commands

git clone [email protected]:arocchi/robotology.git
mv robotology robotology-demo-1
cd robotology-demo-1
git checkout demo-1
scripts/bootstrap.sh

Creating a new Project in Github and adding it to the superbuild

From the new repository page, you need to create a new repository for your module. Once you have the new project, let's call it project_x, you need to create a new file inside the robotology/cmake folder named Buildproject_x.cmake You can take any other file in the same folder as example. Once you add the Buildproject_x.cmake file, you need to modify che main CMakeLists.txt file in the superbuild folder by adding the line:

# Compile project_x
set(SUPERBUILD_project_x ON CACHE BOOL "Flag to compile project_x")
IF(SUPERBUILD_project_x)
        find_or_build_package(project_x)
ENDIF(SUPERBUILD_project_x)

Then you can add the modifications to robotology and push them:

git add cmake/Buildproject_x.cmake
git add CMakeLists.txt
git commit -m "Added project_x to the superbuild"
git push origin master

Freezing dependencies and modules versions

By writing the correct hashtag for dependency or modules in the cmake/ProjectsTags.cmake file it is possible to use a specific set of tags for a build.

Simulation, Robot, Default profiles

One can activate the simulation (control time and simulation time are in sync) or default profile (for use with the robot) by running the (self-explanatory) script

setup.sh

The setup script will call the desired scripts in order to configure the system (e.g. profiles/default/activate-local.sh and profiles/simulation/activate-local.sh)

Known Issues

When editing the include files of my library, I lose changes

When you install your includes, your IDE might prefer to redirect you to the installed versions rather than the one in your src folder when following an #include directive in your code. You might risk not noticing this if you are not used to the workflow of installing your libraries locally. You should therefore list your include files in your CMakeLists.txt file, you can take a look at several other projects already in the superbuild to see how this is done. By doing this you will have a list of includes listed in your IDE which you can edit directly, making sure they come from the src folder instead of the install folder. This step is allowed and encouraged by CMake to communicate to the IDE the list of include files in your project and does not disrupt the compilation process since (the header files are skipped when compiling)[http://www.cmake.org/cmake/help/v2.8.12/cmake.html#prop_sf:HEADER_FILE_ONLY].

Also remember that in qtcreator hovering the mouse over the filename in the top bar shows a tooltip with the full filename.

Additional Steps

You can increase your productivity in using git by following the steps below:

Some Color

Colors in git are very useful, you can enable them by running:

git config --global color.pager true
git config --global color.ui auto

Bash Prompt

Modify your ~/.bashrc file with:

if [ "$color_prompt" = yes ]; then
    PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\[\033[00;32m\]$(__git_ps1 " (%s)")\[\033[00m\]\$ '
else
    PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w$(__git_ps1 " (%s)")\$ '
fi

Dependencies List

If you don't have root access on your machine you will not be able to download the robotology-superbuild dependencies using our scripts; nevertheless here you can find the complete list of the dependencies (for all the repos in the robotology superbuild) for Ubuntu 14.04:

  • build-essential
  • cmake
  • cmake-curses-gui
  • git
  • subversion
  • doxygen
  • graphviz
  • libace-dev
  • libgsl0-dev
  • libgtkmm-2.4-dev
  • libgoocanvasmm-dev
  • libsqlite3-dev
  • swig
  • icub-common
  • libtinyxml2-dev
  • libeigen3-dev
  • libyaml-cpp-dev
  • libxml2-dev
  • ros-indigo-desktop
  • ros-indigo-srdfdom
  • ros-indigo-cmake-modules
  • ros-indigo-openni2-*
  • ros-indigo-moveit-*
  • ros-indigo-joy*
  • ros-indigo-octomap*
  • ros-indigo-urdfdom-py
  • ros-indigo-libg2o
  • ros-indigo-pcl-ros
  • ros-indigo-pcl-conversions
  • ros-indigo-laser-*
  • ros-indigo-ps3joy
  • ros-indigo-stereo-image-proc
  • ros-indigo-image-transport*
  • ros-indigo-rviz-imu-plugin
  • libpcl*
  • python3-sip-dev
  • python-numpy
  • python-scipy
  • python-matplotlib
  • python-pandas
  • libarmadillo-dev
  • libblas-dev
  • liblapack-dev
  • libflann-dev
  • libmumps-seq-dev
  • libpng++-dev
  • python-bs4
  • libsctp-dev
  • mercurial
  • drcsim
  • libhighgui2.4
  • libopensplice64
  • cppcheck
  • python3-empy
  • python3-setuptools
  • python3-nose
  • python3-pip
  • python3-vcstool
  • python-pip
  • gazebo5
  • libgazebo5-dev
  • libopencv2.4 (check $ROBOTOLOGY_ROOT/scripts/admin/get_libopencv_nonfree.sh)
Add a custom footer
Add a custom sidebar
Clone this wiki locally