Home
Omnia currently uses the Jenkins CI framework to automate building and testing of conda packages and tarball distributions of Omnia components.
The master Jenkins installation is http://jenkins.choderalab.org, and is hosted in a special network DMZ at the Memorial Sloan Kettering Cancer Center and administered by the Open Systems Group and the Chodera lab.
[project page] [configuration] [last build log]
osx builds are performed on macmini.choderalab.org, a Mac Mini running OS X 10.9.5. There are plans to move this to a VM to have greater control over the build environment.
Builds are triggered whenever a commit is pushed to the conda-recipes repo. The repo is polled every five minutes for changes.
Roughly, the build proceeds as:
- Create a fresh
minicondainstallation - Ensure that the only
condachannel present is https://conda.binstar.org/omnia
rm -f ~/.condarc # this is checked for channels
conda config --add channels http://conda.binstar.org/omnia # add omnia channel- Install some necessary packages for the build:
conda install --yes --quiet conda-build jinja2 binstar pip- Check out conda recipes
- Attempt to build conda recipes, checking to not rebuild packages already present
cd conda-recipes ; ./conda-build-all --check-against omnia ./*
- Upload built packages to the
omniabinstar channel
binstar --quiet -t $BINSTAR_TOKEN upload -u omnia miniconda/conda-bld/*/*.tar.bz2
Note that the binstar token stored in $BINSTAR_TOKEN must not be made public, since a malicious attacker could replace our packages if they got hold of it. Instead, the binstar token must be encrypted. Currently, we use the Environment Injector Plugin feature "Inject passwords to the build as environment variables" for this purpose on a per-project basis.
[project page] [configuration] [last build log]
linux builds are performed using a vagrant box provisioned from this set of provisioning scripts within the Omnia virtual-machines repo. The VM is run on macmini.choderalab.org.
The build proceeds roughly as with the osx build above, except this build is done within the VM.
Since provisioning the vagrant box takes several hours, we avoid reprovisioning a new box every time. The first commands in the build are
# Start the VM
cd virtual-machines/linux
vagrant halt
vagrant upIf the Jenkins $WORKSPACE directory was completely deleted (via "Workspace > Wipe Out Current Workspace"), the vagrant box will be provisioned from scratch and then booted up. Otherwise, if the vagrant box is found in the existing workspace, it will simply be booted up (which only takes a few seconds). Note that the texlive installation portion of provisioning the vagrant box is prone to failure, and may cause the build to fail if the workspace has recently been deleted.
Windows builds are currently not automated, though there are plans to do this in the near future. @bas-rustenburg will head up this effort.