Infrastructure

Galaxy Infrastructure

aka, what does what?

Resources are generally divided into two domains, usegalaxy.org resources for stuff related to running Test/Main, and galaxyproject.org resources for all the other services.

Note: there is a separate playbook, infrastructure-playbook that is responsible for managing most of the non-Galaxy infrastructure, and even some of the low level stuff on Galaxy VMs and nodes. In specific, checking the inventory files of the various Ansible "environments" should give you some insight on what does what, if it's not covered in this document.

usegalaxy.org resources

The following hosts run services for usegalaxy.org and test.galaxyproject.org.

TACC Hosts

These hosts live under the .tacc.utexas.edu domain (but most have aliases under one or more of our domains). Unless needed for the task you are performing, you should use your own TACC account. See Getting Set Up At TACC.

Admin privileges (for those with access) for all hosts in this section is via sudo. On Galaxy servers, code is installed as gxsrc and run as the appropriate (g2test/g2main) Galaxy user.

host	login	purpose	aliases
galaxy-jump	your TACC user	"Jump" host to resources inside of TACC, running Ansible, etc.
galaxy-test	g2test gxsrc	Galaxy Test web, jobs, and database server	test.galaxyproject.org
galaxy-control		RabbitMQ and Slurm controller
galaxy-cvmfs		CVMFS Stratum 1 server	cvmfs1-tacc0.galaxyproject.org
galaxy-main	g2main gxsrc	Galaxy Main load balancer	galaxy-main.usegalaxy.org
galaxy-main1	g2main gxsrc	Galaxy Main web workers, tusd	galaxy-main1.usegalaxy.org
galaxy-main2	g2main gxsrc	Galaxy Main staging web workers, nginx upload module	galaxy-main2.usegalaxy.org
galaxy-main3	g2main gxsrc	Galaxy Main job handlers, workflow schedulers
galaxy-main4	g2main gxsrc	Galaxy Main web celery
galaxy-db	g2main	Galaxy Main database server
galaxy-vgp	g2main gxsrc	Galaxy VGP server	vgp.usegalaxy.org

• The per-host Galaxy Main aliases can be useful for testing the state of each VM independently
• The VMs run on TACC's VMWare cluster
• VMs are configured with varying amounts of memory/CPU by purpose
• Galaxy logs are in journald, accessible on the appropriate VM via your TACC user with journalctl -u galaxy-*
• Galaxy can be restarted by the gxsrc user e.g. systemctl restart galaxy-main-gunicorn@0 or galaxyctl-main graceful

CVMFS Hosts

These hosts live under the .galaxyproject.org domain. CVMFS contents are also reachable via http://datacache.galaxyproject.org/.

host	login	purpose	virtualization host	aliases
cvmfs0-psu0	g2test (update data) root (admin)	CVMFS release manager (stratum 0) server for the data.galaxyproject.org CVMFS repo	westvleteren
cvmfs0-psu1	g2test (update test) g2main (update main)	CVMFS release manager (stratum 0) server for the test.galaxyproject.org and main.galaxyproject.org CVMFS repos	smart3
cvmfs1-psu0	g2test (snapshot test/data) g2main (snapshot main) root (admin)	CVMFS replica server (stratum 1) server for all *.galaxyproject.org CVMFS repos	thumper
cvmfs1-tacc0	g2test (snapshot test/data) g2main (snapshot main)	CVMFS replica server (stratum 1) server for all *.galaxyproject.org CVMFS repos	galaxy-cvmfs.tacc.utexas.edu
cvmfs1-iu0	g2test (snapshot test/data) g2main (snapshot main) centos (admin)	CVMFS replica server (stratum 1) server for all *.galaxyproject.org CVMFS repos	Jetstream @ IU

Distributed Computing Hosts

host	login	purpose	notes
jetstream2.galaxyproject.org	rocky	Pulsar Slurm controller	Jetstream2
jetstream2-vgp.galaxyproject.org	rocky	Pulsar for vgp.usegalaxy.org	Jetstream @ IU
jeltz.galaxyproject.org	root	datacache	Guest of smart1
login3.stampede3.tacc.utexas.edu	xcgalaxy	Pulsar Test/Main megablast/lastz and manual submissions	TACC Stampede2, login from galaxy-jump only
vm030.bridges2.psc.edu	xcgalaxy	Pulsar Large memory tools (Trinity, Unicycler, etc.)	PSC Bridges

galaxyproject.org resources

These hosts live under the .galaxyproject.org domain.

Virtualization hosts

All virtualization hosts run SmartOS unless otherwise noted.

host	guests	notes
smart1	nat1 jeltz eddie sentry	Hosts NAT for private network hosts, datacache server, Test Toolshed, Sentry
smart2	radegast	Hosts Main Toolshed
smart3	duvel stats cvmfs0-psu1	Hosts InfluxDB/Grafana, Galaxy Test CVMFS repo
smart5	leeroy jenkins0	Hosts Jenkins server/worker
thumper.p	cvmfs1-psu0	Hosts CVMFS stratum 1
westvleteren.p	cvmfs0-psu0 westvleteren	Backups CVMFS data repo
orval.p	orval	depot.galaxyproject.org

Service Hosts

Our standalone AWS account (infrastructure@...) is being phased out in favor of the Penn State Cloud Program due to contract advantages and simplified billing. At this time, you must have a Penn State Access Account (i.e. you must be a PSU employee) to gain access to the AWS Dashboard for these services.

host	login	purpose	virtualization host	notes
newtrail	root (admin)	LDAP Kerberos KDC DNS galaxyproject.org email	EC2
artifact	root (admin)	LDAP Kerberos KDC DNS galaxyproject.org email Website proxies	EC2	Proxies galaxyproject.org, training.galaxyproject.org
duvel	root (admin)	LDAP Kerberos KDC DNS	smart3
radegast	shed gxsrc root (admin)	Toolshed	smart2	toolshed.g2.bx.psu.edu
eddie	shed gxsrc root (admin)	Test Toolshed	smart1	testtoolshed.g2.bx.psu.edu
orval	sites depot bioarchive singularity root (admin)	depot	orval.p	depot.galaxyproject.org
stats	root (admin)	InfluxDB/Grafana	smart3	stats.galaxyproject.org
sentry	root (admin)	Sentry	smart1	sentry.galaxyproject.org
rodenbach	root (admin)	DHCP TFTP PSU local network services	metal
leeroy	jenkins root (admin)	Jenkins server	smart5	jenkins.galaxyproject.org
jenkins0	jenkins root (admin)	Jenkins worker	smart5

Non-host Cloud Services

name	login	purpose	provider
wheels	galaxy-wheels IAM user	wheels.galaxyproject.org	S3/CloudFront
docs	galaxy-docs IAM user	docs.galaxyproject.org	S3/CloudFront
galaxyproject.org	galaxy-hub IAM user	galaxyproject.org	S3/EC2 (artifact)
training		training.galaxyproject.org	S3/GitHub Pages (live) S3/EC2 (artifact) (archive)
biostar.usegalaxy.org		biostar.usegalaxy.org archive	S3/EC2 (artifact)
status	[email protected]	Galaxy services status	statuspage.io