sponge/README at master · stpierre/sponge · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
Sponge is a basic web frontend to Pulp (http://www.pulpproject.org).
It is based around the workflow described at
https://github.com/stpierre/sponge/blob/master/doc/paper.pdf?raw=true
and http://www.usenix.org/events/lisa11/tech/full_papers/Pierre.pdf
(login required).

Sponge runs on Django, using Pulp (of course) and Celery.

Sponge includes code from:

  * jQuery (http://jquery.org)
  * jQuery Tablesorter (http://tablesorter.com)
  * jQuery Quicksearch (https://github.com/riklomas/quicksearch)

OVERVIEW

You should really read
https://github.com/stpierre/sponge/blob/master/doc/paper.pdf?raw=true
for a full understanding of how Sponge works.  This is the tl;dr, plus
implementation some details.

Sponge is a partial web frontend to Pulp, so you need to understand
Pulp before you can understand Sponge.  Go do that; we'll wait for
you.

Okay, now that you have a running Pulp server (isn't it cool?), we can
talk about Sponge.  Sponge does two things:

1.  Sponge takes over your Pulp sync schedule and runs all of your
syncs every 24 hours.  (This is configurable.)  Sponge assumes that
you have set up filters such that this is safe.

2.  Sponge provides a web interface for the aforementioned workflow.
Basically, you'll sync upstream repos, and then clone your local repos
and use filters to mediate the syncs from the local mirror of the
upstream stuff.

For instance, you might mirror the CentOS 6 'os' and 'updates' repos,
but you don't want the 'kernel' package to get updated automatically.
So you'd create local repos like:

CentOS 6 -> CentOS 6 Testing -> CentOS 6 Stable

"CentOS 6" would be a direct mirror of upstream, and would not
actually be used by any systems.  Some systems would use the "CentOS 6
Testing" repos, which is a clone of "CentOS 6," while most systems
would use "CentOS 6 Stable."  When a new kernel is released, you would
promote it manually to Testing, and, if all went well, thence to
Stable.

INSTALLATION

Sponge runs on RHEL 6 and Fedora 15+.  If God smiles upon you, you
might even be able to get it running on RHEL 5, but don't count on
it.

These instructions deal with getting Sponge running under the Django
development server; getting it running under Apache is significantly
more difficult due to poor packaging of Celery.  I'll add instructions
for that some day.

First, clone the Sponge repo.  These instructions will assume you have
cloned it into ~/sponge.

Then install some needed packages:

$ sudo yum install django-celery rabbitmq-server pulp-consumer \
    python-setuptools python-dateutil python-sqlite2 django-picklefield

Next, configure /etc/pulp/consumer/consumer.conf to point at your Pulp
server.

Now you can sync up your database and start the Django development
server:

cd ~/sponge/Sponge
ln -s ../media
python manage.py syncdb
python manage.py runserver

At this point you should be able to access Sponge at
http://localhost:8000/.  You can log in using your existing Pulp
credentials.

Note that when running under the Django development server, the dev
server also serves the media; because all of Sponge requires a login,
you will not be able to access the CSS stuff at the login screen
itself.  I promise it gets prettier once you log in.  (When deploying
Sponge in production, this is not an issue.)

See README.apache for some rough instructions to get Sponge running
under Apache.

KNOWN ISSUES

Sponge is alpha code, at best.  But that's okay, because the same is
true of Pulp.  At a minimum, the following known issues exist:

* Sponge does not implement (and does not try to implement) a full GUI
  for Pulp.  It implements a particular workflow, and does not have
  support for things outside that workflow -- editing roles and
  permissions, for example.  Those can still be done via Pulp.  If
  you're looking for a full frontend to Pulp, check out
  http://katello.org/

* Sponge does no caching, which makes it _slow_.  Real slow.  At some
  point I'll likely add some limited caching, but because Sponge just
  front-ends Pulp, and everything can still be done via Pulp, cache
  expiration isn't really a solvable problem.  So for the meantime,
  Sponge is slow.

* Sponge does not use Django models to implement its conversation with
  Pulp.  It calls the Pulp Python client API directly, without any
  sort of model wrapping.  This makes writing new views for simple
  things harder than it should be.  The right way is clearly to write
  a Pulp model, but that's hard.