v1.1.0 Merge

v1.1.0 Merge

Author: dirtycajunrice

CHANGELOG.md

@@ -1,6 +1,30 @@
 # Change Log
 
-## [1.0.0](https://github.com/pyouroboros/ouroboros/tree/1.0.0) (2019-01-22)
+## [1.1.0](https://github.com/pyouroboros/ouroboros/tree/1.1.0) (2019-01-26)
+[Full Changelog](https://github.com/pyouroboros/ouroboros/compare/1.0.0...1.1.0)
+
+**Implemented enhancements:**
+
+- Notification via Telegram [\#146](https://github.com/pyouroboros/ouroboros/issues/146)
+- Add flag to allow a labels\_only condition [\#142](https://github.com/pyouroboros/ouroboros/issues/142)
+- DRY\_RUN flag [\#140](https://github.com/pyouroboros/ouroboros/issues/140)
+- Notification on startup [\#138](https://github.com/pyouroboros/ouroboros/issues/138)
+- Start/Stop containers in sequence [\#106](https://github.com/pyouroboros/ouroboros/issues/106)
+- Refactor/notifications with apprise [\#151](https://github.com/pyouroboros/ouroboros/pull/151) [[breaking change](https://github.com/pyouroboros/ouroboros/labels/breaking%20change)] [[cleanup](https://github.com/pyouroboros/ouroboros/labels/cleanup)] [[documentation](https://github.com/pyouroboros/ouroboros/labels/documentation)] ([DirtyCajunRice](https://github.com/DirtyCajunRice))
+
+**Fixed bugs:**
+
+- Catch invalid docker socket config [\#148](https://github.com/pyouroboros/ouroboros/issues/148)
+- Explicitly Define true/false [\#141](https://github.com/pyouroboros/ouroboros/issues/141) [[documentation](https://github.com/pyouroboros/ouroboros/labels/documentation)]
+
+**Other Pull Requests**
+
+- v1.1.0 Merge [\#153](https://github.com/pyouroboros/ouroboros/pull/153) ([DirtyCajunRice](https://github.com/DirtyCajunRice))
+- v1.1.0 to develop [\#152](https://github.com/pyouroboros/ouroboros/pull/152) ([DirtyCajunRice](https://github.com/DirtyCajunRice))
+- Patch/group 1 [\#150](https://github.com/pyouroboros/ouroboros/pull/150) ([DirtyCajunRice](https://github.com/DirtyCajunRice))
+- Add volume for docker socket path [\#144](https://github.com/pyouroboros/ouroboros/pull/144) ([mauvehed](https://github.com/mauvehed))
+
+## [1.0.0](https://github.com/pyouroboros/ouroboros/tree/1.0.0) (2019-01-23)
 [Full Changelog](https://github.com/pyouroboros/ouroboros/compare/0.6.0...1.0.0)
 
 **Implemented enhancements:**
@@ -29,8 +53,8 @@
 
 **Other Pull Requests**
 
-- v1.0.0 to develop [\#136](https://github.com/pyouroboros/ouroboros/pull/136) ([DirtyCajunRice](https://github.com/DirtyCajunRice))
 - v1.0.0 Merge [\#137](https://github.com/pyouroboros/ouroboros/pull/137) ([DirtyCajunRice](https://github.com/DirtyCajunRice))
+- v1.0.0 to develop [\#136](https://github.com/pyouroboros/ouroboros/pull/136) ([DirtyCajunRice](https://github.com/DirtyCajunRice))
 - Clean old legacy files [\#134](https://github.com/pyouroboros/ouroboros/pull/134) [[cleanup](https://github.com/pyouroboros/ouroboros/labels/cleanup)] ([DirtyCajunRice](https://github.com/DirtyCajunRice))
 - Cleanup/qemu logic [\#128](https://github.com/pyouroboros/ouroboros/pull/128) [[cleanup](https://github.com/pyouroboros/ouroboros/labels/cleanup)] ([DirtyCajunRice](https://github.com/DirtyCajunRice))
 - fix readme wording for monitoring remote hosts [\#126](https://github.com/pyouroboros/ouroboros/pull/126) [[documentation](https://github.com/pyouroboros/ouroboros/labels/documentation)] ([circa10a](https://github.com/circa10a))

README.md

@@ -18,7 +18,7 @@ A python-based successor to [watchtower](https://github.com/v2tec/watchtower)
 Ouroboros will monitor (all or specified) running docker containers and update them to the (latest or tagged) available image in the remote registry. The updated container uses the same tag and parameters that were used when the container was first created such as volume/bind mounts, docker network connections, environment variables, restart policies, entrypoints, commands, etc.
 
 - Push your image to your registry and simply wait your defined interval for ouroboros to find the new image and redeploy your container autonomously.
-- Notify you via email or platform customized webhooks. (Currently: Discord/Slack/Pushover/HealthChecks/Generic)
+- Notify you via many platforms courtesy of [Apprise](https://github.com/caronc/apprise) 
 - Serve metrics for trend monitoring (Currently: Prometheus/Influxdb)
 - Limit your server ssh access
 - `ssh -i key server.domainname "docker pull ... && docker run ..."` is for scrubs
@@ -55,7 +55,7 @@ pip install ouroboros-cli
 And can then be invoked using the `ouroboros` command:
 
 ```bash
-$ ouroboros --interval 300 --loglevel debug
+$ ouroboros --interval 300 --log-level debug
 ```
 
 > This can be useful if you would like to create a `systemd` service or similar daemon that doesn't run in a container

docker-compose.yml

@@ -12,3 +12,5 @@ services:
       - IGNORE=mongo influxdb postgres mariadb
       - TZ=America/Chicago
     restart: unless-stopped
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock

pyouroboros/__init__.py

@@ -1,2 +1,2 @@
-VERSION = "1.0.0"
+VERSION = "1.1.0"
 BRANCH = "master"

pyouroboros/config.py

@@ -4,11 +4,10 @@
 
 class Config(object):
     options = ['INTERVAL', 'PROMETHEUS', 'DOCKER_SOCKETS', 'MONITOR', 'IGNORE', 'LOG_LEVEL', 'PROMETHEUS_ADDR',
-               'PROMETHEUS_PORT', 'WEBHOOK_URLS', 'REPO_USER', 'REPO_PASS', 'CLEANUP', 'RUN_ONCE', 'LATEST',
+               'PROMETHEUS_PORT', 'NOTIFIERS', 'REPO_USER', 'REPO_PASS', 'CLEANUP', 'RUN_ONCE', 'LATEST',
                'INFLUX_URL', 'INFLUX_PORT', 'INFLUX_USERNAME', 'INFLUX_PASSWORD', 'INFLUX_DATABASE', 'INFLUX_SSL',
-               'INFLUX_VERIFY_SSL', 'DATA_EXPORT', 'PUSHOVER_TOKEN', 'PUSHOVER_USER', 'PUSHOVER_DEVICE', 'SMTP_HOST',
-               'SMTP_PORT', 'SMTP_STARTTLS', 'SMTP_USERNAME', 'SMTP_PASSWORD', 'SMTP_RECIPIENTS', 'SMTP_FROM_EMAIL',
-               'SMTP_FROM_NAME', 'SELF_UPDATE', 'LABEL_ENABLE', 'DOCKER_TLS_VERIFY']
+               'INFLUX_VERIFY_SSL', 'DATA_EXPORT', 'SELF_UPDATE', 'LABEL_ENABLE', 'DOCKER_TLS_VERIFY', 'LABELS_ONLY',
+               'DRY_RUN']
 
     interval = 300
     docker_sockets = 'unix://var/run/docker.sock'
@@ -20,8 +19,10 @@ class Config(object):
     latest = False
     cleanup = False
     run_once = False
+    dry_run = False
     self_update = False
     label_enable = False
+    labels_only = False
 
     repo_user = None
     repo_pass = None
@@ -39,20 +40,7 @@ class Config(object):
     influx_password = 'root'
     influx_database = None
 
-    webhook_urls = []
-
-    pushover_token = None
-    pushover_user = None
-    pushover_device = None
-
-    smtp_host = None
-    smtp_port = 587
-    smtp_starttls = False
-    smtp_username = None
-    smtp_password = None
-    smtp_recipients = None
-    smtp_from_email = None
-    smtp_from_name = 'Ouroboros'
+    notifiers = []
 
     def __init__(self, environment_vars, cli_args):
         self.cli_args = cli_args
@@ -88,18 +76,20 @@ def config_blacklist(self):
     def parse(self):
         for option in Config.options:
             if self.environment_vars.get(option):
-                if option in ['INTERVAL', 'PROMETHEUS_PORT', 'INFLUX_PORT', 'SMTP_PORT']:
+                if option in ['INTERVAL', 'PROMETHEUS_PORT', 'INFLUX_PORT']:
                     try:
                         opt = int(self.environment_vars[option])
                         setattr(self, option.lower(), opt)
                     except ValueError as e:
                         print(e)
-                elif option in ['LATEST', 'CLEANUP', 'RUN_ONCE', 'INFLUX_SSL', 'INFLUX_VERIFY_SSL',
-                                'SMTP_STARTTLS', 'SELF_UPDATE', 'LABEL_ENABLE', 'DOCKER_TLS_VERIFY']:
-                    if self.environment_vars[option].lower() == 'true':
+                elif option in ['LATEST', 'CLEANUP', 'RUN_ONCE', 'INFLUX_SSL', 'INFLUX_VERIFY_SSL', 'DRY_RUN',
+                                'SELF_UPDATE', 'LABEL_ENABLE', 'DOCKER_TLS_VERIFY', 'LABELS_ONLY']:
+                    if self.environment_vars[option].lower() in ['true', 'yes']:
                         setattr(self, option.lower(), True)
+                    elif self.environment_vars[option].lower() in ['false', 'no']:
+                        setattr(self, option.lower(), False)
                     else:
-                        self.logger.error('%s is not a valid option for %s. setting to false',
+                        self.logger.error('%s is not true/yes, nor false/no for %s. Assuming false',
                                           self.environment_vars[option], option)
                 else:
                     setattr(self, option.lower(), self.environment_vars[option])
@@ -113,10 +103,10 @@ def parse(self):
         if self.interval < 30:
             self.interval = 30
 
-        for option in ['docker_sockets', 'webhook_urls', 'smtp_recipients', 'monitor', 'ignore']:
+        for option in ['docker_sockets', 'notifiers', 'monitor', 'ignore']:
             if isinstance(getattr(self, option), str):
                 string_list = getattr(self, option)
-                setattr(self, option, [string.strip(' ') for string in string_list.split(' ')])
+                setattr(self, option, [string.strip(' ').strip('"') for string in string_list.split(' ')])
 
         # Config sanity checks
         if self.data_export == 'influxdb' and not self.influx_database:
@@ -126,17 +116,8 @@ def parse(self):
         if self.data_export == 'prometheus' and self.self_update:
             self.logger.warning("If you bind a port to ouroboros, it will be lost when it updates itself.")
 
-        pushover_config = [self.pushover_token, self.pushover_device, self.pushover_user]
-        if any(pushover_config) and not all(pushover_config):
-            self.logger.error('You must specify a pushover user, token, and device to use pushover. Disabling '
-                              'pushover notifications')
-        elif all(pushover_config):
-            self.webhook_urls.append('https://api.pushover.net/1/messages.json')
-
-        email_config = [self.smtp_host, self.smtp_recipients, self.smtp_from_email]
-        if any(email_config) and not all(email_config):
-            self.logger.error('To use email notifications, you need to specify at least smtp host/recipients/from '
-                              'email. Disabling email notifications')
-            self.smtp_host = None
+        if self.dry_run and not self.run_once:
+            self.logger.warning("Dry run is designed to be ran with run once. Setting for you.")
+            self.run_once = True
 
         self.config_blacklist()

pyouroboros/dockerclient.py

@@ -52,9 +52,9 @@ def monitor_filter(self):
                     monitored_containers.append(container)
                 else:
                     continue
-            elif self.config.monitor and container.name not in self.config.ignore:
+            elif not self.config.labels_only and self.config.monitor and container.name not in self.config.ignore:
                     monitored_containers.append(container)
-            elif container.name not in self.config.ignore:
+            elif not self.config.labels_only and container.name not in self.config.ignore:
                     monitored_containers.append(container)
 
         self.data_manager.monitored_containers[self.socket] = len(monitored_containers)
@@ -80,22 +80,29 @@ def pull(self, image_object):
                     tag = ':'.join(split_tag[:-1])
             tag = f'{tag}:latest'
 
-        self.logger.debug('Pulling tag: %s', tag)
+        self.logger.debug('Checking tag: %s', tag)
         try:
-            if self.config.auth_json:
-                return_image = self.client.images.pull(tag, auth_config=self.config.auth_json)
+            if self.config.dry_run:
+                registry_data = self.client.images.get_registry_data(tag)
+                return registry_data
             else:
-                return_image = self.client.images.pull(tag)
-            return return_image
-
+                if self.config.auth_json:
+                    return_image = self.client.images.pull(tag, auth_config=self.config.auth_json)
+                else:
+                    return_image = self.client.images.pull(tag)
+                return return_image
         except APIError as e:
-            self.logger.critical(e)
             if '<html>' in str(e):
                 self.logger.debug("Docker api issue. Ignoring")
                 raise ConnectionError
             elif 'unauthorized' in str(e):
-                self.logger.critical("Invalid Credentials. Exiting")
-                exit(1)
+                if self.config.dry_run:
+                    self.logger.error('dry run : Upstream authentication issue while checking %s. See: '
+                                      'https://github.com/docker/docker-py/issues/2225', tag)
+                    raise ConnectionError
+                else:
+                    self.logger.critical("Invalid Credentials. Exiting")
+                    exit(1)
             elif 'Client.Timeout' in str(e):
                 self.logger.critical("Couldn't find an image on docker.com for %s. Local Build?", image.tags[0])
                 raise ConnectionError
@@ -106,7 +113,7 @@ def pull(self, image_object):
     def update_containers(self):
         updated_count = 0
         updated_container_tuples = []
-
+        depends_on_list = []
         self.monitored = self.monitor_filter()
 
         if not self.monitored:
@@ -128,6 +135,13 @@ def update_containers(self):
                 except ConnectionError:
                     continue
 
+            if self.config.dry_run:
+                # Ugly hack for repo digest
+                repo_digest_id = current_image.attrs['RepoDigests'][0].split('@')[1]
+                if repo_digest_id != latest_image.id:
+                    self.logger.info('dry run : %s would be updated', container.name)
+                continue
+
             # If current running container is running latest image
             if current_image.id != latest_image.id:
                 if container.name in ['ouroboros', 'ouroboros-updated']:
@@ -138,6 +152,10 @@ def update_containers(self):
                 )
                 self.logger.info('%s will be updated', container.name)
 
+                # Get container list to restart after update complete
+                depends_on = container.labels.get('com.ouroboros.depends-on', False)
+                if depends_on:
+                    depends_on_list.extend([name.strip() for name in depends_on.split(',')])
                 # new container dict to create new container from
                 new_config = set_properties(old=container, new=latest_image)
 
@@ -176,11 +194,21 @@ def update_containers(self):
                 self.data_manager.add(label=container.name, socket=self.socket)
                 self.data_manager.add(label='all', socket=self.socket)
 
-        if updated_count > 0:
-            self.notification_manager.send(container_tuples=updated_container_tuples, socket=self.socket,
-                                           notification_type='data')
+        if depends_on_list:
+            depends_on_containers = []
+            for name in list(set(depends_on_list)):
+                try:
+                    depends_on_containers.append(self.client.containers.get(name))
+                except NotFound:
+                    self.logger.error("Could not find dependant container %s on socket %s. Ignoring", name, self.socket)
 
-        self.notification_manager.send(notification_type='keep_alive')
+            if depends_on_containers:
+                for container in depends_on_containers:
+                    self.logger.debug('Restarting dependant container %s', container.name)
+                    container.restart()
+
+        if updated_count > 0:
+            self.notification_manager.send(container_tuples=updated_container_tuples, socket=self.socket, kind='update')
 
     def update_self(self, count=None, old_container=None, me_list=None, new_image=None):
         if count == 2:

pyouroboros/helpers.py

@@ -16,9 +16,10 @@ def set_properties(old, new, self_name=None):
     return properties
 
 
-EMAIL_TEMPLATE = Template(
+NotificationTemplate = Template(
     'Host Socket: ${HOST_SOCKET}\n'
     'Containers Monitored: ${CONTAINERS_MONITORED}\n'
     'Containers Updated: ${CONTAINERS_UPDATED}\n'
+    'Containers Updated This Pass: {CONTAINERS_THIS_PASS}'
     '${CONTAINER_UPDATES}'
 )