Skip to content

Commit c10a48b

Browse files
v1.3.0 Merge
v1.3.0 Merge
2 parents 24ecb87 + 3599f05 commit c10a48b

File tree

11 files changed

+152
-121
lines changed

11 files changed

+152
-121
lines changed

CHANGELOG.md

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,35 @@
11
# Change Log
22

3-
## [1.2.1](https://github.com/pyouroboros/ouroboros/tree/1.2.1) (2019-02-13)
3+
## [1.3.0](https://github.com/pyouroboros/ouroboros/tree/1.3.0) (2019-02-25)
4+
[Full Changelog](https://github.com/pyouroboros/ouroboros/compare/1.2.1...1.3.0)
5+
6+
**Implemented enhancements:**
7+
8+
- Start new container in detached mode [\#222](https://github.com/pyouroboros/ouroboros/pull/222) ([nightvisi0n](https://github.com/nightvisi0n))
9+
- Optimise dockerfile layers [\#218](https://github.com/pyouroboros/ouroboros/pull/218) ([nightvisi0n](https://github.com/nightvisi0n))
10+
11+
**Fixed bugs:**
12+
13+
- Catch Failed self-updates [\#230](https://github.com/pyouroboros/ouroboros/issues/230)
14+
- Cron scheduled missed following successful runs [\#229](https://github.com/pyouroboros/ouroboros/issues/229)
15+
- Catch attribute.id error [\#226](https://github.com/pyouroboros/ouroboros/issues/226)
16+
- AttachStdout and AttachStderr are not carried over properly [\#221](https://github.com/pyouroboros/ouroboros/issues/221)
17+
- Exception when updating container started with --rm \(autoremove\) [\#219](https://github.com/pyouroboros/ouroboros/issues/219)
18+
- Issue with Swarm Mode V2 [\#216](https://github.com/pyouroboros/ouroboros/issues/216)
19+
- Fix docker swarm mode [\#227](https://github.com/pyouroboros/ouroboros/pull/227) ([mathcantin](https://github.com/mathcantin))
20+
21+
**Other Pull Requests**
22+
23+
- v1.3.0 Merge [\#241](https://github.com/pyouroboros/ouroboros/pull/241) ([DirtyCajunRice](https://github.com/DirtyCajunRice))
24+
- v1.3.0 to develop [\#240](https://github.com/pyouroboros/ouroboros/pull/240) ([DirtyCajunRice](https://github.com/DirtyCajunRice))
25+
- Catch self update apierror [\#238](https://github.com/pyouroboros/ouroboros/pull/238) ([circa10a](https://github.com/circa10a))
26+
- Catch attribute error [\#237](https://github.com/pyouroboros/ouroboros/pull/237) ([circa10a](https://github.com/circa10a))
27+
- Check for autoremove [\#236](https://github.com/pyouroboros/ouroboros/pull/236) ([circa10a](https://github.com/circa10a))
28+
- Add misfire\_grace\_time for cron scheduler [\#234](https://github.com/pyouroboros/ouroboros/pull/234) ([circa10a](https://github.com/circa10a))
29+
- Check all services by default on swarm mode [\#228](https://github.com/pyouroboros/ouroboros/pull/228) [[cleanup](https://github.com/pyouroboros/ouroboros/labels/cleanup)] ([mathcantin](https://github.com/mathcantin))
30+
- remove git in pypi + branch develop + version bump + maintainer\_email [\#214](https://github.com/pyouroboros/ouroboros/pull/214) ([DirtyCajunRice](https://github.com/DirtyCajunRice))
31+
32+
## [1.2.1](https://github.com/pyouroboros/ouroboros/tree/1.2.1) (2019-02-14)
433
[Full Changelog](https://github.com/pyouroboros/ouroboros/compare/1.2.0...1.2.1)
534

635
**Fixed bugs:**

Dockerfile

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,18 @@
11
FROM amd64/python:3.7.2-alpine
22

3-
LABEL maintainers="dirtycajunrice,circa10a,tkdeviant"
3+
LABEL maintainers="dirtycajunrice,circa10a"
44

55
ENV TZ UTC
66

77
WORKDIR /app
88

99
COPY /requirements.txt /setup.py /ouroboros /README.md /app/
1010

11+
RUN apk add --no-cache tzdata && \
12+
pip install --no-cache-dir -r requirements.txt
13+
1114
COPY /pyouroboros /app/pyouroboros
1215

13-
RUN apk add --no-cache tzdata && \
14-
pip install --no-cache-dir .
16+
RUN pip install --no-cache-dir .
1517

1618
ENTRYPOINT ["ouroboros"]

Dockerfile.arm

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,18 @@
11
FROM arm32v6/python:3.7.2-alpine
22

3-
LABEL maintainers="dirtycajunrice,circa10a,tkdeviant"
3+
LABEL maintainers="dirtycajunrice,circa10a"
44

55
ENV TZ UTC
66

77
WORKDIR /app
88

99
COPY /requirements.txt /setup.py /ouroboros /README.md /app/
1010

11+
RUN apk add --no-cache tzdata && \
12+
pip install --no-cache-dir -r requirements.txt
13+
1114
COPY /pyouroboros /app/pyouroboros
1215

13-
RUN apk add --no-cache tzdata && \
14-
pip install --no-cache-dir .
16+
RUN pip install --no-cache-dir .
1517

1618
ENTRYPOINT ["ouroboros"]

Dockerfile.arm64

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,18 @@
11
FROM arm64v8/python:3.7.2-alpine
22

3-
LABEL maintainers="dirtycajunrice,circa10a,tkdeviant"
3+
LABEL maintainers="dirtycajunrice,circa10a"
44

55
ENV TZ UTC
66

77
WORKDIR /app
88

99
COPY /requirements.txt /setup.py /ouroboros /README.md /app/
1010

11+
RUN apk add --no-cache tzdata && \
12+
pip install --no-cache-dir -r requirements.txt
13+
1114
COPY /pyouroboros /app/pyouroboros
1215

13-
RUN apk add --no-cache tzdata && \
14-
pip install --no-cache-dir .
16+
RUN pip install --no-cache-dir .
1517

1618
ENTRYPOINT ["ouroboros"]

Jenkinsfile

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -127,8 +127,6 @@ pipeline {
127127
sh """
128128
python3 -m venv venv && venv/bin/pip install twine
129129
venv/bin/python setup.py sdist && venv/bin/python -m twine upload --skip-existing -u ${PYPI_CREDS_USR} -p ${PYPI_CREDS_PSW} dist/*
130-
git tag ${TAG}
131-
git push --tags
132130
"""
133131
}
134132
}

README.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,6 @@
44
[![BuyUsCoffee](https://img.shields.io/badge/BuyMeACoffee-Donate-ff813f.svg?logo=CoffeeScript&style=flat-square)](https://buymeacoff.ee/ouroboros)
55
[![Build Status](https://jenkins.cajun.pro/buildStatus/icon?job=Ouroboros/master)](https://jenkins.cajun.pro/job/Ouroboros/job/master/)
66
[![Release](https://img.shields.io/github/release/pyouroboros/ouroboros.svg?style=flat-square)](https://hub.docker.com/r/pyouroboros/ouroboros/)
7-
[![Pypi Downloads](https://img.shields.io/pypi/dm/ouroboros-cli.svg?style=flat-square)](https://pypi.org/project/ouroboros-cli/)
87
[![Python Version](https://img.shields.io/pypi/pyversions/ouroboros-cli.svg?style=flat-square)](https://pypi.org/project/ouroboros-cli/)
98
[![Docker Pulls](https://img.shields.io/docker/pulls/pyouroboros/ouroboros.svg?style=flat-square)](https://hub.docker.com/r/pyouroboros/ouroboros/)
109
[![Layers](https://images.microbadger.com/badges/image/pyouroboros/ouroboros.svg)](https://microbadger.com/images/pyouroboros/ouroboros)

pyouroboros/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
VERSION = "1.2.1"
1+
VERSION = "1.3.0"
22
BRANCH = "master"

pyouroboros/dockerclient.py

Lines changed: 77 additions & 95 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
from os.path import isdir, isfile, join
55
from docker.errors import DockerException, APIError, NotFound
66

7-
from pyouroboros.helpers import set_properties
7+
from pyouroboros.helpers import set_properties, remove_sha_prefix, get_digest
88

99

1010
class Docker(object):
@@ -55,7 +55,7 @@ def connect(self):
5555
return client
5656

5757

58-
class Container(object):
58+
class BaseImageObject(object):
5959
def __init__(self, docker_client):
6060
self.docker = docker_client
6161
self.logger = self.docker.logger
@@ -66,6 +66,44 @@ def __init__(self, docker_client):
6666
self.data_manager.total_updated[self.socket] = 0
6767
self.notification_manager = self.docker.notification_manager
6868

69+
def _pull(self, tag):
70+
"""Docker pull image tag"""
71+
self.logger.debug('Checking tag: %s', tag)
72+
try:
73+
if self.config.auth_json:
74+
self.client.login(self.config.auth_json.get(
75+
"username"), self.config.auth_json.get("password"))
76+
77+
if self.config.dry_run:
78+
# The authentication doesn't work with this call
79+
# See bugs https://github.com/docker/docker-py/issues/2225
80+
return self.client.images.get_registry_data(tag)
81+
else:
82+
return self.client.images.pull(tag)
83+
except APIError as e:
84+
if '<html>' in str(e):
85+
self.logger.debug("Docker api issue. Ignoring")
86+
raise ConnectionError
87+
elif 'unauthorized' in str(e):
88+
if self.config.dry_run:
89+
self.logger.error('dry run : Upstream authentication issue while checking %s. See: '
90+
'https://github.com/docker/docker-py/issues/2225', tag)
91+
raise ConnectionError
92+
else:
93+
self.logger.critical("Invalid Credentials. Exiting")
94+
exit(1)
95+
elif 'Client.Timeout' in str(e):
96+
self.logger.critical(
97+
"Couldn't find an image on docker.com for %s. Local Build?", tag)
98+
raise ConnectionError
99+
elif ('pull access' or 'TLS handshake') in str(e):
100+
self.logger.critical("Couldn't pull. Skipping. Error: %s", e)
101+
raise ConnectionError
102+
103+
104+
class Container(BaseImageObject):
105+
def __init__(self, docker_client):
106+
super().__init__(docker_client)
69107
self.monitored = self.monitor_filter()
70108

71109
# Container sub functions
@@ -135,35 +173,7 @@ def pull(self, current_tag):
135173
raise ConnectionError
136174
elif ':' not in tag:
137175
tag = f'{tag}:latest'
138-
self.logger.debug('Checking tag: %s', tag)
139-
try:
140-
if self.config.dry_run:
141-
registry_data = self.client.images.get_registry_data(tag)
142-
return registry_data
143-
else:
144-
if self.config.auth_json:
145-
return_image = self.client.images.pull(tag, auth_config=self.config.auth_json)
146-
else:
147-
return_image = self.client.images.pull(tag)
148-
return return_image
149-
except APIError as e:
150-
if '<html>' in str(e):
151-
self.logger.debug("Docker api issue. Ignoring")
152-
raise ConnectionError
153-
elif 'unauthorized' in str(e):
154-
if self.config.dry_run:
155-
self.logger.error('dry run : Upstream authentication issue while checking %s. See: '
156-
'https://github.com/docker/docker-py/issues/2225', tag)
157-
raise ConnectionError
158-
else:
159-
self.logger.critical("Invalid Credentials. Exiting")
160-
exit(1)
161-
elif 'Client.Timeout' in str(e):
162-
self.logger.critical("Couldn't find an image on docker.com for %s. Local Build?", tag)
163-
raise ConnectionError
164-
elif ('pull access' or 'TLS handshake') in str(e):
165-
self.logger.critical("Couldn't pull. Skipping. Error: %s", e)
166-
raise ConnectionError
176+
return self._pull(tag)
167177

168178
# Filters
169179
def running_filter(self):
@@ -176,7 +186,10 @@ def running_filter(self):
176186
else:
177187
try:
178188
if 'ouroboros' not in container.image.tags[0]:
179-
running_containers.append(container)
189+
if container.attrs['HostConfig']['AutoRemove']:
190+
self.logger.debug("Skipping %s due to --rm property.", container.name)
191+
else:
192+
running_containers.append(container)
180193
except IndexError:
181194
self.logger.error("%s has no tags.. you should clean it up! Ignoring.", container.id)
182195
continue
@@ -238,10 +251,13 @@ def socket_check(self):
238251
latest_image = self.pull(current_tag)
239252
except ConnectionError:
240253
continue
241-
if current_image.id != latest_image.id:
242-
updateable.append((container, current_image, latest_image))
243-
else:
244-
continue
254+
try:
255+
if current_image.id != latest_image.id:
256+
updateable.append((container, current_image, latest_image))
257+
else:
258+
continue
259+
except AttributeError:
260+
self.logger.error("Issue detecting %s's image tag. Skipping...", container.name)
245261

246262
# Get container list to restart after update complete
247263
depends_on = container.labels.get('com.ouroboros.depends_on', False)
@@ -342,25 +358,22 @@ def update_self(self, count=None, old_container=None, me_list=None, new_image=No
342358
self.logger.debug('I need to update! Starting the ouroboros ;)')
343359
self_name = 'ouroboros-updated' if old_container.name == 'ouroboros' else 'ouroboros'
344360
new_config = set_properties(old=old_container, new=new_image, self_name=self_name)
345-
me_created = self.client.api.create_container(**new_config)
346-
new_me = self.client.containers.get(me_created.get("Id"))
347-
new_me.start()
348-
self.logger.debug('If you strike me down, I shall become more powerful than you could possibly imagine')
349-
self.logger.debug('https://bit.ly/2VVY7GH')
350-
sleep(30)
361+
try:
362+
me_created = self.client.api.create_container(**new_config)
363+
new_me = self.client.containers.get(me_created.get("Id"))
364+
new_me.start()
365+
self.logger.debug('If you strike me down, I shall become \
366+
more powerful than you could possibly imagine.')
367+
self.logger.debug('https://bit.ly/2VVY7GH')
368+
sleep(30)
369+
except APIError as e:
370+
self.logger.error("Self update failed.")
371+
self.logger.error(e)
351372

352373

353-
class Service(object):
374+
class Service(BaseImageObject):
354375
def __init__(self, docker_client):
355-
self.docker = docker_client
356-
self.logger = self.docker.logger
357-
self.config = self.docker.config
358-
self.client = self.docker.client
359-
self.socket = self.docker.socket
360-
self.data_manager = self.docker.data_manager
361-
self.data_manager.total_updated[self.socket] = 0
362-
self.notification_manager = self.docker.notification_manager
363-
376+
super().__init__(docker_client)
364377
self.monitored = self.monitor_filter()
365378

366379
def monitor_filter(self):
@@ -371,7 +384,7 @@ def monitor_filter(self):
371384

372385
for service in services:
373386
ouro_label = service.attrs['Spec']['Labels'].get('com.ouroboros.enable')
374-
if ouro_label.lower() in ["true", "yes"]:
387+
if not self.config.label_enable or ouro_label.lower() in ["true", "yes"]:
375388
monitored_services.append(service)
376389

377390
self.data_manager.monitored_containers[self.socket] = len(monitored_services)
@@ -381,38 +394,9 @@ def monitor_filter(self):
381394

382395
def pull(self, tag):
383396
"""Docker pull image tag"""
384-
self.logger.debug('Checking tag: %s', tag)
385-
try:
386-
if self.config.dry_run:
387-
registry_data = self.client.images.get_registry_data(tag)
388-
return registry_data
389-
else:
390-
if self.config.auth_json:
391-
return_image = self.client.images.pull(tag, auth_config=self.config.auth_json)
392-
else:
393-
return_image = self.client.images.pull(tag)
394-
return return_image
395-
except APIError as e:
396-
if '<html>' in str(e):
397-
self.logger.debug("Docker api issue. Ignoring")
398-
raise ConnectionError
399-
elif 'unauthorized' in str(e):
400-
if self.config.dry_run:
401-
self.logger.error('dry run : Upstream authentication issue while checking %s. See: '
402-
'https://github.com/docker/docker-py/issues/2225', tag)
403-
raise ConnectionError
404-
else:
405-
self.logger.critical("Invalid Credentials. Exiting")
406-
exit(1)
407-
elif 'Client.Timeout' in str(e):
408-
self.logger.critical("Couldn't find an image on docker.com for %s. Local Build?", tag)
409-
raise ConnectionError
410-
elif ('pull access' or 'TLS handshake') in str(e):
411-
self.logger.critical("Couldn't pull. Skipping. Error: %s", e)
412-
raise ConnectionError
397+
return self._pull(tag)
413398

414399
def update(self):
415-
updated_count = 0
416400
updated_service_tuples = []
417401
self.monitored = self.monitor_filter()
418402

@@ -423,7 +407,7 @@ def update(self):
423407
image_string = service.attrs['Spec']['TaskTemplate']['ContainerSpec']['Image']
424408
if '@' in image_string:
425409
tag = image_string.split('@')[0]
426-
sha256 = image_string.split('@')[1][7:]
410+
sha256 = remove_sha_prefix(image_string.split('@')[1])
427411
else:
428412
self.logger.error('No image SHA for %s. Skipping', image_string)
429413
continue
@@ -433,13 +417,15 @@ def update(self):
433417
except ConnectionError:
434418
continue
435419

436-
if self.config.dry_run:
437-
# Ugly hack for repo digest
438-
if sha256 != latest_image.id:
420+
latest_image_sha256 = get_digest(latest_image)
421+
self.logger.debug('Latest sha256 for %s is %s', tag, latest_image_sha256)
422+
423+
if sha256 != latest_image_sha256:
424+
if self.config.dry_run:
425+
# Ugly hack for repo digest
439426
self.logger.info('dry run : %s would be updated', service.name)
440-
continue
427+
continue
441428

442-
if sha256 != latest_image.id:
443429
updated_service_tuples.append(
444430
(service, sha256[-10:], latest_image)
445431
)
@@ -452,17 +438,13 @@ def update(self):
452438
socket=self.socket, kind='update', mode='service')
453439

454440
self.logger.info('%s will be updated', service.name)
455-
service.update(image=tag)
456-
457-
updated_count += 1
458-
459-
self.logger.debug("Incrementing total service updated count")
441+
service.update(image=f"{tag}@sha256:{latest_image_sha256}")
460442

461443
self.data_manager.total_updated[self.socket] += 1
462444
self.data_manager.add(label=service.name, socket=self.socket)
463445
self.data_manager.add(label='all', socket=self.socket)
464446

465-
if updated_count > 0:
447+
if updated_service_tuples:
466448
self.notification_manager.send(
467449
container_tuples=updated_service_tuples,
468450
socket=self.socket,

0 commit comments

Comments
 (0)