Skip to content

Commit 9cfa55f

Browse files
committed
[feature] Upload imgages to openwisp-firmware-upgrader API
1 parent 701afd4 commit 9cfa55f

5 files changed

Lines changed: 221 additions & 1 deletion

File tree

README.md

Lines changed: 80 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -263,6 +263,85 @@ and 2 flavours (eg: standard and mini), you will get 8 groups of images:
263263
The images will be created in the directory specified in
264264
`openwisp2fw_bin_dir`.
265265

266+
### 5. Upload images to OpenWISP Firmware Upgrader
267+
268+
The last step is to upload images to the
269+
[OpenWISP Firmware Upgrader module](https://github.com/openwisp/openwisp-firmware-upgrader).
270+
This step is optional and disabled by default.
271+
272+
To enable this feature, the variables ``openwisp2fw_uploader``
273+
and ``openwisp2fw_organizations.categories`` need to be configured
274+
as in the example below:
275+
276+
```yaml
277+
- hosts:
278+
- myhost
279+
roles:
280+
- openwisp.openwisp2-imagegenerator
281+
vars:
282+
openwisp2fw_controller_url: "https://openwisp.myproject.com"
283+
openwisp2fw_organizations:
284+
- name: staging
285+
flavours:
286+
- default
287+
openwisp:
288+
url: "{{ openwisp2fw_controller_url }}"
289+
shared_secret: "xxxxx"
290+
root_password: "xxxxx"
291+
categories:
292+
default: <CATEGORY-UUID>
293+
- name: prod
294+
flavours:
295+
- default
296+
openwisp:
297+
url: "{{ openwisp2fw_controller_url }}"
298+
shared_secret: "xxxxx"
299+
root_password: "xxxxx"
300+
categories:
301+
default: <CATEGORY-UUID>
302+
openwisp2fw_uploader:
303+
enabled: true
304+
url: "{{ openwisp2fw_controller_url }}"
305+
token: "<REST-API-USER-TOKEN>"
306+
image_types:
307+
- ath79-generic-ubnt_airrouter-squashfs-sysupgrade.bin
308+
- ar71xx-generic-ubnt-bullet-m-xw-squashfs-sysupgrade.bin
309+
- ar71xx-generic-ubnt-bullet-m-squashfs-sysupgrade.bin
310+
- octeon-erlite-squashfs-sysupgrade.tar
311+
- ath79-generic-ubnt_nanostation-loco-m-xw-squashfs-sysupgrade.bin
312+
- ath79-generic-ubnt_nanostation-loco-m-squashfs-sysupgrade.bin
313+
- ath79-generic-ubnt_nanostation-m-xw-squashfs-sysupgrade.bin
314+
- ar71xx-generic-ubnt-nano-m-squashfs-sysupgrade.bin
315+
- ath79-generic-ubnt_unifiac-mesh-squashfs-sysupgrade.bin
316+
- x86-64-combined-squashfs.img.gz
317+
- x86-generic-combined-squashfs.img.gz
318+
- x86-geode-combined-squashfs.img.gz
319+
- ar71xx-generic-xd3200-squashfs-sysupgrade.bin
320+
```
321+
322+
The following placeholders in the example will have to be substituted:
323+
324+
- `<CATEGORY-UUID>` is the UUID o the firmware category in OpenWISP Firmware Upgrader
325+
- `<REST-API-USER-TOKEN>` is the REST auth token of a user with permissions to upload images
326+
327+
You can retrieve the REST auth token by sending a POST request using the Browsable API web interface of OpenWISP:
328+
329+
1. Open the browser at `https://<openwisp-base-url>/api/v1/user/token/`.
330+
2. Enter username and password in the form at the bottom of the page.
331+
3. Submit the form and you will get the REST auth token in the response.
332+
333+
The upload script creates a new build object and then uploads the firmware images
334+
specified in `image_types`, which have to correspond to the identifiers like
335+
`ar71xx-generic-tl-wdr4300-v1-il-squashfs-sysupgrade.bin` defined in the
336+
[hardware.py file of OpenWISP Firmware Upgrader](https://github.com/openwisp/openwisp-firmware-upgrader/blob/master/openwisp_firmware_upgrader/hardware.py).
337+
338+
Other important points to know about the `upload_firmware.py` script:
339+
340+
- The script reads `CONFIG_VERSION_DIST` and `CONFIG_VERSION_NUMBER`
341+
from the `.config` file of the OpenWrt source code to determine the build
342+
version.
343+
- The script will fail if a build with the same version and category already exists (it does not attempt to upload images to an existing build).
344+
266345
Adding files to images
267346
======================
268347

@@ -357,7 +436,7 @@ order to compile targets that do not specify a subtarget
357436
openwisp2fw_source_targets:
358437
# Allwinner SOC, Lamobo R1
359438
- system: sunxi
360-
profile: DEVICE_sun7i-a20-lamobo-r1
439+
profile: sun7i-a20-lamobo-r1
361440
# QEMU ARM Virtual Image
362441
- system: armvirt
363442
profile: Default

defaults/main.yml

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -125,3 +125,23 @@ openwisp2fw_default_unmanaged:
125125
- network.loopback
126126
- network.@switch
127127
- network.@switch_vlan
128+
openwisp2fw_uploader:
129+
# whether to upload the results to OpenWISP Firmware Upgrader
130+
enabled: false
131+
# OpenWISP server URL
132+
url: ''
133+
# API token of an user who is authorized to upload images
134+
token: ''
135+
# maps categories of fw-upgrader to builds generated by this tool
136+
category_map:
137+
- organization_name: '' # as in `openwisp2fw_organizations.name`
138+
- category_id: '' # the UUID of the firmware category of openwisp fw-upgrader
139+
- flavour: '' # as in `openwisp2fw_organizations. flavours`
140+
# image types to upload
141+
# eg:
142+
# x86-64-combined-squashfs.img.gz
143+
# x86-generic-combined-squashfs.img.gz
144+
# x86-geode-combined-squashfs.img.gz
145+
# ar71xx-generic-xd3200-squashfs-sysupgrade.bin
146+
# ath79-generic-ubnt_unifiac-mesh-squashfs-sysupgrade.bin
147+
image_types: []

tasks/5-upload.yml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
- name: Prepare upload script
2+
template:
3+
src: upload/upload_firmware.py.j2
4+
dest: "{{ openwisp2fw_source_dir }}/upload_firmware.py"
5+
mode: 0744
6+
7+
- name: ./upload_firmware.py
8+
command: ./upload_firmware.py
9+
args:
10+
chdir: "{{ openwisp2fw_source_dir }}"

tasks/main.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,3 +24,9 @@
2424
# store binaries in versioned directories
2525
- include: 4-build.yml
2626
tags: [openwisp2fw, build]
27+
28+
# upload firmware images
29+
# to openwisp-firmware-upgrader
30+
- include: 5-upload.yml
31+
tags: [openwisp2fw, upload]
32+
when: openwisp2fw_uploader.enabled
Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
#!/usr/bin/env python3
2+
3+
import requests
4+
import subprocess
5+
import json
6+
import sys
7+
8+
# insiel
9+
TOKEN = '{{ openwisp2fw_uploader.token }}'
10+
BASE_URL = '{{ openwisp2fw_uploader.url }}'
11+
# key: filesystem directory
12+
# value[0]: category uuid
13+
# value[1]: flavour
14+
CATEGORIES = {
15+
{% for organization in openwisp2fw_organizations -%}
16+
{% for flavour, category_id in organization.get('categories', {}).items() -%}
17+
'{{ organization.name }}': ('{{ category_id }}', '{{ flavour }}'),
18+
{% endfor %}
19+
{% endfor %}
20+
}
21+
BASE_PATH = '{{ openwisp2fw_bin_dir }}'
22+
23+
IMAGE_TYPES = {{ openwisp2fw_uploader.image_types | to_json }}
24+
25+
26+
def jsonify(data):
27+
return json.dumps(data)
28+
29+
30+
def find_config_line(config_name):
31+
with open('.config') as f:
32+
config_contents = f.read()
33+
lines = config_contents.split('\n')
34+
for line in lines:
35+
if line.startswith(config_name):
36+
parts = line.split('=')
37+
return parts[1].replace('"', '')
38+
else:
39+
raise ValueError(f'{config_name} not found')
40+
41+
42+
API_URL = f'{BASE_URL}/api/v1/firmware/'
43+
BUILD_URL = f'{API_URL}build/'
44+
AUTHORIZATION_HEADER = {'Authorization': f'Bearer {TOKEN}'}
45+
CONTENT_TYPE_HEADER = {'Content-Type': 'application/json'}
46+
HEADERS = AUTHORIZATION_HEADER.copy()
47+
HEADERS.update(CONTENT_TYPE_HEADER)
48+
VERSION_DIST = find_config_line('CONFIG_VERSION_DIST')
49+
VERSION_NUMBER = find_config_line('CONFIG_VERSION_NUMBER')
50+
51+
out = subprocess.Popen(
52+
['./scripts/getver.sh'],
53+
stdout=subprocess.PIPE,
54+
stderr=subprocess.STDOUT
55+
)
56+
stdout, stderr = out.communicate()
57+
58+
REVISION = str(stdout.decode('utf8').strip())
59+
OS_IDENTIFIER = f'{VERSION_DIST} {VERSION_NUMBER} {REVISION}'
60+
IMAGE_PREFIX = OS_IDENTIFIER.lower().replace(' ', '-')
61+
62+
for org_slug, org_data in CATEGORIES.items():
63+
category_id = org_data[0]
64+
flavour = org_data[1]
65+
66+
# create build
67+
response = requests.post(
68+
BUILD_URL,
69+
headers=HEADERS,
70+
data=jsonify({
71+
'version': VERSION_NUMBER,
72+
'category': category_id,
73+
'os': OS_IDENTIFIER
74+
}),
75+
)
76+
77+
if response.status_code != 201:
78+
print('It was not possible to create a build.')
79+
print(response.content.decode())
80+
sys.exit(1)
81+
82+
BUILD_ID = response.json()['id']
83+
UPLOAD_IMAGE_URL = f'{BUILD_URL}{BUILD_ID}/image/'
84+
85+
for image_file in IMAGE_TYPES:
86+
target = image_file.split('-')[0]
87+
BIN_PATH = f'{BASE_PATH}/{org_slug}/latest/{target}/{flavour}/{IMAGE_PREFIX}-{image_file}'
88+
89+
try:
90+
BIN_FILE = open(BIN_PATH, 'rb')
91+
except FileNotFoundError as e:
92+
print(f'Skipping {image_file} because of the following error:\n{e}')
93+
continue
94+
95+
# upload image
96+
response = requests.post(
97+
UPLOAD_IMAGE_URL,
98+
headers=AUTHORIZATION_HEADER,
99+
data={
100+
'type': image_file,
101+
'build': BUILD_ID
102+
},
103+
files={'file': BIN_FILE},
104+
)
105+
assert response.status_code == 201

0 commit comments

Comments
 (0)