Skip to content

Commit acc4695

Browse files
authored
Use asymetric ota process to gain flash memory space (#239)
* Establish flash via sd card as intermediate memory * Introduce Flash App * Download new versions directly from GitHub * Document update process * Reduce log level for frequent GPS log messages.
1 parent 4fed75b commit acc4695

File tree

8 files changed

+647
-29
lines changed

8 files changed

+647
-29
lines changed

README.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,64 @@ the resource limitations.
4646

4747
## Updating
4848

49+
### Step By Step
50+
51+
1. Go to "Update Flash App" and push "Update to vX.Y.ZZZ". Wait
52+
a moment, the data is directly downloaded for GitHub to the OBS.
53+
The latest version of the "Flash App" gets then installed, and you are
54+
redirected to the "Update firmware" page - which looks similar.
55+
This is a one time action, can be omitted is already done and
56+
there is no new version. you can start with 2. then.
57+
58+
1. Go to "Update Firmware" and push "Update to vX.Y.ZZZ". The
59+
version is downloaded directly to your OBS device from GitHub.
60+
After the Download the device reboots and does some housekeeping
61+
so be patient within one or two minutes the device boots up
62+
with the new version.
63+
64+
### Details
65+
66+
The update mechanism consists of 2 parts. A small firmware called Flash App
67+
is used to read the new firmware from the SD card and flash it to the obs.
68+
69+
So as a 1st step you need to make sure that the Flash App part is installed
70+
on you ESP, you can do so by using "Update Flash App" button and start the
71+
update b pressing "Update to vX.Y.ZZZ". The data is directly downloaded from
72+
GitHub, so the ESP needs to have access to the internet via your WiFi. You
73+
can also download the `flash.bin` directly from the
74+
[GitHub releases](https://github.com/openbikesensor/OpenBikeSensorFlash/releases)
75+
page. The direct update from GitHub has currently no progress bar so be
76+
patient.
77+
78+
Now you need to download the OBS Firmware and place it as `/sdflash/app.bin`
79+
on the SD Card. This is also fully automated behind the "Update Firmware"
80+
button, There you can again directly download the latest version and start
81+
the internal update process with the "Update to vX.Y.ZZZ" button. Make sure
82+
it is a version newer v0.6.x. At the time of writing there is no such version
83+
released yet. Do not downgrade to a version prior v0.6.x, once the Flash App
84+
was installed. A hint on the "Update Firmware" page will show the version of
85+
the Flash App you have installed or if the Flash App is not installed hint
86+
you to install the Flash App 1st. With the "File Upload" option you can
87+
also update to locally built versions. Make sure they are pos v0.6 versions!
88+
The 1st update includes a repartitioning of the ESPs flash where you
89+
do not see any indication of progress. Be patient all does not take more
90+
than one or two minutes after the upload was completed.
91+
92+
The About page gives you some insight of the current partitioning a more
93+
detailed documentation will come.
94+
95+
96+
97+
### from v0.5.x or earlier
98+
99+
With version v0.6 a new update path was introduced. To allow updating,
100+
to versions of v0.6 or newer you at least hav to have version v0.6 installed.
101+
102+
The update process is as usual, download the latest v0.6.x from GitHub
103+
and use the "Update Firmware" functionality in server mode.
104+
105+
### from v0.2.x or earlier
106+
49107
update functionality. For the 1st flashing of the firmware or the update
50108
from a version prior v0.3.x you have to plug in USB.
51109
See [flash documentation](https://github.com/openbikesensor/OpenBikeSensorFirmware/blob/master/docs/software/firmware/initial_flash.md)

src/Firmware.cpp

Lines changed: 225 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,225 @@
1+
/*
2+
* Copyright (C) 2019-2021 OpenBikeSensor Contributors
3+
* Contact: https://openbikesensor.org
4+
*
5+
* This file is part of the OpenBikeSensor firmware.
6+
*
7+
* The OpenBikeSensor firmware is free software: you can
8+
* redistribute it and/or modify it under the terms of the GNU
9+
* Lesser General Public License as published by the Free Software
10+
* Foundation, either version 3 of the License, or (at your option)
11+
* any later version.
12+
*
13+
* OpenBikeSensor firmware is distributed in the hope that
14+
* it will be useful, but WITHOUT ANY WARRANTY; without even the
15+
* implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
16+
* PURPOSE. See the GNU Lesser General Public License for more
17+
* details.
18+
*
19+
* You should have received a copy of the GNU Lesser General Public
20+
* License along with the OpenBikeSensor firmware. If not,
21+
* see <http://www.gnu.org/licenses/>.
22+
*/
23+
#include "Arduino.h"
24+
#include "Firmware.h"
25+
#include <WiFiClientSecure.h>
26+
#include <HTTPClient.h>
27+
#include <SD.h>
28+
#include <Update.h>
29+
#include <esp_ota_ops.h>
30+
31+
/* TODO: Introduce cert management. */
32+
const char * const GITHUB_ROOT_CA =
33+
"-----BEGIN CERTIFICATE-----\n"
34+
"MIIDxTCCAq2gAwIBAgIQAqxcJmoLQJuPC3nyrkYldzANBgkqhkiG9w0BAQUFADBs\n"
35+
"MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\n"
36+
"d3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\n"
37+
"ZSBFViBSb290IENBMB4XDTA2MTExMDAwMDAwMFoXDTMxMTExMDAwMDAwMFowbDEL\n"
38+
"MAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3\n"
39+
"LmRpZ2ljZXJ0LmNvbTErMCkGA1UEAxMiRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug\n"
40+
"RVYgUm9vdCBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMbM5XPm\n"
41+
"+9S75S0tMqbf5YE/yc0lSbZxKsPVlDRnogocsF9ppkCxxLeyj9CYpKlBWTrT3JTW\n"
42+
"PNt0OKRKzE0lgvdKpVMSOO7zSW1xkX5jtqumX8OkhPhPYlG++MXs2ziS4wblCJEM\n"
43+
"xChBVfvLWokVfnHoNb9Ncgk9vjo4UFt3MRuNs8ckRZqnrG0AFFoEt7oT61EKmEFB\n"
44+
"Ik5lYYeBQVCmeVyJ3hlKV9Uu5l0cUyx+mM0aBhakaHPQNAQTXKFx01p8VdteZOE3\n"
45+
"hzBWBOURtCmAEvF5OYiiAhF8J2a3iLd48soKqDirCmTCv2ZdlYTBoSUeh10aUAsg\n"
46+
"EsxBu24LUTi4S8sCAwEAAaNjMGEwDgYDVR0PAQH/BAQDAgGGMA8GA1UdEwEB/wQF\n"
47+
"MAMBAf8wHQYDVR0OBBYEFLE+w2kD+L9HAdSYJhoIAu9jZCvDMB8GA1UdIwQYMBaA\n"
48+
"FLE+w2kD+L9HAdSYJhoIAu9jZCvDMA0GCSqGSIb3DQEBBQUAA4IBAQAcGgaX3Nec\n"
49+
"nzyIZgYIVyHbIUf4KmeqvxgydkAQV8GK83rZEWWONfqe/EW1ntlMMUu4kehDLI6z\n"
50+
"eM7b41N5cdblIZQB2lWHmiRk9opmzN6cN82oNLFpmyPInngiK3BD41VHMWEZ71jF\n"
51+
"hS9OMPagMRYjyOfiZRYzy78aG6A9+MpeizGLYAiJLQwGXFK3xPkKmNEVX58Svnw2\n"
52+
"Yzi9RKR/5CYrCsSXaQ3pjOLAEFe4yHYSkVXySGnYvCoCWw9E1CAx2/S6cCZdkGCe\n"
53+
"vEsXCS+0yx5DaMkHJ8HSXPfqIbloEpw8nL+e/IBcm2PN7EeqJSdnoDfzAIJ9VNep\n"
54+
"+OkuE6N36B9K\n"
55+
"-----END CERTIFICATE-----";
56+
57+
static const String FLASH_APP_FILENAME("/sdflash/app.bin");
58+
static const String OPEN_BIKE_SENSOR_FLASH_APP_PROJECT_NAME("OpenBikeSensorFlash");
59+
static const size_t APP_PARTITION_SIZE = 0x380000; // read from part?
60+
static const int SHA256_HASH_LEN = 32;
61+
62+
// todo: error handling
63+
void Firmware::downloadToSd(String url, String filename) {
64+
WiFiClientSecure client;
65+
client.setCACert(GITHUB_ROOT_CA);
66+
HTTPClient http;
67+
http.setUserAgent(mUserAgent);
68+
http.setFollowRedirects(HTTPC_STRICT_FOLLOW_REDIRECTS);
69+
70+
if (http.begin(client, url)) {
71+
int status = http.GET();
72+
log_i("Opening %s got %d.", url.c_str(), status);
73+
log_i("http size: %d. ", http.getSize());
74+
log_i("Will access Stream:, free heap %dkb", ESP.getFreeHeap() / 1024);
75+
if (status == 200) {
76+
File newFile = SD.open(filename, FILE_WRITE);
77+
int written = http.writeToStream(&newFile);
78+
newFile.close();
79+
log_i("Got %d bytes", written);
80+
}
81+
}
82+
http.end();
83+
}
84+
85+
bool Firmware::downloadToFlash(String url,
86+
std::function<void(uint32_t pos, uint32_t size)> progress) {
87+
bool success = false;
88+
WiFiClientSecure client;
89+
client.setCACert(GITHUB_ROOT_CA);
90+
HTTPClient http;
91+
http.setUserAgent(mUserAgent);
92+
http.setFollowRedirects(HTTPC_STRICT_FOLLOW_REDIRECTS);
93+
if (http.begin(client, url)) {
94+
progress(0, 999);
95+
int status = http.GET();
96+
log_i("Opening %s got %d.", url.c_str(), status);
97+
size_t size = http.getSize();
98+
log_i("http size: %d. ", size);
99+
log_i("Will access Stream:, free heap %dkb", ESP.getFreeHeap() / 1024);
100+
if (status == 200) {
101+
Update.begin();
102+
Update.onProgress([progress, size](size_t pos, size_t all) {
103+
progress(pos, size);
104+
});
105+
106+
Stream& stream = http.getStream();
107+
byte buffer[256];
108+
size_t read;
109+
size_t written = 0;
110+
while ((read = stream.readBytes(&buffer[0], sizeof(buffer))) > 0) {
111+
Update.write(buffer, read);
112+
written += read;
113+
}
114+
log_i("Got %d bytes", written);
115+
}
116+
}
117+
http.end();
118+
if (Update.end(true)) { //true to set the size to the current progress
119+
mLastMessage = "Success";
120+
121+
// Suppress the ota Firmware switch!
122+
const esp_partition_t *running = esp_ota_get_running_partition();
123+
esp_ota_set_boot_partition(running);
124+
success = true;
125+
} else {
126+
mLastMessage = Update.errorString();
127+
log_e("Update: %s", mLastMessage.c_str());
128+
}
129+
return success;
130+
}
131+
132+
String Firmware::getLastMessage() {
133+
return mLastMessage;
134+
};
135+
136+
const esp_partition_t* Firmware::findEspFlashAppPartition() {
137+
const esp_partition_t *part
138+
= esp_partition_find_first(ESP_PARTITION_TYPE_APP, ESP_PARTITION_SUBTYPE_APP_OTA_0, nullptr);
139+
140+
esp_app_desc_t app_desc;
141+
esp_err_t ret = ESP_ERROR_CHECK_WITHOUT_ABORT(esp_ota_get_partition_description(part, &app_desc));
142+
143+
if (ret != ESP_OK || OPEN_BIKE_SENSOR_FLASH_APP_PROJECT_NAME != app_desc.project_name) {
144+
part
145+
= esp_partition_find_first(ESP_PARTITION_TYPE_APP, ESP_PARTITION_SUBTYPE_APP_OTA_1,
146+
nullptr);
147+
ret = ESP_ERROR_CHECK_WITHOUT_ABORT(esp_ota_get_partition_description(part, &app_desc));
148+
}
149+
150+
if (ret == ESP_OK && OPEN_BIKE_SENSOR_FLASH_APP_PROJECT_NAME == app_desc.project_name) {
151+
return part;
152+
} else {
153+
return nullptr;
154+
}
155+
}
156+
157+
String Firmware::getFlashAppVersion() {
158+
const esp_partition_t *part = findEspFlashAppPartition();
159+
esp_app_desc_t app_desc;
160+
esp_err_t ret = ESP_ERROR_CHECK_WITHOUT_ABORT(esp_ota_get_partition_description(part, &app_desc));
161+
String version;
162+
if (ret == ESP_OK) {
163+
version = app_desc.version;
164+
}
165+
return version;
166+
}
167+
168+
static void calculateSha256(File &f, uint8_t *shaResult, int32_t bytes) {
169+
mbedtls_md_context_t ctx;
170+
mbedtls_md_type_t md_type = MBEDTLS_MD_SHA256;
171+
mbedtls_md_init(&ctx);
172+
mbedtls_md_setup(&ctx, mbedtls_md_info_from_type(md_type), 0);
173+
mbedtls_md_starts(&ctx);
174+
175+
const uint16_t bufferSize = 8192;
176+
uint8_t *buffer = static_cast<uint8_t *>(malloc(bufferSize));
177+
uint32_t toRead = bufferSize;
178+
uint32_t read;
179+
uint32_t readFromFile = 0;
180+
while ((read = f.read(buffer, toRead)) > 0) {
181+
mbedtls_md_update(&ctx, buffer, read);
182+
readFromFile += read;
183+
if (readFromFile + toRead > bytes) {
184+
toRead = bytes - readFromFile;
185+
}
186+
}
187+
free(buffer);
188+
mbedtls_md_finish(&ctx, shaResult);
189+
mbedtls_md_free(&ctx);
190+
}
191+
192+
String Firmware::checkSdFirmware() {
193+
String error;
194+
File f = SD.open(FLASH_APP_FILENAME, FILE_READ);
195+
if (!f) {
196+
error = "Failed to find '" + FLASH_APP_FILENAME + "' on sd card.";
197+
return error;
198+
}
199+
const int32_t fileSize = f.size();
200+
if (fileSize > APP_PARTITION_SIZE) {
201+
error = "Firmware to flash is to large. ";
202+
} else {
203+
uint8_t shaResult[SHA256_HASH_LEN];
204+
calculateSha256(f, shaResult, fileSize - SHA256_HASH_LEN);
205+
206+
for (int i = 0; i < SHA256_HASH_LEN; i++) {
207+
int c = f.read();
208+
if (shaResult[i] != c) {
209+
error = "Checksum mismatch.";
210+
break;
211+
}
212+
}
213+
}
214+
f.close();
215+
return error;
216+
}
217+
218+
bool Firmware::switchToFlashApp() {
219+
bool success = false;
220+
const esp_partition_t *part = findEspFlashAppPartition();
221+
if (part) {
222+
success = (ESP_ERROR_CHECK_WITHOUT_ABORT(esp_ota_set_boot_partition(part)) == ESP_OK);
223+
}
224+
return success;
225+
}

src/Firmware.h

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
/*
2+
* Copyright (C) 2019-2021 OpenBikeSensor Contributors
3+
* Contact: https://openbikesensor.org
4+
*
5+
* This file is part of the OpenBikeSensor firmware.
6+
*
7+
* The OpenBikeSensor firmware is free software: you can
8+
* redistribute it and/or modify it under the terms of the GNU
9+
* Lesser General Public License as published by the Free Software
10+
* Foundation, either version 3 of the License, or (at your option)
11+
* any later version.
12+
*
13+
* OpenBikeSensor firmware is distributed in the hope that
14+
* it will be useful, but WITHOUT ANY WARRANTY; without even the
15+
* implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
16+
* PURPOSE. See the GNU Lesser General Public License for more
17+
* details.
18+
*
19+
* You should have received a copy of the GNU Lesser General Public
20+
* License along with the OpenBikeSensor firmware. If not,
21+
* see <http://www.gnu.org/licenses/>.
22+
*/
23+
#ifndef OPENBIKESENSORFIRMWARE_FIRMWARE_H
24+
#define OPENBIKESENSORFIRMWARE_FIRMWARE_H
25+
26+
27+
#include <functional>
28+
29+
class Firmware {
30+
public:
31+
explicit Firmware(String userAgent) : mUserAgent(userAgent) {};
32+
void downloadToSd(String url, String filename);
33+
bool downloadToFlash(String url, std::function<void(uint32_t, uint32_t)> progress);
34+
String getLastMessage();
35+
36+
static String getFlashAppVersion();
37+
static String checkSdFirmware();
38+
static bool switchToFlashApp();
39+
40+
private:
41+
String mLastMessage;
42+
String mUserAgent;
43+
static const esp_partition_t *findEspFlashAppPartition();
44+
};
45+
46+
47+
#endif //OPENBIKESENSORFIRMWARE_FIRMWARE_H

src/OpenBikeSensorFirmware.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@
3333

3434
// --- Global variables ---
3535
// Version only change the "vN.M" part if needed.
36-
const char *OBSVersion = "v0.5" BUILD_NUMBER;
36+
const char *OBSVersion = "v0.6" BUILD_NUMBER;
3737

3838
const uint8_t LEFT_SENSOR_ID = 1;
3939
const uint8_t RIGHT_SENSOR_ID = 0;

0 commit comments

Comments
 (0)