Getting a codec error but what does that actually mean?

[pvandervelde]

I'm using reqwless 0.13.0 on an esp32-c6 dev board. I've got all default features turned off and only the embedded-tls and log features turned on. I'm suing embedded-tls 0.17.0. I'm taking an environmental reading (pressure, humidity and temperature) using a BME280. Then this data should be send to Grafana cloud via the Influx protocol.

The output I'm getting is

INFO tank_sensor_level_embedded::data_recording] Received sample at 2025-01-28T10:45:01 UTC
 INFO tank_sensor_level_embedded::data_recording]  ┣ Temperature: 25.58 C
 INFO tank_sensor_level_embedded::data_recording]  ┣ Humidity:    46.29 %
 INFO tank_sensor_level_embedded::data_recording]  ┗ Pressure:    1015.17 hPa
 INFO tank_sensor_level_embedded::data_recording] Sending data to grafana ...
DEBUG tank_sensor_level_embedded::data_recording] Creating HTTP client ...
DEBUG tank_sensor_level_embedded::data_recording] Creating request ...
DEBUG embedded_tls::write_buffer] start_record(Handshake(false))
TRACE embedded_tls::asynch] State ClientHello -> ServerHello
DEBUG embedded_tls::record_reader] advance: Handshake - content_length = 123 bytes
TRACE embedded_tls::handshake] handshake = ServerHello
TRACE embedded_tls::extensions::messages] Extension buffer: 79
DEBUG embedded_tls::extensions::messages] Read extension type SupportedVersions
TRACE embedded_tls::extensions::messages] Extension data length: 2
TRACE embedded_tls::extensions::messages] Extension buffer: 73
DEBUG embedded_tls::extensions::messages] Read extension type KeyShare
TRACE embedded_tls::extensions::messages] Extension data length: 69
TRACE embedded_tls::extensions::messages] Read 2 extensions
DEBUG embedded_tls::handshake::server_hello] server cipher_suite TlsAes128GcmSha256
DEBUG embedded_tls::handshake::server_hello] server extensions [SupportedVersions(SupportedVersionsServerHello { selected_version: ProtocolVersion(772) }), KeyShare(KeyShareServerHello(KeyShareEntry { group: Secp256r1, opaque: [4, 75, 238, 204, 35, 98, 225, 228, 87, 50, 64, 207, 139, 135, 215, 162, 33, 235, 15, 223, 21, 237, 3, 43, 111, 203, 212, 167, 161, 176, 148, 148, 167, 171, 148, 194, 254, 142, 18, 75, 163, 128, 90, 85, 184, 191, 70, 146, 170, 50, 239, 149, 201, 222, 76, 221, 215, 145, 184, 49, 117, 119, 115, 45, 244] }))]
TRACE embedded_tls::connection] ********* ServerHello
TRACE embedded_tls::asynch] State ServerHello -> ServerVerify
DEBUG embedded_tls::record_reader] advance: ChangeCipherSpec - content_length = 1 bytes
TRACE embedded_tls::connection] Not decrypting: content_type = ChangeCipherSpec
TRACE embedded_tls::asynch] State ServerVerify -> ServerVerify
DEBUG embedded_tls::record_reader] advance: ApplicationData - content_length = 27 bytes
TRACE embedded_tls::connection] Decrypting: content type = Handshake
TRACE embedded_tls::handshake] handshake = EncryptedExtensions
TRACE embedded_tls::extensions::messages] Extension buffer: 4
DEBUG embedded_tls::extensions::messages] Read extension type ServerName
TRACE embedded_tls::extensions::messages] Extension data length: 0
TRACE embedded_tls::extensions::messages] Read 1 extensions
TRACE embedded_tls::asynch] State ServerVerify -> ServerVerify
DEBUG embedded_tls::record_reader] advance: ApplicationData - content_length = 3206 bytes
TRACE embedded_tls::connection] Decrypting: content type = Handshake
TRACE embedded_tls::handshake] handshake = Certificate
TRACE embedded_tls::extensions::messages] Read 0 extensions
TRACE embedded_tls::extensions::messages] Read 0 extensions
DEBUG embedded_tls::connection] Certificate verified!
TRACE embedded_tls::asynch] State ServerVerify -> ServerVerify
DEBUG embedded_tls::record_reader] advance: ApplicationData - content_length = 281 bytes
TRACE embedded_tls::connection] Decrypting: content type = Handshake
TRACE embedded_tls::handshake] handshake = CertificateVerify
DEBUG embedded_tls::connection] Signature verified!
TRACE embedded_tls::asynch] State ServerVerify -> ServerVerify
DEBUG embedded_tls::record_reader] advance: ApplicationData - content_length = 53 bytes
TRACE embedded_tls::connection] Decrypting: content type = Handshake
TRACE embedded_tls::handshake] handshake = Finished
TRACE embedded_tls::asynch] State ServerVerify -> ClientFinished
DEBUG embedded_tls::write_buffer] start_record(Handshake(true))
TRACE embedded_tls::connection] output size 53
TRACE embedded_tls::asynch] State ClientFinished -> ApplicationData
DEBUG tank_sensor_level_embedded::data_recording] Sending request ...
DEBUG embedded_tls::write_buffer] start_record(ApplicationData)
DEBUG tank_sensor_level_embedded::data_recording] Processing response ...
ERROR tank_sensor_level_embedded::data_recording] Failed to send data to grafana: error Codec
ERROR tank_sensor_level_embedded::data_recording] Could not send data to Grafana: RequestFailed

The relevant code is:

use core::fmt::Write;

use embassy_net::tcp::client::TcpClientState;
use embassy_net::Stack;
use embassy_net::{dns::DnsSocket, tcp::client::TcpClient};
use embassy_sync::channel::Sender;
use embassy_sync::{blocking_mutex::raw::NoopRawMutex, channel::Receiver};

use heapless::String;

use hifitime::Epoch;
use log::info;
use log::{debug, error};

use rand_core::RngCore as _;

use reqwless::client::{HttpClient, TlsConfig, TlsVerify};
use reqwless::{
    headers::ContentType,
    request::{Method, RequestBuilder},
};

use thiserror::Error;

use uom::si::pressure::pascal;
use uom::si::{pressure::hectopascal, ratio::percent, thermodynamic_temperature::degree_celsius};

const GRAFANA_CLOUD_URL: &str = "https://influx-prod-41-prod-au-southeast-1.grafana.net/api/v1";
const GRAFANA_USER_NAME: &str = "my-user";
const GRAFANA_API_KEY: &str = env!("GRAFANA_METRICS_API_KEY");

/// A clock error
#[derive(Error, Debug)]
pub enum Error {
    #[error("The response code does not indicate success.")]
    NonSuccessResponseCode,

    #[error("The request failed to send.")]
    RequestFailed,
}

#[derive(Clone, Debug, Default)]
pub struct EnvironmentalData {
    /// Temperature
    pub temperature: Temperature,

    /// Humidity
    pub humidity: Humidity,

    /// Air Pressure
    pub pressure: Pressure,
}

/// A reading, i.e. a pair (time, sample)
pub type Reading = (Epoch, EnvironmentalData);

// Use the influx line protocol from here: https://docs.influxdata.com/influxdb/v1/write_protocols/line_protocol_tutorial/
fn format_metrics(boot_count: u32, environmental_data: Reading) -> String<512> {
    let mut buffer: String<512> = String::new();

    buffer
}

async fn send_data_to_grafana<'a>(
    stack: Stack<'a>,
    rng_wrapper: &mut RngWrapper,
    boot_count: u32,
    environmental_data: Reading,
) -> Result<(), Error> {
    info!("Sending data to grafana ...");

    let metrics = format_metrics(boot_count, environmental_data);
    let bytes = metrics.as_bytes();

    let dns_socket = DnsSocket::new(stack);

    let tcp_client_state = TcpClientState::<1, 4096, 4096>::new();
    let tcp_client = TcpClient::new(stack, &tcp_client_state);

    let seed = rng_wrapper.next_u64();
    let mut read_record_buffer = [0_u8; 16640];
    let mut write_record_buffer = [0_u8; 16640];

    let tls_config = TlsConfig::new(
        seed,
        &mut read_record_buffer,
        &mut write_record_buffer,
        TlsVerify::None,
    );

    debug!("Creating HTTP client ...");
    let mut client = HttpClient::new_with_tls(&tcp_client, &dns_socket, tls_config);

    debug!("Creating request ...");
    let mut rx_buf = [0; 4096];
    let mut resource = client.resource(GRAFANA_CLOUD_URL).await.unwrap();
    let response = resource
        .post("push/influx/write")
        .content_type(ContentType::TextPlain)
        .basic_auth(GRAFANA_USER_NAME, GRAFANA_API_KEY)
        .body(bytes);

    debug!("Sending request ...");
    let response = response.send(&mut rx_buf).await;

    debug!("Processing response ...");
    match response {
        Ok(r) => {
            if r.status.is_successful() {
                debug!("Send data to grafana. Status code: {:?}", r.status);
                Ok(())
            } else {
                error!("Failed to send data to grafana: Status code {:?}", r.status,);
                Err(Error::NonSuccessResponseCode)
            }
        }
        Err(e) => {
            error!("Failed to send data to grafana: error {:?}", e);
            Err(Error::RequestFailed)
        }
    }
}

What I'm after is some explanation of what the error actually means. Is it a problem with the TLS or with something else when connecting with the grafana cloud. When I use the address and user / key I can push data from postman to grafana cloud.

[pvandervelde]

Having looked at the source code I think my problem is on this line:

reqwless/src/request.rs

Line 129 in 232e40f

let mut combined: String<128> = String::new();

. That seems to create a buffer with a size of a 128 characters. However the Grafana API key is 165 characters in length (+ the username of about 10 characters). So I think my Codec issue is caused by the buffer being too small for the API key I have been given.

[rmja]

One way around this is to encode the header manually.

[rmja]

I wonder if it would be possible to stream the base64 encoding in batches instead of just increasing the buffer?

[pvandervelde]

I'll try encoding the header manually.