[spi-hdlc-adapter] validate RCP frame lengths against the MTU (openthread#13206)

The slave (RCP) controls the `data_len` and `accept_len` fields of the SPI
header. Both carry a payload length that excludes the 5-byte header, as
shown by `sMTU = MAX_FRAME_SIZE - HEADER_LEN`, so the largest valid value
is the MTU, `MAX_FRAME_SIZE - HEADER_LEN`.

The two sanity checks in `push_pull_spi()` only rejected values greater
than `MAX_FRAME_SIZE`, allowing an RCP to advertise a length in the range
(MAX_FRAME_SIZE - HEADER_LEN, MAX_FRAME_SIZE]. That value flows into
`spi_xfer_bytes`, and `do_spi_xfer()` then transfers
`spi_xfer_bytes + HEADER_LEN + sSpiRxAlignAllowance` bytes into
`sSpiRxFrameBuffer` / `sSpiTxFrameBuffer`. Those buffers are sized
`MAX_FRAME_SIZE + SPI_RX_ALIGN_ALLOWANCE_MAX`, so the extra HEADER_LEN
added by the transfer can write up to 5 bytes past the end of both.

Clamp both checks to `MAX_FRAME_SIZE - HEADER_LEN` so the advertised
payload plus the header always fits within the existing buffers.

Author: evilgensec

tools/spi-hdlc-adapter/spi-hdlc-adapter.c

@@ -531,8 +531,9 @@ static int push_pull_spi(void)
     spi_header_set_accept_len(sSpiTxFrameBuffer, 0);
     spi_header_set_data_len(sSpiTxFrameBuffer, 0);
 
-    // Sanity check.
-    if (slave_data_len > MAX_FRAME_SIZE)
+    // Sanity check. The header `data_len` carries the payload length only
+    // (it excludes HEADER_LEN), so the largest valid value is the MTU.
+    if (slave_data_len > MAX_FRAME_SIZE - HEADER_LEN)
     {
         slave_data_len = 0;
     }
@@ -630,8 +631,8 @@ static int push_pull_spi(void)
     slave_max_rx   = spi_header_get_accept_len(spiRxFrameBuffer);
     slave_data_len = spi_header_get_data_len(spiRxFrameBuffer);
 
-    if (((slave_header & SPI_HEADER_PATTERN_MASK) != SPI_HEADER_PATTERN_VALUE) || (slave_max_rx > MAX_FRAME_SIZE) ||
-        (slave_data_len > MAX_FRAME_SIZE))
+    if (((slave_header & SPI_HEADER_PATTERN_MASK) != SPI_HEADER_PATTERN_VALUE) ||
+        (slave_max_rx > MAX_FRAME_SIZE - HEADER_LEN) || (slave_data_len > MAX_FRAME_SIZE - HEADER_LEN))
     {
         sSpiGarbageFrameCount++;
         sSpiTxRefusedCount++;