fnjson.cpp Value Mangling Analysis

fnjson.cpp Value Mangling Analysis

Summary

The JSON parser in FujiNet firmware is mangling numeric values during the conversion from JSON to strings. The issue occurs inside fnJson in the getValue() function, which converts all numbers through floating-point types and loses precision.

Root Cause

Location

fnjson.cpp:205-228

The Problem

When handling JSON numbers, the code follows this process:

1. Line 207: cJSON extracts the number as a double

   double num = cJSON_GetNumberValue(item);

2. Line 208: Checks if the number is "approximately an integer"

   bool isInt = isApproximatelyInteger(num);

3. Lines 210-216: If it looks like an integer, casts to int64_t and outputs

   if (isInt) {
       ss << (int64_t)num;
   }

4. Lines 218-225: Otherwise, outputs as double with only 10 digits of precision

   else {
       ss << std::setprecision(10) << num;
   }

The isApproximatelyInteger Function

Located in utils.cpp:1064-1066:

bool isApproximatelyInteger(double value, double tolerance) {
    return std::abs(value - std::floor(value)) < tolerance;
}

This function checks if a number is within 1e-6 (0.000001) of an integer value.

How Values Get Mangled

For Integer Values

• Large integers like 1234567890123 in JSON are stored as double by cJSON
• When cast to int64_t, precision can be lost if the value exceeds what can be exactly represented as a double
• IEEE 754 double-precision can only represent integers exactly up to 2^53 (about 9 quadrillion)
• Values larger than this lose precision in the double → int64_t → string conversion

For Floating-Point Values

• Limited to 10 digits of precision via std::setprecision(10)
• Original string representation is lost
• Values like 3.141592653589793 become 3.141592654 (rounded to 10 digits)
• Scientific notation and trailing zeros are not preserved

Examples of Mangling

Original JSON Value	What cJSON Stores	What Gets Output	Problem
1234567890123456789	1.234567890123457e+18 (double)	1234567890123456768	Lost precision in large integer
3.141592653589793	3.141592653589793 (double)	3.141592654	Truncated to 10 digits
1.0	1.0 (double)	1	Converted to integer, loses .0
1e10	10000000000.0 (double)	10000000000	Scientific notation lost

Where the Issue Occurs

The mangling happens INSIDE fnJson, specifically in the FNJSON::getValue() method. The data is already mangled before it's sent back over the wire to the client.

The issue is NOT in:

• The network protocol layer
• The client-side parsing
• The wire transmission

CRITICAL BUG: Buffer Handling in readValue()

Caution

There is a critical bug in fnjson.cpp:278-286 that could cause random/incorrect values!

bool FNJSON::readValue(uint8_t *rx_buf, unsigned short len)
{    
    if (_item == nullptr)
        return true; // error

    memcpy(rx_buf, getValue(_item).data(), len);  // ⚠️ DANGEROUS!

    return false; // no error.
}

The Problem

The code creates a temporary std::string from getValue(_item), calls .data() on it to get the pointer, then immediately destroys the string after the memcpy. This is undefined behavior because:

1. getValue(_item) returns a temporary string
2. .data() returns a pointer to the temporary's internal buffer
3. The temporary is destroyed at the end of the expression
4. memcpy may be reading from freed memory
5. The buffer might contain garbage, old data, or corrupted values

Why This Causes Wrong Values

When copying from freed memory:

• You might get leftover data from previous calls
• Buffer contents could be partially overwritten
• Values from different JSON fields might bleed through
• Garbage memory could be interpreted as numbers

This explains seeing 56 instead of 57 or 955.9 instead of 974.0 - you're likely getting a mix of old values, garbage, or data from other JSON fields!

The Fix Required

bool FNJSON::readValue(uint8_t *rx_buf, unsigned short len)
{    
    if (_item == nullptr)
        return true; // error

    std::string value = getValue(_item);  // Store in variable first!
    memcpy(rx_buf, value.data(), std::min(len, (unsigned short)value.size()));

    return false; // no error.
}

Potential Solutions

Option 1: Preserve Original String from JSON (Recommended)

Modify the code to preserve the original string representation from the JSON input instead of converting through numeric types. This would require:

• Capturing the original string during parsing
• Storing it alongside the cJSON structure
• Returning the original string instead of the converted value

Option 2: Use cJSON's Print Functions

Use cJSON_Print() or cJSON_PrintUnformatted() to get the value as cJSON would serialize it:

char* value_str = cJSON_Print(item);
ss << value_str;
free(value_str);

Option 3: Increase Precision

Increase the precision from 10 to a higher value (e.g., 17 for doubles):

ss << std::setprecision(17) << num;

This helps but doesn't solve the fundamental issue of large integer precision loss.

Option 4: Use a Different JSON Parser

Switch to a JSON parser that preserves original string representations, such as:

• RapidJSON
• nlohmann/json
• simdjson

Recommendations

1. Short-term fix: Use cJSON_Print() to serialize individual values, preserving their original representation
2. Long-term solution: Consider switching to a JSON parser that preserves numeric precision and original formatting
3. Testing: Add test cases for:
   • Large integers (> 2^53)
   • High-precision floating-point numbers
   • Scientific notation
   • Trailing zeros in decimals

Code References

• Main issue: fnjson.cpp:166-273 (FNJSON::getValue())
• Helper function: utils.cpp:1064-1066 (isApproximatelyInteger())
• Header: fnjson.h