get_file_contents corrupts binary files (PDF, images, archives): Base64 content is force-decoded to UTF-8

[xiwan]

Summary

get_file_contents irreversibly corrupts every binary file in a repository (PDF, PNG/JPG, ZIP, XLSX/DOCX, fonts, compiled artifacts). The GitLab API returns file content as Base64, but getFileContents() force-decodes it into a UTF-8 string. Any byte sequence that is not valid UTF-8 is replaced with U+FFFD (EF BF BD), so the original bytes are permanently lost before the content ever reaches the MCP client.

Reported originally as "fetching a PDF loses data" — the file is not merely unusable, the byte stream is destroyed and cannot be reconstructed client-side.

Affected versions

• Confirmed on @zereight/mcp-gitlab@2.1.30
• Still present on main (926d42c) / 2.1.46 — the code is unchanged

Root cause

index.ts#L2054-L2058

// Decode Base64-encoded file content to UTF-8
if (!Array.isArray(parsedData) && parsedData.content) {
  parsedData.content = Buffer.from(parsedData.content, "base64").toString("utf8");
  parsedData.encoding = "utf8";
}

This conversion is unconditional — there is no binary detection and no way to opt out.

Note that GITLAB_REPO_FILE_ENCODING / --repo-file-encoding does not help here. It is only referenced in the write paths (encodeRepoFilePayloadContent() at L4783, plus L4821 and L4907); getFileContents() never consults it. So a user hitting this has no configuration-level workaround.

Reproduction

No GitLab instance required — this reproduces the exact transformation the server applies:

const header = Buffer.from('%PDF-1.7\n', 'latin1');
const binaryStream = Buffer.from([
  0x25,0xC7,0xEC,0x8F,0xA2,0x0A,
  0x78,0x9C,0xED,0x9A,0x4B,0x6F,0xDB,0x30,
  0x0C,0xC7,0xEF,0xFE,0x8A,0x82,0x9E,0x0A,
  0xFF,0xD8,0xFF,0xE0,0x00,0x10,0x4A,0x46
]);
const footer = Buffer.from('\n%%EOF\n', 'latin1');
const pdf = Buffer.concat([header, binaryStream, footer]);

const apiContent = pdf.toString('base64');          // what GitLab returns

// index.ts:2057
const afterMcp = Buffer.from(apiContent, 'base64').toString('utf8');
const restored = Buffer.from(afterMcp, 'utf8');     // best a client can do

console.log('original bytes :', pdf.length);                       // 46
console.log('restored bytes :', restored.length);                  // 73
console.log('identical      :', pdf.equals(restored));             // false
console.log('U+FFFD count   :', (afterMcp.match(/\uFFFD/g)||[]).length); // 14

Output:

original bytes : 46
restored bytes : 73
identical      : false
U+FFFD count   : 14

Byte-level diff:

original: 255044462d312e370a 25 c7 ec8fa2 0a 78 9c ed 9a 4b6f db 300c c7 ef fe 8a 82 9e 0a ffd8 ffe0 00104a460a2525454f460a
restored: 255044462d312e370a 25 efbfbd ec8fa2 0a 78 efbfbd efbfbd 4b6f efbfbd 300c efbfbd efbfbd efbfbd efbfbd efbfbd efbfbd 0a efbfbd efbfbd efbfbd efbfbd 00104a460a2525454f460a

Corruption is partial and unpredictable: ec8fa2 survives because it happens to form a valid UTF-8 sequence, while c7, 9c, ed, ff each collapse into efbfbd. This is why the symptom presents as "the PDF lost some data" rather than a clean failure.

Expected behaviour

Binary content should reach the client losslessly. A backward-compatible fix is to only decode when the round-trip is lossless, and otherwise pass the Base64 through with encoding left as base64 so the client can decode it itself:

if (!Array.isArray(parsedData) && parsedData.content) {
  const buf = Buffer.from(parsedData.content, "base64");
  const asText = buf.toString("utf8");
  if (Buffer.from(asText, "utf8").equals(buf)) {
    parsedData.content = asText;      // text file — unchanged behaviour
    parsedData.encoding = "utf8";
  } else {
    parsedData.encoding = "base64";   // binary — preserve bytes
  }
}

This keeps the current behaviour for all valid-UTF-8 files, so existing text workflows are unaffected.

Related: the write path has the same weakness

encodeRepoFilePayloadContent() does Buffer.from(content) with no explicit encoding, which defaults to UTF-8. Uploading binary content through create_or_update_file therefore corrupts it in the same way. Worth addressing together, since a round-trip (read then write) currently destroys data twice.

[quseijuro-design]

Hi, I’d like to work on this if it is still available.

I confirmed that getFileContents() unconditionally converts Base64 content to UTF-8. My proposed scope is the read path only: preserve the current UTF-8 result for losslessly decoded text, and retain Base64 with encoding: "base64" for non-UTF-8 content, with regression tests for both cases.

The issue also identifies a separate write-path concern. Would you prefer this PR to stay read-only, or include that in the same change?