Unable to view large files containing Chinese

[bfdyanshe]

lnav version
lnav 0.12.4 -- linux-musl-x86_64 version

Describe the bug
When opening a file that is too large and contains Chinese characters and Chinese symbols, it will display a hexadecimal view. For example, "lazy fox jump over the 狗子。"

To Reproduce
here is a bash script to generate test file.

#!/bin/bash

output_file="chinese_text.log"
chinese_text="Lazy 狐狸 jump over the 狗子。"
target_size=$((80 * 1024 * 1024))

> "$output_file"

line_size=$(echo -n "$chinese_text" | wc -c)
line_size=$((line_size + 1))

lines_per_write=10000
write_size=$((line_size * lines_per_write))

buffer=""
for i in $(seq 1 $lines_per_write); do
  buffer+="$chinese_text $i \n"
done

while true; do
  current_size=$(stat -c%s "$output_file")
  if [ "$current_size" -ge "$target_size" ]; then
    break
  fi
  printf "%b" "$buffer" >> "$output_file"
done

echo "check $output_file"

screenshot

less

lnav

[tstack]

Trying on macOS, it's showing up fine and not switching to hex view.

The hex view is shown because lnav considers the file to be binary, likely because it found some invalid UTF-8 content. Can you lnav with -d /tmp/lnav.dbg and attach the resulting debug log. It might give some more details on what is invalid.

[bfdyanshe]

lnav.dbg.txt

[tstack]

I pushed a change that seems to work for me, thank you for taking the time to file this issue.