Educational decoder unable to decode real world .zst files?

[zhangboyang]

Describe the bug
Hello, I'm new to Zstandard. I found the educational decoder can't decode real world .zst files. I'm not sure if educational decoder is expected to decode real world .zst files. If I'm wrong, please point out. Thank you!

To Reproduce

1. Downloads linux-5.11.12.tar.xz and xz -d linux-5.11.12.tar.xz
2. zstd linux-5.11.12.tar
3. Compile educational decoder: gcc -O2 -o harness harness.c zstd_decompress.c
4. Decode using educational decoder: ./harness linux-5.11.12.tar.zst tmp

Expected behavior
Output of the educational decoder (tmp) is same as input (linux-5.11.12.tar)

Screenshots and charts

$ ../../zstd linux-5.11.12.tar
linux-5.11.12.tar    : 16.72%   (1064488960 => 178014949 bytes, linux-5.11.12.tar.zst) 
$ gcc -O2 -o harness  harness.c zstd_decompress.c
$ ./harness linux-5.11.12.tar.zst tmp
Error: Corruption detected while decompressing
$ 

Desktop (please complete the following information):

• OS: Ubuntu 20.04
• Compiler: gcc
• Flags: default
• Build system: Makefile

[terrelln]

Yes, the educational decoder should correctly decompress any zstd compressed file. We will look into fixing this at some point (or would accept PRs that fix it).

However, I would not recommend using the educational decoder in production. It is meant as an aid for understanding the zstd format, and writing compatible decoders. The zstd decoder is heavily optimized, hardened, and fuzz tested. The educational decoder is none of the above.

[zhangboyang]

OK, Thank you for quick explanation and advice.