Allow optional strict validation

[jeremy-visionaid]

Improve error tolerance and make strict validation optionally enabled.

[Sicos1977]

Hi Jeremy,

Is it possible for you to release a version that fixes the issue Creation time must be zero for streams and root

I use OpenMCDF in my MSGReader tool and also some other office things and have at least 10 errors a day with Excel documents. These things worked before I needed to upgrade all the code because of the new OpenMCDF v3+ version.

[jeremy-visionaid]

Heya. I'm looking to do that possibly later today... ironfede has just added a NuGet API key (I don't have admin access), so #299 should be unblocked.

One thing I'm still missing though is the source of the corruption. I can't repro with vanilla Excel. Do you what steps are required to be able to produce a corrupt file? (i.e. Excel version, plugins/external tools?)

[Sicos1977]

The Excels that are giving me the errors just come from external customers. So somehow Excel or another tool that can makes Excel documents generate these streams with a set creation time

[Sicos1977]

I see that a new 3.02 version has been placed on NuGet. Thank you and greetings from the past (10 hours earlier here :-))

[SteAmeR]

I understand the MS-CFB specification and RFC2119 standards. However, it seems somewhat inconsistent that applications like Microsoft Office and LibreOffice can handle these seemingly non-compliant files without any issues, while OpenMcdf cannot due to its strict validation rules.

I'm absolutely not opposed to performing these validations - they serve an important purpose. However, I believe that throwing exceptions and terminating the structural process entirely may not be the most reasonable approach.

Alternative suggestion:
Instead of throwing exceptions, what if OpenMcdf collected these strict rule violations into a Directory or List array and returned them to the caller at the end of the operation? This approach would provide several benefits:

1. File accessibility: Files would still open (like they did in v2.4.1)
2. Validation transparency: Detected issues would be reported to the caller
3. Developer choice: The calling application could decide how to handle validation warnings
4. Backward compatibility: Existing workflows wouldn't break
5. Standards compliance: Validation still occurs, but non-fatally

This would be similar to how many parsers handle warnings vs errors - they complete the parsing operation but inform the caller about potential issues. The caller could then choose whether to:

• Log the warnings and continue
• Present warnings to the user
• Treat specific violations as critical based on their use case
• Implement their own fallback logic

Would this kind of "validation with warnings" approach be feasible for addressing these compatibility concerns while maintaining the quality standards of OpenMcdf 3.0+?

[jeremy-visionaid]

I'm not saying that some recovery of validation errors shouldn't/won't be handled (espcially for simple cases). I'm saying it's not a goal for v3.0 (or a bug). The degree to which a client handles violations of the specification is at the client's discretion. In the general case I can't enforce consistency since there are multiple different implementations and I don't control their behaviour. My main concern for v3.0 is that OpenMcdf is able to read and write valid CFB files according to the spec. There are myriad ways that validation/corruption might technically be recovered from and the complexity of that can't necessarily be covered by your alternative suggestion (CFB is akin to a file-system - it's not a parser per-se).

To set expectations, my time available to work on OpenMcdf is fairly limited. If the community has a need for a particular validation strategy, then that effort would likely need to be a community contribution or possibly even a fork. I am happy to fix bugs with reasonably high priority, and to act as a reviewer as time allows.

For now, if you can provide a sample, I think it's reasonable to just disable that check since it's unlikely to have other side effects. However, I still don't have a sample file and reproduction of an issue is often most of the battle, and required for preventing regressions etc.

FWIW, it looks like Wine creates invalid storage entries. So that's one potential culprit:
https://github.com/wine-mirror/wine/blob/1d1e5fb3e51b2acb0143e86c16463dfed1bc90aa/dlls/ole32/storage32.c#L2205

[SteAmeR]

Hello,

Thank you for your positive and constructive approach. I should clarify the context of my use case, which might help explain the nature of the files I'm encountering.

I use the OpenMcdf library for malware analysis and detection - specifically to identify techniques used by malicious software developers. As you can appreciate, malware authors are primarily focused on successfully infecting target systems rather than adhering to specifications. Consequently, when they create CFB files for their attacks, they utilize dozens of different creator applications and tools, often resulting in files that don't strictly conform to specifications.

Unfortunately, these malicious files work perfectly fine in legitimate applications (Office Excel, LibreOffice, StarOffice, etc.) and successfully infect victims, which is precisely why they're effective attack vectors.

Regarding the current issue:
I believe the file I'm experiencing problems with is indeed affected by the Wine library issue you mentioned. A temporary disable check for this validation would be tremendously helpful for our security research work.

Context for the community:
This use case highlights why error tolerance in file format libraries is valuable - not just for handling legacy files or application quirks, but also for security research and forensic analysis where we need to examine potentially malformed files that still function in real-world scenarios.

Additional Context - File Source Identification:
I'd also like to share a GitHub project that creates these types of problematic files. The project's purpose is to enable manipulation of Office macros, and it's a perfect example of legitimate security research tools that produce files with validation issues:

EvilClippy: https://github.com/outflanknl/EvilClippy

This tool is used by security researchers and red teams to test macro-based attack vectors. I've included a sample file created by this tool in the attachments, which should help demonstrate the validation issues we're encountering.

This further illustrates why error tolerance is valuable for security research - we need to analyze files created by various tools (both legitimate and malicious) that may not strictly conform to specifications but still function in real-world environments.

Thank you for considering this fix, and I'll make sure to provide the sample file for testing.

Best regards

NameX_Evil.docx

[jeremy-visionaid]

Related: #335