security: add resource limits to config parser to prevent DoS

[ibrahimu8]

Security Fix: Prevent Config Parser Resource Exhaustion

Vulnerability:
Nebula's configuration parser lacked resource limits, allowing attackers to cause Denial of Service through memory/CPU exhaustion via specially crafted large config files.

Impact:

• Memory exhaustion: Config with 200,000 keys consumed 142MB
• CPU exhaustion: Processing time increased significantly with large configs
• Service disruption: Could become unresponsive or crash

Changes:

1. Added resource limits:

   • maxConfigFileSize = 5 * 1024 * 1024 (5MB max file size)
   • maxConfigKeys = 10000 (max keys per config file)

2. Added validation checks:

   • File size check before YAML parsing in parse()
   • Key count check after unmarshaling in both parse() and parseRaw()
   • Clear error messages for exceeded limits

Testing:

• ✅ Normal configs continue to work unchanged
• ✅ Configs exceeding limits are rejected with descriptive errors
• ✅ Original attack vectors now properly blocked

Backward Compatibility

• No breaking changes for legitimate configurations
• Only affects malicious/abusive config files
• Maintains full functionality for normal use cases

Context:
This addresses a resource exhaustion vulnerability where attackers could crash Nebula by providing extremely large configuration files.

[salesforce-cla]

Thanks for the contribution! Before we can merge this, we need @servis to sign the Salesforce Inc. Contributor License Agreement.

[jrwren]

Thanks for the patch

This doesn't seem like it actually does any prevention other than displaying an errors message instead of continuing.

I'd much rather see limits used on the file input read e.g. by wrapping the file reader in a io.LimitedReader, that is, if and only if we agree that this is something we want to address. I can't immediately think of a situation where an untrusted source would create a large configuration file, but maybe I'm not trying hard enough.

[JackDoan]

Hi @ibrahimu8, thanks for the contribution!

Limits are usually a good thing, but in this case I think it's important to point out that if an attacker is able to control the contents of the nebula config file, it's over. DOS would be a kindness compared to what would otherwise be possible.

That said, I'm open to merging if you wanted to take the io.LimitedReader approach.