[SECURITY] Improve SSRF checks, strict path check for well_known_path

[ajinabraham]

Describe the Pull Request

* Improved SSRF checks (credential checks, length check, port check, path, query, and params check, ipv6, ipv4 coverage, handle possible decimal or hex IP bypasses)
* Add additional strict path check for Applink well known path
* Moved `valid_host` to `security.py`

Checklist for PR

• Run MobSF unit tests and lint tox -e lint,test
• Tested Working on Linux, Mac, Windows, and Docker
• Add unit test for any new Web API (Refer: StaticAnalyzer/tests.py)
• Make sure tests are passing on your PR

Additional Comments (if any)

DESCRIBE HERE

[github-actions]

👋 @ajinabraham
Thank you for sending this pull request ❤️.
Please make sure you have followed our contribution guidelines. We will review it as soon as possible

[Sim4n6]

Why not. Use host.scheme here?

[ajinabraham]

host is not parsed yet, we can have hosts with user controlled scheme or no scheme at all, So forcing HTTP scheme on it.

[Sim4n6]

Each time this is called a DNS rebinding is possible

[Sim4n6]

Internal port scanning is still possible with the next urls, the ip_addresses = socket.getaddrinfo(hostname, None) does issue a DNS query thus permitting the host to switch from 1.1.1.1:80 to 127.0.0.1:80

• "prefix1-make-1.1.1.1-rebindfor30safter1times-127.0.0.1-rr.1u.ms:80"
• "prefix1-make-1.1.1.1-rebindfor30safter1times-127.0.0.1-rr.1u.ms:81"
• ....
• "prefix1-make-1.1.1.1-rebindfor30safter1times-127.0.0.1-rr.1u.ms:443"

measuring the response time would yield if the port is open or not.

[ajinabraham]

The PR doesn't do anything against DNS rebind. We cannot removesocket.getaddrinfo as it's required to translate IPs from hostnames dynamically to validate against a known private/internal list.

I did some overall refactoring to SSRF checks and enforced strict hardcoded path check for the 2 places we invoke user controlled requests.

[ajinabraham]

Since firebase and assetlink looks at https://, or http://, I can also do a port check.

[Sim4n6]

how about using the initially retrieved value of socket.getaddreinfo to avoid the DNS rebind ?

[ajinabraham]

We cannot rely on the IP alone to resolve a resource. It depends on the virtual hosting or domain based routing configurations of the server.

a single IP address can host different resources at the same path across multiple domain names.

[Sim4n6]

indeed, but after the resolving of socket.getaddrinfo() the next call to .get() would point to the second DNS rebinded host, thus still vulnerable to a certain extent with a limited impact.

Regards

[ajinabraham]

Yes, I acknowledge that this PR does not address the DNS rebind issue. A potential fix could involve monkey patching requests/urllib3 or implementing a custom HTTP request library with fixed domain name and IP. However, I am hesitant to add this unless there is a medium/high impact real-world exploit case. The current checks in place offer a balanced approach to mitigating the SSRF issue and its primary exploit use cases.

[Sim4n6]

I see now, anyway, keep me posted about when you would release the advisory. Thank you