fix: Python upgrade readiness changes by removing requirements.txt #148

munna-shaik-s1 · 2024-12-04T08:37:23Z

Jira Link: https://sentinelone.atlassian.net/browse/{TICKET}

🥅 Goal

Explain why is this change needed

🛠️ Solution

Provide description of changes

🏫 Testing

How the changes were tested

TA_dataset/lib/requests/auth.py

+            def md5_utf8(x):
+                if isinstance(x, str):
+                    x = x.encode("utf-8")
+                return hashlib.md5(x).hexdigest()


To fix the problem, we need to replace the weak hashing algorithms (MD5 and SHA-1) with a strong, modern cryptographic hash function. For password hashing, we should use a computationally expensive algorithm such as Argon2, bcrypt, or PBKDF2. In this case, we will use the argon2-cffi library to implement Argon2 hashing.

Steps to fix the issue:

Install the argon2-cffi library if it is not already installed.

Replace the MD5 and SHA-1 hashing functions with Argon2 hashing functions.

Update the code to use the new hashing functions.

TA_dataset/lib/requests/auth.py

+            def sha_utf8(x):
+                if isinstance(x, str):
+                    x = x.encode("utf-8")
+                return hashlib.sha1(x).hexdigest()


To fix the problem, we need to replace the use of the SHA-1 hashing algorithm with a more secure and computationally expensive algorithm suitable for password hashing. One of the best options is to use the argon2 algorithm, which is designed for secure password hashing.

Steps to fix the problem:

Import the argon2 library.

Replace the SHA-1 hashing function with the argon2 hashing function.

Ensure that the new hashing function is used consistently throughout the code.

TA_dataset/lib/requests/auth.py

+            def sha256_utf8(x):
+                if isinstance(x, str):
+                    x = x.encode("utf-8")
+                return hashlib.sha256(x).hexdigest()


To fix the problem, we need to replace the weak hashing algorithms (MD5, SHA-1, and SHA-256) with a strong, modern cryptographic hash function suitable for password hashing. Argon2 is a good choice for this purpose as it is designed to be computationally expensive and includes a per-password salt by default.

Replace the MD5, SHA-1, and SHA-256 hashing functions with Argon2.

Import the argon2 library.

Update the code to use Argon2 for hashing passwords.

TA_dataset/lib/requests/auth.py

+            def sha512_utf8(x):
+                if isinstance(x, str):
+                    x = x.encode("utf-8")
+                return hashlib.sha512(x).hexdigest()


To fix the problem, we need to replace the weak cryptographic hashing algorithms (MD5, SHA-1, and SHA-512) with a strong, modern cryptographic hash function suitable for password hashing. The best way to do this is to use the argon2 algorithm, which is designed for secure password hashing.

Steps to fix:

Replace the MD5, SHA-1, and SHA-512 hashing functions with Argon2.

Import the argon2 library.

Update the code to use Argon2 for hashing passwords.

TA_dataset/lib/socks.py

+                msg = "Error connecting to {} proxy {}".format(
+                    printable_type, proxy_server
+                )
+                log.debug("%s due to: %s", msg, error)


To fix the problem, we should avoid logging sensitive information such as passwords. Instead of logging the full proxy address and port, we can log a sanitized version that does not include sensitive details. This can be achieved by masking or omitting the sensitive parts of the information before logging.

In the file TA_dataset/lib/socks.py, we need to modify the logging statement on line 852 to ensure that sensitive information is not logged. We can create a sanitized version of the proxy_server variable that omits the password.

TA_dataset/lib/splunktalib/rest.py

+            if resp.status_code not in (200, 201):
+                if not (method == "GET" and resp.status_code == 404):
+                    log.logger.debug(
+                        msg_temp, splunkd_uri, resp.status_code, code_to_msg(resp)


To fix the problem, we should avoid logging sensitive information such as splunkd_uri directly. Instead, we can log a sanitized version of the URI or avoid logging it altogether. We will modify the logging statements to ensure that sensitive data is not exposed.

In the file TA_dataset/lib/splunktalib/rest.py, we will update the logging statements on lines 66 and 71 to exclude the splunkd_uri or replace it with a sanitized version.

We will introduce a helper function to sanitize the splunkd_uri before logging.

TA_dataset/lib/splunktalib/rest.py

+            if resp.status_code not in (200, 201):
+                if not (method == "GET" and resp.status_code == 404):
+                    log.logger.debug(
+                        msg_temp, splunkd_uri, resp.status_code, code_to_msg(resp)


To fix the problem, we need to ensure that sensitive information is not logged in clear text. We can achieve this by masking or redacting sensitive data before logging it. Specifically, we should modify the code_to_msg function to redact sensitive information from the response text before logging it.

Modify the code_to_msg function in TA_dataset/lib/splunktalib/rest.py to redact sensitive information.

Add a helper function to redact sensitive information from the response text.

Ensure that the logging statements use the redacted response text.

TA_dataset/lib/urllib3/util/ssl_.py

+                "ssl_version": self.protocol,
+                "server_side": server_side,
+            }
+            return wrap_socket(socket, ciphers=self.ciphers, **kwargs)


To fix the problem, we need to ensure that a secure protocol version is specified when calling wrap_socket. The best way to do this is to explicitly set the ssl_version to ssl.PROTOCOL_TLSv1_2 or higher. This change should be made in the wrap_socket method of the SSLContext class.

We will update the kwargs dictionary to include ssl_version: ssl.PROTOCOL_TLSv1_2 before passing it to the wrap_socket function. Additionally, we need to import the ssl module to access the PROTOCOL_TLSv1_2 constant.

TA_dataset/lib/urllib3/util/ssl_.py

+        return SSLTransport(sock, ssl_context, server_hostname)
+
+    if server_hostname:
+        return ssl_context.wrap_socket(sock, server_hostname=server_hostname)


To fix the problem, we need to ensure that the ssl_context is configured to use a secure protocol version, such as TLSv1.2 or above. This can be done by setting the minimum_version attribute of the ssl_context to ssl.TLSVersion.TLSv1_2. This change should be made in the ssl_wrap_socket function where the ssl_context is created or passed.

TA_dataset/lib/urllib3/util/ssl_.py

+    if server_hostname:
+        return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
+    else:
+        return ssl_context.wrap_socket(sock)


To fix the problem, we need to ensure that the ssl_context is configured to use a secure protocol version, such as TLSv1.2 or higher. We can achieve this by setting the minimum_version attribute of the ssl_context to ssl.TLSVersion.TLSv1_2. This change should be made in the _ssl_wrap_socket_impl function where the ssl_context is used.

fix: Python upgrade readiness cahnges

a16e7b3

github-advanced-security bot found potential problems Dec 4, 2024

View reviewed changes

munna-shaik-s1 changed the title ~~fix: Python upgrade readiness cahnges~~ fix: Python upgrade readiness changes by removing requirements.txt Dec 4, 2024

munna-shaik-s1 closed this Dec 4, 2024

@@ -9,2 +9,3 @@
             import hashlib
+            from argon2 import PasswordHasher
             import os
@@ -143,18 +144,11 @@
                     # lambdas assume digest modules are imported at the top level
-                    if _algorithm == "MD5" or _algorithm == "MD5-SESS":
+                    if _algorithm == "MD5" or _algorithm == "MD5-SESS" or _algorithm == "SHA":
-                        def md5_utf8(x):
+                        def argon2_utf8(x):
+                            ph = PasswordHasher()
                             if isinstance(x, str):
                                 x = x.encode("utf-8")
-                            return hashlib.md5(x).hexdigest()
+                            return ph.hash(x)
-                        hash_utf8 = md5_utf8
-                    elif _algorithm == "SHA":
-                        def sha_utf8(x):
-                            if isinstance(x, str):
-                                x = x.encode("utf-8")
-                            return hashlib.sha1(x).hexdigest()
-                        hash_utf8 = sha_utf8
+                        hash_utf8 = argon2_utf8
                     elif _algorithm == "SHA-256":

Package	Version	Security advisories
argon2-cffi (pypi)	23.1.0	None

@@ -9,2 +9,3 @@
             import hashlib
+            from argon2 import PasswordHasher
             import os
@@ -153,8 +154,7 @@
-                        def sha_utf8(x):
-                            if isinstance(x, str):
-                                x = x.encode("utf-8")
-                            return hashlib.sha1(x).hexdigest()
+                        def argon2_hash(x):
+                            ph = PasswordHasher()
+                            return ph.hash(x)
-                        hash_utf8 = sha_utf8
+                        hash_utf8 = argon2_hash
                     elif _algorithm == "SHA-256":

Package	Version	Security advisories
argon2-cffi (pypi)	23.1.0	None

@@ -9,2 +9,3 @@
             import hashlib
+            from argon2 import PasswordHasher
             import os
@@ -143,34 +144,38 @@
                     # lambdas assume digest modules are imported at the top level
-                    if _algorithm == "MD5" or _algorithm == "MD5-SESS":
+                    ph = PasswordHasher()
-                        def md5_utf8(x):
-                            if isinstance(x, str):
-                                x = x.encode("utf-8")
-                            return hashlib.md5(x).hexdigest()
-                        hash_utf8 = md5_utf8
-                    elif _algorithm == "SHA":
-                        def sha_utf8(x):
-                            if isinstance(x, str):
-                                x = x.encode("utf-8")
-                            return hashlib.sha1(x).hexdigest()
-                        hash_utf8 = sha_utf8
-                    elif _algorithm == "SHA-256":
-                        def sha256_utf8(x):
-                            if isinstance(x, str):
-                                x = x.encode("utf-8")
-                            return hashlib.sha256(x).hexdigest()
-                        hash_utf8 = sha256_utf8
-                    elif _algorithm == "SHA-512":
-                        def sha512_utf8(x):
-                            if isinstance(x, str):
-                                x = x.encode("utf-8")
-                            return hashlib.sha512(x).hexdigest()
+                    def argon2_hash(x):
+                        if isinstance(x, str):
+                            x = x.encode("utf-8")
+                        return ph.hash(x)
-                        hash_utf8 = sha512_utf8
+                    hash_utf8 = argon2_hash
+                    KD = lambda s, d: hash_utf8(f"{s}:{d}")  # noqa:E731
+                    if hash_utf8 is None:
+                        return None
+                    # XXX not implemented yet
+                    entdig = None
+                    p_parsed = urlparse(url)
+                    #: path is request-uri defined in RFC 2616 which should not be empty
+                    path = p_parsed.path or "/"
+                    if p_parsed.query:
+                        path += f"?{p_parsed.query}"
+                    A1 = f"{self.username}:{realm}:{self.password}"
+                    A2 = f"{method}:{path}"
+                    HA1 = hash_utf8(A1)
+                    HA2 = hash_utf8(A2)
+                    if nonce == self._thread_local.last_nonce:
+                        self._thread_local.nonce_count += 1
+                    else:
+                        self._thread_local.nonce_count = 1
+                    ncvalue = f"{self._thread_local.nonce_count:08x}"
+                    s = str(self._thread_local.nonce_count).encode("utf-8")
+                    s += nonce.encode("utf-8")
+                    s += time.ctime().encode("utf-8")

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: Python upgrade readiness changes by removing requirements.txt #148

fix: Python upgrade readiness changes by removing requirements.txt #148

Uh oh!

munna-shaik-s1 commented Dec 4, 2024

Uh oh!

Check failure

Copilot Autofix

Check failure

Copilot Autofix

Check failure

Copilot Autofix

Check failure

Copilot Autofix

Check failure

Copilot Autofix

Check failure

Copilot Autofix

Check failure

Copilot Autofix

Check failure

Copilot Autofix

Check failure

Copilot Autofix

Check failure

Copilot Autofix

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

@@ -2 +2,3 @@
             # we use 3.8 for development
+            argon2-cffi==23.1.0

@@ -848,4 +848,5 @@
+                            sanitized_proxy_server = "{}:{}".format(proxy_addr, "****")
                             msg = "Error connecting to {} proxy {}".format(
-                                printable_type, proxy_server
+                                printable_type, sanitized_proxy_server
                             )

@@ -65,3 +65,3 @@
                     except Exception:
-                        log.logger.error(msg_temp, splunkd_uri, "unknown", format_exc())
+                        log.logger.error(msg_temp, sanitize_uri(splunkd_uri), "unknown", format_exc())
                     else:
@@ -70,3 +70,3 @@
                                 log.logger.debug(
-                                    msg_temp, splunkd_uri, resp.status_code, code_to_msg(resp)
+                                    msg_temp, sanitize_uri(splunkd_uri), resp.status_code, code_to_msg(resp)
                                 )

@@ -78,5 +78,11 @@
+            def redact_sensitive_info(text: str) -> str:
+                # Redact sensitive information such as passwords
+                redacted_text = text.replace("password", "******")
+                return redacted_text
             def code_to_msg(response: requests.Response):
+                redacted_text = redact_sensitive_info(response.text)
                 code_msg_tbl = {
-: "Request error. reason={}".format(response.text),
+: "Request error. reason={}".format(redacted_text),
 : "Authentication failure, invalid access credentials.",
@@ -85,6 +91,6 @@
 : "Requested endpoint does not exist.",
-: "Invalid operation for this endpoint. reason={}".format(response.text),
-: "Unspecified internal server error. reason={}".format(response.text),
+: "Invalid operation for this endpoint. reason={}".format(redacted_text),
+: "Unspecified internal server error. reason={}".format(redacted_text),
 : "Feature is disabled in the configuration file. reason={}".format(
-                        response.text
+                        redacted_text
                     ),
@@ -92,2 +98,2 @@
-                return code_msg_tbl.get(response.status_code, response.text)
+                return code_msg_tbl.get(response.status_code, redacted_text)

@@ -5,2 +5,3 @@
             import os
+            import ssl
             import sys
@@ -176,3 +177,3 @@
                             "cert_reqs": self.verify_mode,
-                            "ssl_version": self.protocol,
+                            "ssl_version": ssl.PROTOCOL_TLSv1_2,
                             "server_side": server_side,

@@ -370,2 +370,5 @@
             ):
+                if ssl_context is None:
+                    ssl_context = ssl.create_default_context()
+                ssl_context.minimum_version = ssl.TLSVersion.TLSv1_2
                 """

fix: Python upgrade readiness changes by removing requirements.txt #148

fix: Python upgrade readiness changes by removing requirements.txt #148

Uh oh!

Conversation

munna-shaik-s1 commented Dec 4, 2024

🥅 Goal

🛠️ Solution

🏫 Testing

Uh oh!

Check failure

Uh oh!

Uh oh!

Copilot Autofix

Check failure

Uh oh!

Uh oh!

Copilot Autofix

Check failure

Uh oh!

Copilot Autofix

Check failure

Uh oh!

Copilot Autofix

Check failure

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot Autofix

Check failure

Uh oh!

Uh oh!

Copilot Autofix

Check failure

Uh oh!

Uh oh!

Copilot Autofix

Check failure

Copilot Autofix

Check failure

Uh oh!

Uh oh!

Copilot Autofix

Check failure

Uh oh!

Uh oh!

Copilot Autofix

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants