final troubleshooting and debugging

Author: cutler-scott-newrelic

README.md

@@ -239,8 +239,8 @@ Keep in mind that when you submit your pull request, you'll need to sign the CLA
     - [ ] Multi-threading
     - [ ] Better context detection and false positive filtering (GitHound, machine learning)
     - [ ] Use Rusoto instead of s3-rust
-    - [ ] Add JIRA scanner
-    - [ ] Add file-system & archive scanner
+    - [x] Add JIRA scanner
+    - [x] Add file-system & archive scanner
     - [ ] Use Rust features to reduce compilation dependencies?
 
 - 1.2: Integration with larger scripts and UIs

scripts/README.md

@@ -25,7 +25,7 @@ sudo systemctl daemon-reload
 sudo systemctl enable ghe_secret_monitor
 ```
 
-You can then perform a one-time execution of secret_scanner with the command 
+You can then perform a one-time execution of secret_monitor with the command 
 `sudo service ghe_secret_monitor start` and examine the results in /var/log/messages
 
 ## jira_secret_monitor.py
@@ -34,7 +34,7 @@ This is a Python script, also re-written based on Douglas Day's work, that perfo
 1) It scans all JIRA tickets modified in the last 24 hours for secrets using gottingen_hog.
 2) Within those JIRA tickets it looks for GDrive links and scans those docs for secrets using anakmali_hog.
 It then collects the results and outputs them to New Relic Insights. You can use 
-the same installation method as above, substituting jira_secret_scanner for secret_scanner
+the same installation method as above, substituting jira_secret_monitor for secret_monitor
 in each step.
 
 ## gh_org_scanner.py
@@ -53,7 +53,7 @@ blacklist of words. It outputs the results as output_filtered.csv
 
 This is a simple script meant to retrieve the latest pypi package (provided through environment variables) and perform a
 Rusty Hog scan on the contents of the download. It will then post the results to Insights.  You can use 
-the same installation method as above, substituting pypi_secret_monitor for secret_scanner
+the same installation method as above, substituting pypi_secret_monitor for secret_monitor
 in each step.
 
 You will need to supply 4 environment variables to it:
@@ -66,7 +66,7 @@ DUROC_HOG_PATH - the path to the duroc hog binary (relative or absolute)
 
 Based on pypi_secret_monitor, this is a simple script meant to retrieve the latest rubygem package (provided through 
 environment variables) and perform a Rusty Hog scan on the contents of the download. It will then post the results to 
-Insights. You can use the same installation method as above, substituting rubygem_secret_monitor for secret_scanner
+Insights. You can use the same installation method as above, substituting rubygem_secret_monitor for secret_monitor
 in each step.
 
 You will need to supply 4 environment variables to it:

scripts/ghe_secret_monitor.py

@@ -17,7 +17,7 @@
 # sudo systemctl enable ghe_secret_monitor
 # ```
 #
-# You can then perform a one-time execution of secret_scanner with the command
+# You can then perform a one-time execution of secret_monitor with the command
 # `sudo service ghe_secret_monitor start` and examine the results in /var/log/messages
 
 from datetime import datetime, timedelta

scripts/jira_secret_monitor.py

@@ -73,7 +73,7 @@
     # hit the JIRA API to retrieve the comments for each issue
     url = f"{JIRA_URL}rest/api/2/issue/{issue['key']}/comment"
     r = requests.get(url, auth=(JIRA_USERNAME, JIRA_PASSWORD))
-    comments = r.json()['comments']
+    comments = r.json().get('comments', [])
     for comment in comments:
         # find any google doc links in the comment and add them to our list (links)
         matches = gdoc_re.findall(comment['body'])

scripts/jira_secret_monitor.timer

@@ -1,5 +1,5 @@
 [Unit]
-Description=Run secret_scanner every day at 11:30pm pacific
+Description=Run secret_monitor every day at 11:30pm pacific
 Requires=jira_secret_monitor.service
 
 [Timer]

src/bin/gottingen_hog.rs

@@ -145,11 +145,22 @@ fn run(arg_matches: &ArgMatches) -> Result<(), SimpleError> {
 
     let json_results = get_issue_json(client, auth_headers, &full_url);
 
-    let description = json_results
-        .get("fields").unwrap()
-        .get("description").unwrap()
-        .as_str().unwrap()
-        .as_bytes();
+    let fields = json_results.get("fields").unwrap();
+
+    let description = match fields.get("description") {
+        Some(d) => match d.as_str() {
+            Some(e) => e.as_bytes(),
+            None => {
+                info!("The JIRA ticket description was set to null!");
+                "".as_bytes()
+            }
+        }
+        None => {
+            info!("The JIRA ticket description was not present!");
+            "".as_bytes()
+        }
+    };
+
 
     // find secrets in issue body
     let mut secrets = get_findings(&secret_scanner, base_url, issue_id,  description, String::from("Issue Description"));