Update to 1.5 🗺️ (#12)

* Add visualization option with python and javascript using the Cytoscape.js framework

* Add logic to callmapper.py that hosts an existing data.json file and hosts its content. Add README.md for CallMapper.

* Update callmapper to filter out processes without network activity and design. Update badges. Update README for callmapper. Update .gitignore.

* Change logic for adding APIlookups. Add change to design of nodes depending on the node if they're malicious or not as they're bigger and have the color red.

* Split API lookup classes to separate file. Add file for custom API lookup classes. Update index.html page. Update README.md

* Add more logical structuring of files for CallMapper. Update version and the respective badges. Update main README.md and CallMapper README.md

* Update README.mds

* Update badges and README.md

* Fix issue from resolving build warnings where an empty string was passed rather than null. Fix issue by correctly adding process name to started process. Update CallMapper. Add walkthrough gif of CallMapper. Refactor and clean code. Extend GitHub actions pipelines

* Attempt fix for GitHub actions job

* Replace video for new at later time

* Attempt fix GitHub Actions file

* Attempt fix to GitHub actions. Add gif for CallMapper.

* Attempt fix to GitHub actions. Im blaming the lack of sleep.

* Change logic to not retrieve the executable name when supplying a PID to listen to. This better manages protected processes as it's not always possible to retrieve their executbale path, causing issue

* Fix issue with Listen mode where the retrieved process name of the PID wasn't successfully added as a monitored process

* Update GitHub actions to include privileged and unprivileged executions

* Add "Buy me a coffe" for support". Update github actions since password for new low priv user doesnt match pw policy

* Fix github actions attempt - shorter password

* Attempt fix of github actions

Author: H4NM

.github/FUNDING.yml

@@ -0,0 +1 @@
+buy_me_a_coffee: H4NM

.github/workflows/build-and-run.yml

@@ -0,0 +1,92 @@
+name: Compile and run WhoYouCalling
+
+on:
+  push:
+    branches:
+      - main
+      - dev
+
+jobs:
+  build:
+    runs-on: windows-latest
+
+    steps:
+    - name: Checkout Code
+      uses: actions/checkout@v4
+
+    - name: Setup .NET
+      uses: actions/setup-dotnet@v4
+      with:
+        dotnet-version: '8.0'
+
+    - name: Restore dependencies
+      run: dotnet restore
+
+    - name: Publish application
+      run: dotnet publish -c Release -r win-x64 --self-contained false -o output
+
+    - name: Upload compiled binary
+      uses: actions/upload-artifact@v4
+      with:
+        name: compiled-binary
+        path: output
+
+  test-illuminate:
+    needs: build
+    runs-on: windows-latest
+
+    steps:
+      - name: Download compiled binary
+        uses: actions/download-artifact@v4
+        with:
+          name: compiled-binary
+          path: output
+
+      - name: Run with Illuminate mode
+        run: output\wyc.exe --illuminate --nopcap --timer 20 -d
+
+  test-executable-privileged:
+    needs: build
+    runs-on: windows-latest
+
+    steps:
+      - name: Download compiled binary
+        uses: actions/download-artifact@v4
+        with:
+          name: compiled-binary
+          path: output
+
+      - name: Run with Execute mode with cmd.exe as privileged user
+        run: |
+          output\wyc.exe --executable "C:\Windows\System32\cmd.exe" --arguments "/c whoami" --nopcap --killprocesses --timer 10 --privileged -d
+
+  test-executable-unprivileged:
+    needs: build
+    runs-on: windows-latest
+
+    steps:
+      - name: Download compiled binary
+        uses: actions/download-artifact@v4
+        with:
+          name: compiled-binary
+          path: output
+
+      - name: Run with Execute mode with cmd.exe as low privileged user
+        run: |
+          net user ga ETphon3H0me@_1 /add
+          output\wyc.exe --executable "C:\Windows\System32\cmd.exe" --user ga --password ETphon3H0me@_1 --arguments "/c whoami" --nopcap --killprocesses --timer 10 -d
+
+  test-pid:
+    needs: build
+    runs-on: windows-latest
+
+    steps:
+      - name: Download compiled binary
+        uses: actions/download-artifact@v4
+        with:
+          name: compiled-binary
+          path: output
+
+      - name: Run with Listen mode against System proces
+        run: output\wyc.exe --PID 4 --nopcap --timer 10 -d
+

.github/workflows/build.yml

@@ -6,7 +6,14 @@ Caps/
 TestApplication*
 args.txt
 requirements.txt
-*.py
+autobadge.py
+requesttest.py
+TestApplication.py
+
+Result.json
+Summary.txt
+data.json
+Events.txt
 *.spec
 *.sh
 build/

.gitignore

@@ -0,0 +1,107 @@
+# CallMapper
+CallMapper offers a network graph with analytics of looking up domains and IP addresses against APIs, or via static links to websites such as [ipinfo.io](https://ipinfo.io/), [whois.com](https://www.whois.com/), [abuseipdb.com](https://www.abuseipdb.com/) and [virustotal.com](https://www.virustotal.com/).
+
+## How it works:
+1. `callmapper.py` parses the JSON results file from WhoYouCalling and creates a `data.json` file in the same directory as the script. If the flag for API lookups is provided, the data in the `data.json` files are enriched with stored HTML.
+2. `callmapper.py` hosts a HTTP server in the same directory as the script at localhost port 8080 that serves the `data.json` and the `index.html` with other related resources (css, js and icon).
+3. You can now view the visualization via http://127.0.0.1:8080 in a web browser
+
+## Usage:
+
+**Visualize the output from WhoYouCalling**:
+```
+python callmapper.py -r ./Result.json
+```
+> **Note:** You can visualize an already existing data.json file by not providing a Result.json and if that data.json file exists in the same directory as callmapper.py.
+
+**Visualize the output from WhoYouCalling and enrich the data with API lookups**:
+```
+python callmapper.py --results-file ./Result.json --api-lookup
+```
+
+## Dependencies
+**CallMapper** has been tested and works with Python version 3.11 or later. The packages that are used: 
+- Visualization: 
+  - [Cytoscape.js](https://github.com/cytoscape/cytoscape)
+- API Lookups:
+  - [requests](https://pypi.org/project/requests/) (*Optional - if API lookup of IPs and domains is wanted.*)
+
+In order to run **CallMapper**, all you really need is Python.
+
+## Using API lookups
+When running `callmapper.py` with the flag `--api-lookup` or `-a` for short, you will be prompted to choose which processes with network activity you want to lookup. 
+Thereafter, you will be asked which API's you want to use to perform the lookups against. Both of the prompts accept an empty answer for selecting everything.
+
+The list of available API's can be found in `callmapper.py` in the variable `AVAILABLE_APIS`.
+`AVAILABLE_APIS` is a dict with the title of the API as a key, with two subkeys; `api_key` and `api`. 
+
+```python
+AVAILABLE_APIS = {
+    'VirusTotal': {
+        'api_key': '', 
+        'api': VirusTotal,
+    },
+    'AbuseIPDB': {
+        'api_key': '', 
+        'api': AbuseIPDB,
+    }
+}
+```
+
+The included APIs, `VirusTotal` and `AbuseIPDB`, both require an API key. Their defined class, found in `/lib/api_lookups.py`, specifiy if the API source requires an API key or not. The API key is added in their respective respective `api_key` field in `AVAILABLE_APIS`. 
+If the field is empty and the API source requires an API key, and you as a user specified you want to use that api during the prompt, it will simply be skipped.
+
+## Add you own API integration
+> **Note:** Only REST APIs are supported.
+
+To create your own API integration, there's a template in `/custom/custom_api_lookups.py`. 
+Any API integration must have the following structure:
+
+```python
+
+class MyCustomAPILookupClass(APILookup):
+
+    def __init__(self, api_source:str, api_key:str = ""):
+        super().__init__(api_source, api_key)
+        self.headers = {"x-api-key": self.api_key}
+        self.api_key_required = True
+        self.lookup_types = [LookupType.IP, LookupType.DOMAIN]
+        
+    def get_data(self, endpoint: str, lookup_type) -> dict:
+        url = f"https://my.own.api/api/v2/check?{endpoint}"
+        response = self.requests.get(url, headers=self.headers)
+        #...
+        json_response = response.json()
+    
+    def get_presentable_data_for_ip(self, returned_data: dict) -> Tuple[dict, bool]:
+        presentable_data: dict = {}
+        is_potentially_malicious: bool = False
+        #...
+        return presentable_data, is_potentially_malicious
+    
+    def get_presentable_data_for_domain(self, returned_data: dict) -> Tuple[dict, bool]:
+        presentable_data: dict = {}
+        is_potentially_malicious: bool = False
+        #.... 
+        return presentable_data, is_potentially_malicious
+```
+The function `__init__` is invoked when the object of the class is initiated. In there, you need to define:
+    1. If the API-key is required or not: `self.api_key_required = True`
+    2. Should you lookup IPs, domains or both: `self.lookup_types = [LookupType.IP, LookupType.DOMAIN]`
+You can also define the requests header if needed, e.g. `self.headers = {"x-api-key": self.api_key}`. Otherwise you can define it in `get_data`.
+
+The function `get_data` is the one conducting the actual HTTP REST API lookup. It will simply query the endpoint, using `self.requests` (yes, that's an object inherited requests). The reason behind assigning the library `requests` to an object variable was to ensure that CallMapper doesn't require the library `requests` to run - this also why there's no `requirements.txt` file here :-). The `get_data` function processes the request to the extent of validating if successful data was returned or not. Thereafter it's only returned as a JSON object. Worth noting is that `get_data` may have a different URL depending on the endpoint type, in which is needs to be able to process both types. It is possible to return, as of now, three different API error types. If `MAJOR_ERROR`, `QUOTA_EXCEEDED`, or `WRONG_CREDENTIALS` are returned, the remaining types of endpoints will be skipped. If any other type of error is returned, it will simply attempt to lookup the next endpoint.   
+
+```python
+class APIErrorType:
+    NO_RESULTS = "NO_RESULTS"
+    INVALID_FORMAT = "INVALID_FORMAT"
+    ERROR = "ERROR"
+    WRONG_CREDENTIALS = "WRONG_CREDENTIALS"
+    QUOTA_EXCEEDED = "QUOTA_EXCEEDED"
+    MAJOR_ERROR = "MAJOR_ERROR"
+```
+
+The function `get_presentable_data_for_ip` and `get_presentable_data_for_domain` simply takes the returned JSON object retrieves the fields that are of value and places them within a flat dict (not nested). The keys in the dict will be the titles presented in the visualization and the data with be the corresponding values. The functions will return the dict and a bool wether the retrieved data indicates that the endpoint may be malicious. If the bool variable is returned `True` (potentially malicious), the nodes take a red star shape in the network graph, clearly indicating that they're worth investigating. 
+
+When it's done and ready, import the custom API you have defined in `/custom/` (e.g. `from custom.MyCustomAPILookupClass import *`) in `callmapper.py`, then simply add it in the same fashion as `VirusTotal` and `AbuseIPDB` are in `AVAILABLE_APIS`. 

CallMapper/README.md

@@ -0,0 +1,95 @@
+
+import os
+import sys
+import argparse
+from pathlib import Path
+
+#=====================================
+#  CUSTOM LIBRARIES 
+#========  FUNCTIONS & CLASSES =======
+from lib.functions import *
+from lib.static import SCRIPT_BANNER
+from lib.output import * 
+
+#==========  API LOOKUPS  ============
+from lib.lookups import *
+#from custom.MyCustomAPILookupClass import *
+
+#=====================================
+#  CHANGABLE VARIABLES AND FUNCTIONS
+#=====================================
+HTTP_HOST_ADRESS:str = "127.0.0.1"
+HTTP_HOST_PORT:int = 8080
+AVAILABLE_APIS = {
+    'VirusTotal': {
+        'api_key': '', 
+        'api': VirusTotal,
+    },
+    'AbuseIPDB': {
+        'api_key': '', 
+        'api': AbuseIPDB,
+    }
+}
+
+
+#==================================================
+#  Dont touch anything below :-) 
+#==================================================
+
+def main() -> None:
+    SCRIPT_DIRECTORY: Path.parent = Path(__file__).parent
+    DATA_FILE: str = SCRIPT_DIRECTORY / "data.json"
+    
+    parser = argparse.ArgumentParser(description="A script demonstrating argparse with flags.")
+    parser.add_argument("-r", "--results-file", type=str, help="Results file")
+    parser.add_argument("-a", "--api-lookup", action="store_true", help="Lookup endpoints against defined APIs")
+    args = parser.parse_args()
+    print(SCRIPT_BANNER)
+    
+    if not file_exists_in_same_script_folder(SCRIPT_DIRECTORY, "index.html"):
+        ConsoleOutputPrint(msg=f"Unable to find index.html in the same directory as the script", print_type="fatal")
+        sys.exit(1)
+        
+    if not args.results_file:
+        if not file_exists_in_same_script_folder(SCRIPT_DIRECTORY, "data.json"):
+            ConsoleOutputPrint(msg=f"Unable to find data.json in the same directory as the script. Please supply a Results.json file or move data.json to the same path as the script", print_type="fatal")
+            sys.exit(1)
+        if not valid_data_file_exists(DATA_FILE):
+            ConsoleOutputPrint(msg=f"{DATA_FILE} has an invalid JSON structure", print_type="fatal")
+            sys.exit(1)
+    
+    if args.api_lookup and not requests_is_installed():
+        ConsoleOutputPrint(REQUESTS_LIBRARY_MISSING_MSG, print_type="fatal")
+        sys.exit(1)
+        
+    if args.results_file:
+        ConsoleOutputPrint(msg=f"Retrieving data from results file", print_type="info")
+        monitored_processes: list = get_results_file_data(args.results_file)
+        
+        if args.api_lookup:
+            unique_process_names: set = get_unique_process_names_with_external_network_activity(monitored_processes)
+            processes_to_lookup_with_network_activity: list = prompt_user_for_processes_to_lookup(unique_process_names)
+            endpoints: dict = get_unique_endpoints_to_lookup(monitored_processes, processes_to_lookup_with_network_activity)
+            apis_to_use: list = prompt_user_for_apis_to_use(AVAILABLE_APIS)
+            lookup_endpoints(AVAILABLE_APIS, endpoints, apis_to_use) 
+        
+        ConsoleOutputPrint(msg=f"Creating visualization data", print_type="info")
+        visualization_data = get_visualization_data(monitored_processes)
+        if os.path.isfile(DATA_FILE):
+            if prompt_user_for_overwrite_of_data_file():
+                ConsoleOutputPrint(msg=f"Overwriting existing data.json.", print_type="info")
+                output_visualization_data(DATA_FILE, visualization_data)
+            else:
+                ConsoleOutputPrint(msg=f"Keeping existing data.json", print_type="info")
+        else:
+            output_visualization_data(DATA_FILE, visualization_data)
+    else:
+        ConsoleOutputPrint(msg=f"Visualizing from existing results file", print_type="info")
+    ConsoleOutputPrint(msg=f"Hosting visualization via http://{HTTP_HOST_ADRESS}:{HTTP_HOST_PORT}", print_type="info")
+    try:
+        start_http_server(directory=SCRIPT_DIRECTORY, host=HTTP_HOST_ADRESS, port=HTTP_HOST_PORT)
+    except KeyboardInterrupt:
+        ConsoleOutputPrint(msg=f"Keyboard interuppt. Goodbye!", print_type="info")
+
+if __name__ == "__main__":
+    main()