Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
ef33e7a
trivy CLI starters
cjllanwarne May 15, 2025
3e30f4d
diff working
cjllanwarne May 15, 2025
56cf4c2
Reopened findings too
cjllanwarne May 15, 2025
1610ad0
diff to json output
cjllanwarne May 15, 2025
558b222
merging working
cjllanwarne May 16, 2025
3a07e57
cli groups
cjllanwarne May 16, 2025
3313a1c
zap alerts to findings
cjllanwarne May 17, 2025
41da6c9
zap diff working
cjllanwarne May 17, 2025
4b3653a
first two CIS tools
cjllanwarne May 19, 2025
0718f59
CIS diff
cjllanwarne May 20, 2025
524d123
updated single diff apply
cjllanwarne May 20, 2025
8ec0c52
group CIS poams correctly
cjllanwarne May 20, 2025
ebce0a5
fix groupings
cjllanwarne May 20, 2025
0fc5263
fix CIS fields
cjllanwarne May 20, 2025
4554cd0
update zap input format
cjllanwarne May 20, 2025
7109062
progress
cjllanwarne May 27, 2025
34168b3
remove anything after connected sheet too
cjllanwarne May 27, 2025
14a4645
exclude info findings from poam diffs
cjllanwarne Sep 25, 2025
b55ed18
make param order consistent across scan types
cjllanwarne Sep 25, 2025
11dac04
merge diffs when applying
cjllanwarne Sep 25, 2025
09031b7
diff tests
cjllanwarne Sep 25, 2025
15ee743
Use WORKING for trivy commands
cjllanwarne Oct 14, 2025
ff386cd
poams weekly-update process
cjllanwarne Oct 14, 2025
1009358
Complete weekly update process command
cjllanwarne Oct 14, 2025
460c315
diff apply in weekly update
cjllanwarne Oct 14, 2025
a81ae10
claude.md
cjllanwarne Mar 5, 2026
2a29e77
better gitignore
cjllanwarne Mar 5, 2026
76c8838
fix missing trivy severity
cjllanwarne Mar 5, 2026
c14ee12
remove old cruft
cjllanwarne Mar 5, 2026
dd4b47b
update docs
cjllanwarne Mar 5, 2026
c2504c9
feedback
cjllanwarne Mar 5, 2026
439e94e
helper function for review parsing
cjllanwarne Mar 5, 2026
7e095e5
pytest
cjllanwarne Mar 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: Tests

on:
pull_request:

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Install dependencies
run: pip install -r requirements.txt pytest

- name: Run tests
run: python -m pytest tests/ -q
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -175,5 +175,9 @@ cython_debug/
# PyPI configuration file
.pypirc

# macOS
.DS_Store

# Ignore any findings.db files
findings.db
findings.db
working/
68 changes: 68 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Overview

A security findings tracker that manages POA&M (Plan of Action and Milestones) lifecycle for compliance purposes. The CLI (`cli/cli.py`) is the sole entry point.

## Commands

### Setup
```bash
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```

### CLI Usage
```bash
./cli/cli.py --help
./cli/cli.py poams weekly-update # Interactive guided weekly update
./cli/cli.py poams apply-diff <poam.xlsx> <diff.json> [<diff2.json> ...]
./cli/cli.py poams merge-diffs <diff1.json> <diff2.json> -o merged.json
./cli/cli.py trivy download-alerts [-d <dest>]
./cli/cli.py trivy convert-alerts <alerts.json> [-o <output.csv>]
./cli/cli.py trivy alerts-diff <poam.xlsx> <alerts.csv>
./cli/cli.py zap alerts-to-findings <report.csv> [-o <output.json>]
./cli/cli.py zap alerts-diff <poam.xlsx> <findings.json>
./cli/cli.py cis split-connected-sheet <file.xlsx> [-o <output_dir>]
./cli/cli.py cis csv-to-findings <file.csv> [-o <output.json>]
./cli/cli.py cis alerts-diff <poam.xlsx> <findings.json>
```

### Tests
```bash
python -m pytest # Run all tests
python -m pytest tests/test_diff.py # Run a single test file
python -m pytest tests/test_diff.py::test_name # Run a single test
```

## Architecture

### Data Flow (weekly update)
1. **Download/collect** raw scan data: Trivy alerts from GitHub API, CIS Excel sheet, ZAP CSV report
2. **Convert** to normalized intermediate formats: `.findings.csv` (Trivy) or `.findings.json` (CIS, ZAP)
3. **Diff** each findings file against the existing POAM Excel → produces `.diff.json` files
4. **Apply** one or more diff JSONs to create an updated POAM Excel file

### Core Data Structures (`tools/`)
- `tools/findings.py` — `Finding` dataclass: source-agnostic normalized security finding
- `tools/poam.py` — `PoamEntry` dataclass + `PoamFile` class: reads POAM Excel files (headers on row 5 of "Open POA&M Items" / "Closed POA&M Items" sheets)
- `tools/diff.py` — `compare_findings_to_poams()` and `PoamFileDiff`: matches findings to POAMs by exact `weakness_name` + asset coverage; produces lists of new/existing/closed/reopened
- `tools/diff_apply.py` — `apply_diff()`: writes changes back to Excel using openpyxl

### Source-Specific Modules
Each scanner type (`trivy/`, `zap/`, `cis/`) has:
- `alerts.py` or `converter.py` — converts raw scanner output to `Finding` objects
- `diff.py` — calls `compare_findings_to_poams()` with the right POAM filter and generator
- `poam_generator.py` — generates `PoamEntry` objects with appropriate POAM IDs

POAM ID formats: Trivy → `YYYY-TRIVYXXXX`, CIS → `YYYY-CISXXXX`

### Working Directory Convention
By default, files are saved to `working/YYYY-MM-DD/`. The `WORKING` environment variable can override the base path.

### Authentication
- GitHub (for Trivy downloads): `gh auth login` or `GITHUB_TOKEN` env var
- Google services: `gcloud auth application-default login` or `GOOGLE_APPLICATION_CREDENTIALS` env var
83 changes: 55 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Security Findings Tracker

A Streamlit-based application for tracking and managing security findings from weekly CSV uploads. The application provides an interface for analyzing security findings, tracking their status, and managing their lifecycle.
A CLI tool for tracking and managing POA&M (Plan of Action and Milestones) lifecycle for compliance purposes. Processes weekly security findings from Trivy, ZAP, and CIS scans and applies them to a POAM Excel file.

## Requirements

Expand All @@ -9,12 +9,10 @@ A Streamlit-based application for tracking and managing security findings from w

## Features

- Upload and process weekly CSV security findings
- Track new, existing, and resolved findings
- Download and process Trivy, ZAP, and CIS findings
- Diff findings against existing POAMs to identify new, closed, and reopened items
- Automatic due date assignment based on severity
- Interactive data visualization
- Export capabilities for active and resolved findings
- Modern, responsive UI inspired by hail.is
- Apply diffs to update the POAM Excel file

## Installation

Expand All @@ -40,33 +38,62 @@ source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
```

## Usage
## Command Line Tools

The application includes command line tools for automation and data management. These tools are available through the `cli.py` script in the `cli` directory.

### Command Groups

The CLI is organized into the following command groups:

1. `poams` - Commands for working with POAMs:
```bash
# Interactive weekly update process
./cli/cli.py poams weekly-update

# Preview Trivy POAMs from an Excel file
./cli/cli.py poams preview-trivy <file_path> [--limit <n>]

# Apply diff changes to a POAM Excel file
./cli/cli.py poams apply-diff <poam_file> <diff_file>
```

2. `trivy` - Commands for working with Trivy:
```bash
# Download Trivy alerts from GitHub code scanning API
./cli/cli.py trivy download-alerts [--destination <file_path>]

# Convert GitHub Trivy alerts JSON to POAM CSV format
./cli/cli.py trivy convert-alerts <alerts_file> [--output <file_path>]

# Compare Trivy alerts against existing POAMs
./cli/cli.py trivy alerts-diff <poam_file> <alerts_csv>
```

To see all available commands and their options:
```bash
./cli/cli.py --help
```

1. Start the Streamlit application (make sure you're in the repository root directory):
For help on a specific command group:
```bash
# On Unix/macOS:
PYTHONPATH=$PYTHONPATH:. streamlit run app/main.py
./cli/cli.py poams --help
./cli/cli.py trivy --help
```

# On Windows (PowerShell):
$env:PYTHONPATH = "$env:PYTHONPATH;."
streamlit run app/main.py
### Authentication

# On Windows (Command Prompt):
set PYTHONPATH=%PYTHONPATH%;.
streamlit run app/main.py
```
The tools that interact with Google services use Application Default Credentials (ADC). To set up authentication:

2. Open your web browser and navigate to the URL shown in the terminal (typically http://localhost:8501)
1. Using gcloud (recommended for development):
```bash
gcloud auth application-default login
```

3. Use the application:
- Upload your weekly CSV file using the file uploader
- Set the analysis date
- View the summary statistics and visualizations
- Track recurrances and resolutions through additional scans and uploads.
- Export findings as needed
2. Or using a service account (recommended for production):
```bash
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account-key.json"
```

## Database
### Available Commands

The application uses SQLite for data storage. The database file is created at `app/database/findings.db` and includes tables for:
- Findings tracking
- Upload history
3 changes: 0 additions & 3 deletions app/__init__.py

This file was deleted.

29 changes: 0 additions & 29 deletions app/components/IssuesList.py

This file was deleted.

Loading