Skip to content

Latest commit

 

History

History
216 lines (170 loc) · 12.3 KB

BACKEND_MODULE_DOCUMENTATION.md

File metadata and controls

216 lines (170 loc) · 12.3 KB

AIris Security Backend Module Documentation

This is the detailed backend module reference for the current tracked backend/ tree.

1. Module Purpose

backend/ is the FastAPI runtime responsible for:

  • JWT authentication and user profile lookup
  • scanner orchestration and scanner output normalization
  • scan/job state persistence and live progress logging in MongoDB
  • ML inference integration via ml/src/inference.py
  • PDF report generation

2. Current Tracked Backend Structure

backend/
  .env.example
  .env.scanners
  .gitignore
  BACKEND_MODULE_DOCUMENTATION.md
  README.md
  requirements.txt
  server.py
  setup_backend.ps1

  app/
    __init__.py
    config.py

    api/
      __init__.py
      auth.py
      report.py

    core/
      config.py
      init_.py
      security.py

    db/
      __init__.py
      mongodb.py

    models/
      __init__.py
      report.py
      scan.py
      user.py

    routes/
      scan.py

    scanners/
      __init__.py
      base.py
      dirsearch_scanner.py
      nikto_scanner.py
      nmap_scanner.py
      sslscan_scanner.py

    schemas/
      __init__.py
      auth.py
      report.py
      scan.py

    services/
      __init__.py
      aggregator_service.py
      auth_service.py
      ml_service.py
      pdf_report.py
      scan_service.py
      scanner_orchestrator.py
      vulnerability_aggregator.py

  security_tools/
    dirsearch
    nikto
    sslscan
    tool_paths.txt

Notes:

  • security_tools/* folders are tracked bundled tool directories/pointers consumed by scanner discovery logic.
  • Runtime artifacts (__pycache__, local venvs, logs) are intentionally excluded.

3. Runtime Flow (Current)

  1. backend/server.py creates the FastAPI app, configures CORS/logging, and includes routers under /api.
  2. Startup/shutdown hooks open and close MongoDB using app/db/mongodb.py.
  3. POST /api/scan/start inserts scan metadata and queues run_all_scanners as a background task.
  4. Each scanner adapter returns normalized findings + tool-specific metadata + raw output.
  5. ML analysis is run from MLService.analyze_scan inside an executor thread.
  6. Scan document is finalized with status, findings, ML output, and report data.
  7. PDF endpoint generates binary output using PDFReportGenerator.

4. Detailed File-by-File Reference

4.1 Backend Root Files

File What It Contains Detailed Behavior
backend/.env.example Base environment template Defines default env keys for API metadata, DB URL/name, auth key/algorithm, scanner and ML settings.
backend/.env.scanners Scanner-focused env template Documents scanner path and tuning env options used by scanner adapters when present.
backend/.gitignore Ignore policy Excludes local env/runtime files in backend scope (env files, caches, temp artifacts).
backend/BACKEND_MODULE_DOCUMENTATION.md Backend module knowledge base This document.
backend/README.md Operator guide Installation, scanner setup, endpoint overview, troubleshooting.
backend/requirements.txt Python dependency manifest Includes FastAPI stack, Motor, auth libs, ML libs, PDF/charting libs, and dev/test packages.
backend/server.py FastAPI entrypoint Sets Windows Proactor loop policy, initializes app, wires startup/shutdown DB lifecycle, serves /api and /api/health, mounts auth/scan/report routers.
backend/setup_backend.ps1 Backend setup script Automates environment bootstrap for Windows backend dev/runtime setup.

4.2 App Package Root

File What It Contains Detailed Behavior
backend/app/__init__.py Package marker Marks app as a Python package for imports.
backend/app/config.py Legacy config constants Loads dotenv and exposes simple constants (MONGO_URI, DB_NAME, JWT_SECRET, ALGORITHM, expiry); superseded by app/core/config.py in active app wiring.

4.3 API Layer (backend/app/api)

File What It Contains Detailed Behavior
backend/app/api/__init__.py Package marker Marks API package.
backend/app/api/auth.py Auth router POST /auth/register, POST /auth/login, GET /auth/me; uses AuthService; maps ValueError to HTTP 4xx and unexpected errors to 500.
backend/app/api/report.py Report router GET /report/{scan_id}; requires auth via get_current_user; resolves report through MLService.get_report(scan_id, user_id).

Implementation note:

  • /auth/me in backend/app/api/auth.py currently has unusual dependency wiring (Depends(get_database) assigned to current_user) and then manually calls get_current_user().

4.4 Core Layer (backend/app/core)

File What It Contains Detailed Behavior
backend/app/core/config.py Pydantic settings object Defines Settings(BaseSettings) with API metadata, auth configuration, DB config (MONGO_URL, DB_NAME), CORS, and exports singleton settings.
backend/app/core/init_.py Stray marker file Named init_.py (single underscore); not used as package initializer.
backend/app/core/security.py Auth/security primitives Password hash/verify (passlib), JWT issue/decode (jose), and FastAPI bearer dependency get_current_user returning user_id/email.

4.5 Database Layer (backend/app/db)

File What It Contains Detailed Behavior
backend/app/db/__init__.py Package marker Marks DB package.
backend/app/db/mongodb.py Mongo connection manager Maintains global db_instance (client, db), provides connect_to_mongo(), close_mongo_connection(), and dependency get_database().

4.6 Domain Models (backend/app/models)

File What It Contains Detailed Behavior
backend/app/models/__init__.py Package marker Marks models package.
backend/app/models/user.py User Pydantic models Defines User and UserInDB with user_id, email, username, hashed_password, active status, timestamps.
backend/app/models/scan.py Scan Pydantic model + enum Defines ScanStatus (pending/running/completed/failed) and Scan structure with target, lifecycle times, scanner/aggregate outputs.
backend/app/models/report.py Report Pydantic model Defines report data model with scan/user IDs, attack types, risk/confidence, vulnerability list, recommendations, summary.

4.7 Schemas (backend/app/schemas)

File What It Contains Detailed Behavior
backend/app/schemas/__init__.py Package marker Marks schemas package.
backend/app/schemas/auth.py Auth API contracts Request models: UserRegister, UserLogin; response models: Token, UserResponse.
backend/app/schemas/scan.py Scan API contracts ScanCreate with target validator, plus status/results response models (ScanResponse, ScanStatusResponse, ScanResultsResponse).
backend/app/schemas/report.py Report API contracts ReportResponse and VulnerabilitySummary for typed report payload structure.

4.8 Route Layer (backend/app/routes)

File What It Contains Detailed Behavior
backend/app/routes/scan.py Active orchestration endpoints + background worker Exposes GET /scan/health, GET /scan/scanners, GET /scan/history, POST /scan/start, GET /scan/status/{scan_id}, GET /scan/results/{scan_id}, GET /scan/report/{scan_id}/pdf; background run_all_scanners handles progress updates, log streaming, scanner execution, ML analysis, and final persistence.

4.9 Scanner Layer (backend/app/scanners)

File What It Contains Detailed Behavior
backend/app/scanners/__init__.py Scanner registry bootstrap Registers known scanner classes, instantiates for availability checks, builds AVAILABLE_SCANNERS, exports get_scanner, get_available_scanners, get_scanner_info.
backend/app/scanners/base.py Scanner interface BaseScannerInterface with async scan(target) contract and tool name handling.
backend/app/scanners/nmap_scanner.py Nmap adapter Resolves Nmap binary (tool_paths.txt/common paths/PATH), runs nmap -sV -T4 --top-ports 100 -oX -, parses XML + fallback text, applies canonical service mapping and severity heuristics, emits normalized findings and formatted raw output.
backend/app/scanners/nikto_scanner.py Nikto adapter Resolves Nikto + Perl paths, enforces HTTP scheme, runs Nikto with bounded tuning/time, parses line-based findings with dedupe and type/severity mapping, generates recommendations and formatted raw output.
backend/app/scanners/sslscan_scanner.py SSL/TLS adapter Hybrid scanner: optional testssl.sh execution plus Python-native TLS probe; detects deprecated protocols, weak ciphers, cert/self-signed/expiry states; converts into standardized findings and summary.
backend/app/scanners/dirsearch_scanner.py DirSearch adapter Resolves script/module path, executes plain-output scan, parses multiple output formats, classifies sensitive paths, maps severity + recommendation, returns exposed_paths, findings, summary, raw output.

4.10 Service Layer (backend/app/services)

File What It Contains Detailed Behavior
backend/app/services/__init__.py Package marker Marks services package.
backend/app/services/auth_service.py Auth business logic register_user, login_user, get_user_by_id; performs uniqueness checks, bcrypt hashing/verification, JWT issue, sanitized user responses.
backend/app/services/scan_service.py Alternate scan lifecycle service Creates scans and launches async scanner execution via asyncio.gather; updates scan status transitions and exposes status/results/history retrieval helpers.
backend/app/services/scanner_orchestrator.py Legacy orchestrator Async orchestrator currently wired for Nmap + Nikto only; SSLScan/DirSearch sections are intentionally commented.
backend/app/services/aggregator_service.py Generic result aggregator Normalizes multi-scanner findings, computes severity distribution, groups findings by type, and emits scanner execution summary.
backend/app/services/vulnerability_aggregator.py Compatibility aggregator Converts Nmap/Nikto result shapes (new and legacy formats) into simplified vulnerability records and CVSS estimates; SSL/DirSearch logic is commented out.
backend/app/services/ml_service.py ML bridge + report service Imports advanced predictor from ml/src/inference.py (from src.inference import predict_attack), exposes hybrid/rule-based analysis paths, legacy RF helpers, report creation (_create_report) and retrieval (get_report), and currently used analyze_scan(findings, scanner_results) wrapper.
backend/app/services/pdf_report.py PDF rendering engine ReportLab/matplotlib report generation, large REMEDIATION_MAP and _lookup_remediation, risk/severity chart embedding, ML analysis + finding tables + raw output sections.

4.11 Bundled Scanner Tool Paths (backend/security_tools)

File What It Contains Detailed Behavior
backend/security_tools/dirsearch Bundled DirSearch folder/pointer Referenced by DirSearchScanner path resolution fallback.
backend/security_tools/nikto Bundled Nikto folder/pointer Referenced by NiktoScanner path resolution fallback.
backend/security_tools/sslscan Bundled SSL scanner folder/pointer Referenced by SSL scanner discovery/configuration fallback.
backend/security_tools/tool_paths.txt Scanner path override file Optional operator-defined explicit locations for Nmap/Nikto/Perl/DirSearch/SSL tools.

5. Data Model in MongoDB

Primary collections used by current code paths:

  • users: account identity and authentication state fields.
  • scans: scan lifecycle, logs, findings, raw output, scanner details, ML summary.
  • reports: generated report documents keyed by scan_id and user_id.

6. Known Quirks and Maintenance Notes

  • backend/app/config.py and backend/app/core/config.py coexist; active server wiring uses backend/app/core/config.py.
  • backend/app/core/init_.py looks accidental (init_.py vs __init__.py).
  • /auth/me handler wiring in backend/app/api/auth.py is non-standard and should be cleaned to pure dependency injection.
  • Multiple orchestration/aggregation variants coexist; route-level execution in backend/app/routes/scan.py is currently the active API path.