Add multi-ecosystem support

Author: rotemreiss

README.md

@@ -13,6 +13,9 @@ A security tool that detects malicious packages from external vulnerability feed
 ## Table of Contents
 
 - [Features](#️-features)
+- [Ecosystem Support](#️-ecosystem-support)
+  - [Blocking Pattern Examples](#blocking-pattern-examples)
+  - [Multi-Ecosystem Usage](#multi-ecosystem-usage)
 - [Package Blocking & Security](#-package-blocking--security)
   - [How Exclusion Patterns Work](#how-exclusion-patterns-work)
   - [Blocking Commands](#blocking-commands)
@@ -44,6 +47,7 @@ A security tool that detects malicious packages from external vulnerability feed
 
 ## 🛡️ Features
 
+- **Multi-Ecosystem Support**: Supports scanning and blocking across 10 major package ecosystems
 - **OSV Feed Integration**: Fetches malicious package data from Google Cloud Storage OSV vulnerability database
 - **JFrog Artifactory Search**: Searches for packages in your Artifactory repositories using AQL (Artifactory Query Language)
 - **Security Cross-Reference**: Compares OSV malicious packages against your JFrog repositories to identify potential threats
@@ -54,6 +58,63 @@ A security tool that detects malicious packages from external vulnerability feed
 - **Rich CLI Interface**: Interactive command-line interface with progress bars and formatted output
 - **Comprehensive Health Checks**: Validates connectivity to OSV and JFrog services
 
+## 📦 Ecosystem Support
+
+Malifiscan supports 10 major package ecosystems with varying levels of OSV scanning, JFrog searching, and blocking capabilities:
+
+| Ecosystem   | OSV Scanning | JFrog Scanning | JFrog Blocking | Notes |
+|-------------|--------------|----------------|----------------|-------|
+| **npm**     | ✅ Full      | ✅ Full        | ✅ Full        | Complete support with scoped packages |
+| **PyPI**    | ✅ Full      | ✅ Full        | ✅ Full        | Complete support with normalized names |
+| **Maven**   | ✅ Full      | ✅ Full        | ✅ Full        | Complete GAV (GroupId:ArtifactId:Version) support |
+| **Go**      | ✅ Full      | ✅ Full        | ✅ Full        | Complete module path and version support |
+| **NuGet**   | ✅ Full      | ✅ Full        | ✅ Full        | Complete package ID and version support |
+| **RubyGems** | ✅ Full     | ✅ Full        | ✅ Basic       | Standard gem file patterns |
+| **crates.io** | ✅ Full    | ✅ Full        | ✅ Basic       | Standard crate file patterns |
+| **Packagist** | ✅ Full    | ✅ Full        | ✅ Basic       | Composer vendor/package patterns |
+| **Pub**     | ✅ Full      | ✅ Full        | ⚠️ Limited     | Generic patterns only (Dart/Flutter) |
+| **Hex**     | ✅ Full      | ✅ Full        | ⚠️ Limited     | Generic patterns only (Elixir) |
+
+**Legend:**
+- **OSV Scanning**: Fetches malicious package data from OSV vulnerability database
+- **JFrog Scanning**: Searches for packages in JFrog Artifactory repositories using AQL
+- **JFrog Blocking**: Creates exclusion patterns to block package downloads
+  - **✅ Full**: Complete blocking support with ecosystem-specific patterns
+  - **✅ Basic**: Good support with standard file patterns  
+  - **⚠️ Limited**: Basic patterns with limited blocking effectiveness
+
+### Blocking Pattern Examples
+
+Different ecosystem support levels create different types of exclusion patterns when blocking packages:
+
+**✅ Full Support (Precise Patterns)**
+- **npm**: `axios/-/axios-1.12.2.tgz` (targets exact tarball file)
+- **PyPI**: `simple/requests/requests-2.28.1*` (follows PyPI simple API structure)  
+- **Maven**: `com/example/evil-lib/1.0.0/**` (follows GAV structure: GroupId/ArtifactId/Version)
+- **Go**: `github.com/user/module/@v/v1.2.3*` (follows Go module proxy structure)
+- **NuGet**: `packagename/1.0.0/**` (follows NuGet repository layout)
+
+**✅ Basic Support (Standard Patterns)**
+- **RubyGems**: `gems/package-name-1.0.0.gem` (standard gem file format)
+- **crates.io**: `crates/package-name/package-name-1.0.0.crate` (standard crate format)
+- **Packagist**: `vendor/package/1.0.0/**` (Composer vendor/package structure)
+
+**⚠️ Limited Support (Generic Patterns)**
+- **Pub/Hex**: `**/package-name-1.0.0*` or `**/package-name/**` (broad wildcards)
+
+The blocking effectiveness decreases from **Full** (surgical precision) to **Limited** (broad patterns that might block more than intended or miss some variations).
+
+### Multi-Ecosystem Usage
+
+```bash
+# Scan all available ecosystems (default behavior)
+uv run python cli.py scan crossref
+
+# Scan specific ecosystem only
+uv run python cli.py scan crossref --ecosystem npm
+uv run python cli.py scan crossref --ecosystem PyPI
+```
+
 ## 🚫 Package Blocking & Security
 
 Malifiscan can automatically block malicious packages in your JFrog Artifactory repositories using **exclusion patterns**. This prevents developers from downloading compromised packages while preserving existing patterns.
@@ -218,9 +279,6 @@ uv run python cli.py registry list-blocked npm
 uv run python cli.py scan crossref
 uv run python cli.py scan crossref --hours 24
 
-# Test security scan
-uv run python cli.py scan test
-
 # Interactive mode
 uv run python cli.py interactive
 ```
@@ -262,12 +320,6 @@ python cli.py scan crossref --hours 6 --ecosystem npm --limit 100
 ```
 Fetches malicious packages from OSV (last 6 hours by default) and searches for them in your JFrog repositories.
 
-**Test Security Scan**
-```bash
-python cli.py scan test
-```
-Runs a test scan with known packages to validate the system.
-
 #### Using Docker
 
 Replace `python cli.py` with `docker run --env-file .env rotemreiss/malifiscan python cli.py` for any command. Add `-v $(pwd)/data:/app/data` for persistent storage.
@@ -508,4 +560,4 @@ This project is licensed under the MIT License - see the LICENSE file for detail
 
 ---
 
-**⚠️ Note**: This tool is for security assessment purposes. Always validate results before taking action on package repositories.
+**⚠️ Note**: This tool is provided as-is for security assessment purposes - users are responsible for testing and validating all results before taking any action, and the author assumes no responsibility for issues arising from its use. 🤷

cli.py

@@ -271,12 +271,17 @@ async def registry_search(self, package_name: str, ecosystem: str = "npm") -> bo
                 await registry.close()
             return False
 
-    async def security_crossref(self, hours: int = 6, ecosystem: str = "npm", limit: Optional[int] = None, no_report: bool = False, block: bool = False, no_notifications: bool = False) -> bool:
+    async def security_crossref(self, hours: int = 6, ecosystem: Optional[str] = None, limit: Optional[int] = None, no_report: bool = False, block: bool = False, no_notifications: bool = False) -> bool:
         """Cross-reference malicious packages from feed with package registry."""
         try:
             self.console.print(f"🔍 Security Cross-Reference Analysis", style="bold cyan")
             self.console.print(f"📅 Looking for malicious packages from the last {hours} hours")
-            self.console.print(f"🏗️ Ecosystem: {ecosystem}")
+            
+            if ecosystem:
+                self.console.print(f"🏗️ Ecosystem: {ecosystem}")
+            else:
+                self.console.print(f"🏗️ Ecosystems: All available (registry-first optimization)")
+                
             if block:
                 self.console.print(f"🚫 Block mode: Will proactively block malicious packages", style="bold red")
             self.console.print()
@@ -310,10 +315,20 @@ async def security_crossref(self, hours: int = 6, ecosystem: str = "npm", limit:
                 malicious_packages = fetch_result["packages"]
 
             if not malicious_packages:
-                self.console.print(f"✅ No malicious {ecosystem} packages found in the last {hours} hours", style="green")
+                ecosystem_desc = ecosystem if ecosystem else "packages across all ecosystems"
+                self.console.print(f"✅ No malicious {ecosystem_desc} found in the last {hours} hours", style="green")
                 return True
 
-            self.console.print(f"📦 Found {len(malicious_packages)} malicious {ecosystem} packages from feed", style="green")
+            # Create ecosystem description for display
+            ecosystem_desc = ecosystem if ecosystem else ""
+            self.console.print(f"📦 Found {len(malicious_packages)} malicious packages from feed{' (' + ecosystem + ')' if ecosystem else ''}", style="green")
+            
+            # Show ecosystem breakdown if available and not filtering by single ecosystem
+            if not ecosystem and fetch_result.get("ecosystems"):
+                ecosystems = fetch_result["ecosystems"]
+                for eco, count in ecosystems.items():
+                    self.console.print(f"  • {eco}: {count} packages", style="blue")
+                self.console.print()  # Add blank line for better readability
 
             # Step 2: Block packages (if selected)
             if block:
@@ -392,6 +407,14 @@ async def security_crossref(self, hours: int = 6, ecosystem: str = "npm", limit:
             self.console.print("🛡️ SECURITY ANALYSIS RESULTS", style="bold cyan")
             self.console.print("="*80, style="bold")
 
+            # Display ecosystem information
+            ecosystems_scanned = analysis_result.get("ecosystems_scanned", [])
+            if ecosystems_scanned:
+                self.console.print(f"🏗️ Ecosystems scanned: {', '.join(ecosystems_scanned)}", style="blue")
+                if len(ecosystems_scanned) > 1:
+                    self.console.print(f"📊 Multi-ecosystem analysis with registry-first optimization", style="blue")
+                self.console.print()
+            
             # Get registry name and dynamic field names
             registry_name = await self._get_registry_name()
             field_names = await self._get_dynamic_field_names()
@@ -1190,7 +1213,7 @@ async def main():
 
     crossref_parser = scan_subparsers.add_parser("crossref", help="Cross-reference malicious packages from feed with package registry")
     crossref_parser.add_argument("--hours", type=int, default=6, help="Hours ago to look for recent malicious packages (default: 6)")
-    crossref_parser.add_argument("--ecosystem", default="npm", help="Package ecosystem (default: npm)")
+    crossref_parser.add_argument("--ecosystem", help="Package ecosystem (default: all available ecosystems)")
     crossref_parser.add_argument("--limit", type=int, help="Maximum number of malicious packages to check (default: no limit)")
     crossref_parser.add_argument("--no-report", action="store_true", help="Skip saving scan report to storage")
     crossref_parser.add_argument("--block", action="store_true", help="Block malicious packages from OSV feed before searching (default: false)")

src/core/entities/scan_result.py

@@ -28,6 +28,7 @@ class ScanResult:
     malicious_packages_list: List[MaliciousPackage]  # Previously packages_already_present
     errors: List[str]
     execution_duration_seconds: float
+    ecosystems_scanned: Optional[List[str]] = None  # Multi-ecosystem support
 
     @property
     def has_new_threats(self) -> bool:

src/core/interfaces/packages_feed.py

@@ -10,13 +10,14 @@ class PackagesFeed(ABC):
     """Abstract interface for malicious packages feed providers."""
 
     @abstractmethod
-    async def fetch_malicious_packages(self, max_packages: Optional[int] = None, hours: Optional[int] = None) -> List[MaliciousPackage]:
+    async def fetch_malicious_packages(self, max_packages: Optional[int] = None, hours: Optional[int] = None, ecosystems: Optional[List[str]] = None) -> List[MaliciousPackage]:
         """
         Fetch list of malicious packages from the feed.
         
         Args:
             max_packages: Maximum number of packages to fetch (None for all)
             hours: Fetch packages modified within the last N hours (None for all time)
+            ecosystems: List of ecosystems to fetch (None for all available ecosystems)
         
         Returns:
             List of MaliciousPackage entities
@@ -26,6 +27,19 @@ async def fetch_malicious_packages(self, max_packages: Optional[int] = None, hou
         """
         pass
 
+    @abstractmethod
+    async def get_available_ecosystems(self) -> List[str]:
+        """
+        Get list of available ecosystems in the feed.
+        
+        Returns:
+            List of ecosystem names available in the feed
+            
+        Raises:
+            FeedError: If the feed cannot be accessed
+        """
+        pass
+    
     @abstractmethod
     async def health_check(self) -> bool:
         """