feat: enhance repository search with relevance scoring and diversity

- Add relevance scoring system for search results (exact match: 100, prefix: 50, contains: 25, description: 5)
- Implement round-robin result interleaving for multi-repository diversity
- Add deduplication of packages within each repository (same name+version)
- Refactor search to apply limit at manager level for better efficiency
- Fix cache entry data access (packages -> data attribute)
- Improve search result quality and user experience

Author: alvagante

CHANGELOG.md

@@ -8,6 +8,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Added
+- **Repository Search Relevance Scoring**: Implemented intelligent search result ranking system
+  - New `_calculate_relevance_score()` method in UniversalRepositoryDownloader for scoring search results
+  - Exact name matches score 100, name prefix matches score 50, name contains query scores 25, description matches score 5
+  - Search results automatically sorted by relevance score (highest first)
+  - Improved search quality by prioritizing more relevant packages
+- **Repository Search Result Diversity**: Enhanced search to show results from multiple repositories
+  - Round-robin interleaving of results from different repositories for better diversity
+  - Configurable max results per repository (minimum 3) to ensure representation from multiple sources
+  - Prevents single repository from dominating search results
+  - Better user experience with varied package sources
+- **Repository Search Deduplication**: Added deduplication of packages within each repository
+  - Removes duplicate packages with same name+version (e.g., different architecture variants)
+  - Reduces noise in search results while maintaining unique packages
+  - Improved logging to show both total and unique package counts
 - **RPM Repository Parser**: Complete implementation of RPM package metadata parser with comprehensive format support
   - New enhanced RPM parser for parsing repomd.xml and primary.xml metadata
   - Support for standard repomd.xml format (Rocky, AlmaLinux, CentOS Stream)
@@ -218,6 +232,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Security Enhancements**: File size limits for provider YAML files to prevent DoS attacks
 
 ### Changed
+- **Repository Search Implementation**: Refactored search logic for better performance and accuracy
+  - Search now applies limit at manager level instead of CLI level for better efficiency
+  - Removed redundant limit application in CLI after manager already limits results
+  - Enhanced search to return pre-sorted, deduplicated, and limited results
+- **Repository Cache Access**: Fixed cache entry data access pattern
+  - Changed from `cache_entry.packages` to `cache_entry.data` for correct attribute access
+  - Ensures proper retrieval of cached package data
+  - Consistent with cache entry data model structure
 - **LLM Provider Manager**: Enhanced to support multiple instances of the same provider type
   - Provider initialization now extracts base type from configuration or name
   - Improved error messages showing both provider name and type

saigen/cli/repositories.py

@@ -311,13 +311,13 @@ async def _search_packages(
         async with manager:
             click.echo(f"Searching for '{query}'...")
 
-            # Search packages
+            # Search packages with limit for better diversity
             result = await manager.search_packages(
-                query=query, platform=platform, repository_names=None
+                query=query, platform=platform, repository_names=None, limit=limit
             )
 
-            # Apply limit
-            packages = result.packages[:limit] if limit else result.packages
+            # Packages are already limited by manager
+            packages = result.packages
 
             if output_format == "json":
                 data = {

saigen/repositories/cache.py

@@ -399,8 +399,8 @@ async def get_all_packages(self, include_expired: bool = False) -> List[Reposito
                 cache_key = meta_file.stem
                 cache_entry = await self.get(cache_key)
 
-                if cache_entry and cache_entry.packages:
-                    all_packages.extend(cache_entry.packages)
+                if cache_entry and cache_entry.data:
+                    all_packages.extend(cache_entry.data)
 
             except Exception as e:
                 # Log error but continue with other entries
@@ -443,8 +443,8 @@ async def get_packages_by_repository(self, repository_name: str) -> List[Reposit
                 cache_key = meta_file.stem
                 cache_entry = await self.get(cache_key)
 
-                if cache_entry and cache_entry.packages:
-                    packages.extend(cache_entry.packages)
+                if cache_entry and cache_entry.data:
+                    packages.extend(cache_entry.data)
 
             except Exception as e:
                 # Log error but continue with other entries

saigen/repositories/downloaders/universal.py

@@ -306,7 +306,7 @@ def _decompress_content(self, content: bytes, headers: Dict[str, str]) -> bytes:
         return content
 
     async def search_package(self, name: str) -> List[RepositoryPackage]:
-        """Search for specific package."""
+        """Search for specific package with relevance scoring."""
         search_url = self.endpoints.get("search")
 
         if search_url:
@@ -316,16 +316,17 @@ async def search_package(self, name: str) -> List[RepositoryPackage]:
                 url = search_url.replace("{query}", name).replace("{package}", name)
                 packages = await self._download_and_parse(session, url)
 
-                # Filter results to match search query
+                # Filter and score results
                 name_lower = name.lower()
-                matching_packages = []
+                scored_packages = []
                 for package in packages:
-                    if name_lower in package.name.lower() or (
-                        package.description and name_lower in package.description.lower()
-                    ):
-                        matching_packages.append(package)
+                    score = self._calculate_relevance_score(package, name_lower)
+                    if score > 0:
+                        scored_packages.append((score, package))
 
-                return matching_packages
+                # Sort by relevance score (highest first)
+                scored_packages.sort(key=lambda x: x[0], reverse=True)
+                return [pkg for _, pkg in scored_packages]
 
             except Exception as e:
                 logger.debug(f"Search endpoint failed for {name}: {e}")
@@ -336,20 +337,47 @@ async def search_package(self, name: str) -> List[RepositoryPackage]:
             all_packages = await self.download_package_list()
 
             name_lower = name.lower()
-            matching_packages = []
+            scored_packages = []
 
             for package in all_packages:
-                if name_lower in package.name.lower() or (
-                    package.description and name_lower in package.description.lower()
-                ):
-                    matching_packages.append(package)
+                score = self._calculate_relevance_score(package, name_lower)
+                if score > 0:
+                    scored_packages.append((score, package))
 
-            return matching_packages
+            # Sort by relevance score (highest first)
+            scored_packages.sort(key=lambda x: x[0], reverse=True)
+            return [pkg for _, pkg in scored_packages]
 
         except Exception as e:
             logger.error(f"Failed to search packages in {self.repository_info.name}: {e}")
             return []
 
+    def _calculate_relevance_score(self, package: RepositoryPackage, query: str) -> float:
+        """Calculate relevance score for search results.
+        
+        Scoring:
+        - Exact name match: 100
+        - Name starts with query: 50
+        - Name contains query: 25
+        - Description contains query: 5
+        """
+        score = 0.0
+        pkg_name_lower = package.name.lower()
+        
+        # Name matching (highest priority)
+        if pkg_name_lower == query:
+            score += 100
+        elif pkg_name_lower.startswith(query):
+            score += 50
+        elif query in pkg_name_lower:
+            score += 25
+        
+        # Description matching (lower priority)
+        if package.description and query in package.description.lower():
+            score += 5
+        
+        return score
+
     async def get_package_details(
         self, name: str, version: Optional[str] = None
     ) -> Optional[RepositoryPackage]:

saigen/repositories/universal_manager.py

@@ -420,24 +420,59 @@ async def search_packages(
             task = asyncio.create_task(downloader.search_package(query), name=f"search_{name}")
             tasks.append((name, task))
 
-        # Collect search results
+        # Collect search results by repository
+        results_by_repo = {}
         for name, task in tasks:
             try:
                 packages = await task
                 if packages:
-                    # Apply limit per repository if specified
-                    if limit:
-                        packages = packages[:limit]
-
-                    all_packages.extend(packages)
+                    # Deduplicate packages by name+version within each repository
+                    # This handles cases where repos return multiple arch variants
+                    seen = set()
+                    deduped = []
+                    for pkg in packages:
+                        key = (pkg.name, pkg.version)
+                        if key not in seen:
+                            seen.add(key)
+                            deduped.append(pkg)
+                    
+                    results_by_repo[name] = deduped
                     repository_sources.append(name)
-                    logger.debug(f"Found {len(packages)} matches in {name}")
+                    if len(deduped) < len(packages):
+                        logger.debug(f"Found {len(packages)} matches ({len(deduped)} unique) in {name}")
+                    else:
+                        logger.debug(f"Found {len(packages)} matches in {name}")
             except Exception as e:
                 logger.error(f"Search failed in {name}: {e}")
 
-        # Apply global limit if specified
-        if limit and len(all_packages) > limit:
-            all_packages = all_packages[:limit]
+        # Interleave results from different repositories for diversity
+        # This ensures we show results from multiple repos, not just the first one
+        if limit and results_by_repo:
+            # Calculate max results per repository to ensure diversity
+            num_repos = len(results_by_repo)
+            max_per_repo = max(3, limit // num_repos + 1)  # At least 3 per repo
+            
+            # Limit each repository's results
+            limited_results = {
+                name: pkgs[:max_per_repo] 
+                for name, pkgs in results_by_repo.items()
+            }
+            
+            # Round-robin through repositories
+            repo_iterators = {name: iter(pkgs) for name, pkgs in limited_results.items()}
+            while len(all_packages) < limit and repo_iterators:
+                for name in list(repo_iterators.keys()):
+                    try:
+                        pkg = next(repo_iterators[name])
+                        all_packages.append(pkg)
+                        if len(all_packages) >= limit:
+                            break
+                    except StopIteration:
+                        del repo_iterators[name]
+        else:
+            # No limit, just concatenate all results
+            for packages in results_by_repo.values():
+                all_packages.extend(packages)
 
         # Calculate search time
         search_time = (datetime.utcnow() - start_time).total_seconds()