Feat/dynamic discovery #4
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Resolved conflicts in: - crates/bittensor/src/utils.rs: Kept feature branch's proper signature implementation - justfile: Kept feature branch's validator public key generation logic - scripts/executor/Dockerfile: Kept cleaner ENV declaration from feature branch - scripts/executor/build.sh: Kept comprehensive key handling from feature branch - scripts/validator/docker-compose.yml: Kept file from feature branch (was deleted in main)
- Add metadata generation tool for different bittensor networks (finney, local, test) - Create generate-metadata binary to fetch and store chain metadata - Update build.rs to use pre-generated metadata files - Clean up public API routes to use proper handler signatures - Remove unnecessary request field from route handlers - Add subxt dependency for runtime metadata generation This improves build times by pre-generating metadata and simplifies API route implementations.
- Add localnet configuration files for all services (executor, miner, validator, public-api) - Create comprehensive localnet deployment scripts with Docker Compose - Add wallet setup and subnet registration automation - Update Docker configurations for localnet environment - Add service testing and verification scripts - Include documentation for localnet deployment process
- Fix missing newlines at end of files - Update generate-metadata binary formatting - Clean up build.rs metadata path handling - Fix validator test module visibility
- Add localnet SSH keys to gitignore - Add localnet wallets directory to gitignore - Add localnet keys directory to gitignore
Replace println\! macro string concatenation with inline formatting for cleaner code.
Enhanced error context and logging across miner client, test discovery, tests, and verification components.
Remove unnecessary debug statements from miner prover discovery module and related test files to clean up codebase
Add script to fix NVIDIA Management Library (NVML) symlinks for proper GPU monitoring functionality
Add comprehensive validation script to check health and discovery mechanisms of deployed Basilica components (miner, validator, executor)
epappas
force-pushed
the
feat/dynamic_discovery
branch
from
58d07a0 to
3159a57
Compare
Add SSH session management with ephemeral keys, protocol buffer definitions for miner discovery, and enhanced miner communications for validator access to executors
epappas
force-pushed
the
feat/dynamic_discovery
branch
from
3159a57 to
185070d
Compare
- Add comprehensive SSH session management with expiration tracking - Implement automated key cleanup and session validation - Add GPU benchmarking improvements and unused variable fixes - Update miner and validator configurations for SSH automation - Add session metadata tracking in authorized_keys files - Implement discovery verification and scheduler enhancements
- Refactor string formatting to use modern format syntax - Add AutomaticVerificationConfig to validator configuration - Implement enhanced verification scheduler with discovery cycles - Add comprehensive verification workflow with step tracking - Improve SSH session management and cleanup procedures - Add detailed executor discovery and verification methods
- Update string formatting to use modern format syntax - Add comprehensive endpoint validation in verification engine - Improve IPv4/IPv6 address parsing and validation in discovery - Enhance miner client timeout and retry logic for better reliability - Fix scheduler cleanup by removing unused shutdown channel - Add debug logging for endpoint validation failures
- Add detailed [EVAL_FLOW] logging throughout verification workflow - Remove miner filtering and prioritization to evaluate ALL miners - Enhance SSH session management with detailed timing metrics - Add comprehensive error handling and status tracking - Remove unused automatic config parameter in MinerProver - Fix scheduler loop to continue execution properly
- Update string formatting to use modern format syntax - Add comprehensive SSH automation status and configuration methods - Create verification engine builder and automation components modules - Add SSH key manager builder for modular construction - Enhance SSH automation status display and debugging capabilities - Add dynamic discovery controller for better SSH session management
epappas
force-pushed
the
feat/dynamic_discovery
branch
from
5b0eb01 to
ea32b85
Compare
- Use modern string formatting with format\!("{var}") syntax
- Remove unnecessary port range validation in discovery
- Add comprehensive development deployment script with SSH automation
- Fix string interpolation in verification error messages
- Update SSH key manager path generation to use modern formatting
- Add advertised address configuration for server components - Implement advertised endpoint validation and health monitoring - Support address separation between internal binding and external endpoints - Enhance executor and miner configurations with advertised address support - Add comprehensive endpoint configuration validation - Improve client IP extraction for gRPC requests - Add URL validation for advertised endpoints - Implement endpoint health monitoring and accessibility checks - Update string formatting and validation across components
- Update string formatting to use interpolation syntax - Fix gRPC port determination logic for miner client - Improve error chain validation in SSH key manager tests - Use default gRPC port when no offset configured - Enhance error message accessibility validation
…egration - Add validator assignment configuration to miner config - Implement round-robin assignment strategy selection - Integrate SSH session orchestrator with executor connection manager - Update miner state initialization with SSH automation - Fix gRPC endpoint port handling in miner client - Update verification engine builder with SSH key manager - Add validator assignment strategy configuration - Enhance executor connection management with SSH automation - Fix test assertions for endpoint port handling
Feat/auto executor
- Add atomic SSH key deduplication with file locking in SimpleSshKeys - Implement idempotent key addition preventing duplicate entries - Switch to persistent SSH keys instead of ephemeral session keys - Add libc dependency for file locking operations - Improve error handling and logging throughout SSH components - Add validator assignment configuration with enabled flag - Enhance SSH session orchestrator with better debugging - Update gRPC client to use validator-provided SSH public keys
- Fix code formatting issues throughout codebase - Improve SSH key management with atomic operations - Add proper error handling and logging - Update executor identity validation - Enhance SSH session orchestration - Fix clippy warnings and formatting issues
- Add correct configuration files for all services - Create comprehensive deployment script with SSH management - Include health checks and log following capabilities - Support wallet synchronization across servers - Add SSH access setup between miner and executor
- Fix executor identity migration tests with proper struct initialization - Improve stake monitor tests with proper test configuration - Add validator discovery tests with network isolation - Fix SSH key manager builder error validation - Ensure test configurations don't require external dependencies
- Add back std::sync::Arc import needed for integration tests - Improve SSH key manager test error chain validation - Make error checking more robust for nested error sources
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.