feat: Add comprehensive federated learning, LLM inference, and IPFS storage infrastructure (#16)

* Update client configuration

* feat: add llm command to CLI

* feat: enhance README and add LLM model listing command to CLI

* refactor: update configuration for Filecoin integration - Change Ethereum references to Filecoin in .env.sample and related CLI files - Adjust server settings and API endpoints accordingly

* feat: add federated learning commands and client - Implement federated learning session management commands in CLI - Create a new FederatedLearningClient for session operations - Add storage management commands for IPFS/Filecoin integration - Update configuration structure for Filecoin settings

* feat: update .env.sample for IPFS/Filecoin and Federated Learning configurations - Add new environment variables for IPFS storage settings and federated learning parameters

* feat: update README with detailed configuration for Filecoin and Federated Learning - Reorganize environment variables and clarify settings for Filecoin network and federated learning parameters

* feat: Standardize federated learning commands and add comprehensive API docs

- Standardized all FL command structures with consistent patterns
- Added GetTrainedModel client method for model retrieval
- Enhanced configuration with proper FederatedLearning section
- Fixed URL construction in federated learning client
- Added comprehensive API reference documentation
- Implemented consistent help text and examples across all FL commands
- Added proper flag validation and error handling

* refactor: remove emoji from print statements in federated learning, llm, and storage commands for consistency

* feat: add comprehensive federated learning CLI commands

- Implement complete FL session lifecycle management (create, start, get, list)
- Add federated learning commands with extensive configuration options
- Support for dataset upload with IPFS/Filecoin integration
- Add model update submission with gradients file support
- Implement trained model retrieval and export functionality
- Add differential privacy configuration options
- Support for custom model configuration files
- Add comprehensive parameter validation and error handling
- Include detailed CLI examples and usage documentation
- Integrate with wallet authentication system for FL operations

* feat: add comprehensive data partitioning for truly distributed federated learning

- Implement 5 partitioning strategies: random, stratified, sequential, non-IID (Dirichlet), and label skew
- Add PartitionConfig struct with configurable parameters (alpha, min_samples, overlap_ratio)
- Update TrainingDataInfo to include metadata for partition configuration
- Enhance CLI commands with partitioning flags (--alpha, --min-samples, --overlap-ratio)
- Add LoadPartitionedData method to support distributed data loading
- Create comprehensive test script for all partitioning strategies
- Add detailed documentation for data partitioning functionality

This enables truly distributed federated learning where each participant
trains on different subsets of data, simulating realistic FL scenarios
with data heterogeneity and privacy-preserving data separation.

* docs: add comprehensive documentation for data partitioning functionality

* docs: add comprehensive documentation for FL weights fix

- Document the root cause of empty weights issue
- Explain the technical solution across all three repositories
- Provide code examples showing before/after states
- Include testing instructions and benefits achieved
- Add technical notes for future reference

* docs: update README and federated learning commands for enhanced clarity and required parameters

- Revise README to include detailed federated learning features and usage instructions
- Add model configuration examples and clarify required parameters for session creation
- Update CLI commands to enforce required flags for aggregation method, learning rate, batch size, and local epochs
- Improve error handling for missing model configuration
- Enhance documentation for differential privacy and data partitioning strategies

* chore: update .env.sample and CLI commands to reflect USDFC token usage

- Revise .env.sample to include new server and blockchain identity configurations
- Update CLI commands to replace PRTY with USDFC for token balance and stake operations
- Add new configuration options for logging, caching, and security settings
- Introduce GetReputationCommand in command structure

* chore: clean up .env.sample by removing unnecessary comments

- Removed comments regarding the importance of the client and private key requirements
- Streamlined the configuration options for better clarity

* feat: implement gologger improvements and cleanup

- Add DisableTimestamp support to gologger
- Remove outdated documentation and test files
- Clean up repository structure
- Enhance logging configuration options

* fix: update gologger submodule to latest main commit

- Point gologger to commit 768e1ca (latest main)
- Ensure consistency across all repositories

* feat: update gologger with LOG_DISABLE_TIMESTAMP support

- Update gologger submodule to commit 2e729e3
- Add environment variable support for disabling timestamps
- Enable runtime control of timestamp display

* fix: update gologger with ConfigForMode environment variable support

- Update gologger submodule to commit 3a60352
- Fix LOG_DISABLE_TIMESTAMP being ignored when using InitWithMode
- Ensure environment variables are respected across all logging modes

* fix: update gologger to resolve infinite recursion

- Update gologger submodule to commit 032e8ba
- Fix stack overflow caused by recursive DefaultConfig/ConfigForMode calls
- Ensure LOG_DISABLE_TIMESTAMP works properly without crashes

* fix: update API endpoint paths in proxy and task handlers

- Modify target URL in proxy handler to include '/api/' prefix
- Update upload URL in task handler to use '/api/v1/tasks' path

* fix: update repository clone URL in README.md

* refactor: update configuration and service references from Filecoin to Blockchain

- Rename FilecoinNetwork to BlockchainNetwork in config and related files
- Update environment variables in .env.sample and README.md
- Refactor wallet adapter initialization to use BlockchainNetwork settings
- Modify storage service references to reflect the change from Filecoin to Blockchain
- Clean up related comments and documentation for clarity

* refactor: improve resource cleanup and logging in client and service files

- Update response body and file closure handling to log errors on close
- Remove unnecessary linter configurations in .golangci.yml
- Simplify lint command in Makefile
- Ensure proper error handling for port listener closure in network.go
- Enhance error handling for JSON body reading in http.go

* chore: update golangci-lint action version in workflow

- Upgrade golangci-lint-action from v3 to v4
- Specify golangci-lint version as v2.2.2 for consistency

* chore: simplify golangci-lint arguments in workflow

- Remove unnecessary output format argument from golangci-lint action configuration

* chore: remove timeout argument from golangci-lint action in workflow

* chore: update golangci-lint configuration and workflow

- Disable all linters by default and selectively enable necessary ones in .golangci.yml
- Update local-prefixes for goimports to reflect new repository structure
- Downgrade golangci-lint-action from v4 to v3 and update version to v1.54.2 in workflow
- Adjust golangci-lint arguments for improved output formatting and timeout settings

* chore: remove golangci-lint configuration file

* chore: update golangci-lint configuration to enable additional linters

- Add govet, ineffassign, and unused linters to .golangci.yml for improved code quality checks

* chore: update golangci-lint configuration to remove unnecessary linters and adjust local-prefixes

- Remove govet, ineffassign, and unused linters from .golangci.yml
- Update local-prefixes for goimports to reflect new repository structure

* chore: update Go version in code quality workflow to 1.22

* chore: update golangci-lint configuration and workflow

- Add new linter for handling internal import errors in .golangci-ci.yml
- Upgrade Go toolchain version to 1.24.5 in go.mod
- Update Go version to 1.23 and golangci-lint version to latest in code quality workflow

Author: virajbhartiya

.env.sample

@@ -1,15 +1,28 @@
-# Runner config
-RUNNER_WEBHOOK_PORT=
-RUNNER_API_PREFIX=
-RUNNER_SERVER_URL=
-
-# Server config
-SERVER_HOST=
-SERVER_PORT=
-SERVER_ENDPOINT=
-
-# Ethereum config
-ETHEREUM_TOKEN_ADDRESS=
-ETHEREUM_STAKE_WALLET_ADDRESS=
-ETHEREUM_CHAIN_ID=
-ETHEREUM_RPC=
+# PLGenesis Server Configuration
+SERVER_URL="http://localhost:8080"
+SERVER_HOST="localhost"
+SERVER_PORT=8080
+SERVER_ENDPOINT="/api/v1"
+
+# Blockchain Network Configuration
+BLOCKCHAIN_RPC="https://your-blockchain-node.com"
+BLOCKCHAIN_CHAIN_ID=1
+TOKEN_ADDRESS="0xYourTokenAddressHere"
+TOKEN_SYMBOL=PRTY
+STAKE_WALLET_ADDRESS="0xYourStakeWalletAddressHere"
+
+# Smart Contract Addresses
+REPUTATION_CONTRACT_ADDRESS="0x..."  
+
+# Blockchain Identity Configuration
+PRIVATE_KEY=""  
+
+# IPFS Storage Configuration
+IPFS_ENDPOINT="https://api.web3.storage"
+GATEWAY_URL="https://gateway.ipfs.io"
+
+# Task Configuration
+TASK_MAX_REWARD=100.0  # Token reward
+
+# Logging Configuration
+LOG_FILE_PATH="./logs/client.log"

.github/workflows/code-quality.yml

@@ -10,20 +10,24 @@ jobs:
   lint:
     name: Linting
     runs-on: ubuntu-latest
+    env:
+      GOTOOLCHAIN: local
     steps:
       - uses: actions/checkout@v3
         with:
           submodules: recursive
       - name: Set up Go
         uses: actions/setup-go@v4
         with:
-          go-version: "1.21"
+          go-version: "1.23"
           cache: true
       - name: Run golangci-lint
         uses: golangci/golangci-lint-action@v3
+        env:
+          GOTOOLCHAIN: local
         with:
           version: latest
-          args: --timeout=5m --out-format=colored-line-number
+          args: --timeout=5m --out-format=colored-line-number --config=.golangci-ci.yml
           skip-cache: true
           skip-pkg-cache: true
           skip-build-cache: true

.golangci-ci.yml

@@ -0,0 +1,42 @@
+linters:
+  disable-all: true
+  enable:
+    - govet
+    - ineffassign
+    - unused
+
+run:
+  timeout: 5m
+  concurrency: 4
+  tests: true
+
+issues:
+  exclude-dirs:
+    - vendor
+    - node_modules
+    - tmp
+  max-issues-per-linter: 0
+  max-same-issues: 0
+  exclude-use-default: false
+  exclude-rules:
+    - path: _test\.go
+      linters:
+        - errcheck
+    - text: "could not import sync/atomic"
+      linters:
+        - typecheck
+    - text: "unsupported version: 2"
+      linters:
+        - typecheck
+    - text: "could not load export data: internal error in importing"
+      linters:
+        - goanalysis_metalinter
+
+output:
+  sort-results: true
+
+linters-settings:
+  gofmt:
+    simplify: true
+  goimports:
+    local-prefixes: "github.com/theblitlabs/parity-runner"

.golangci.yml

@@ -1,52 +1,40 @@
 linters:
+  disable-all: true
   enable:
-    - errcheck
-    - gosimple
     - govet
     - ineffassign
-    - staticcheck
-    - typecheck
     - unused
   disable:
-    - exhaustruct # Can be noisy
-    - wsl # Whitespace linting can be too strict
-    - gofmt # Temporarily disabled
-    - goimports # Temporarily disabled
+    - errcheck
+    - staticcheck
 
 run:
   timeout: 5m
   concurrency: 4
-  issues-exit-code: 1
   tests: true
 
-# Default is 3 lines of context, which is often insufficient
-output:
-  sort-results: true
-
 issues:
   exclude-dirs:
     - vendor
     - node_modules
     - tmp
-  # Maximum issues count per one linter
   max-issues-per-linter: 0
-  # Maximum count of issues with the same text
   max-same-issues: 0
   exclude-use-default: false
   exclude-rules:
     - path: _test\.go
-      linters:
-        - errcheck # Allow unchecked errors in tests
-    - text: "Error return value of `.*(Close|Flush)` is not checked"
       linters:
         - errcheck
-    - text: "shadow: declaration of"
-      linters:
-        - govet
+
+# Default is 3 lines of context, which is often insufficient
+output:
+  sort-results: true
 
 # Settings for specific linters
 linters-settings:
   gofmt:
     simplify: true
   goimports:
-    local-prefixes: "github.com/theblitlabs/parity-protocol"
+    local-prefixes: "github.com/theblitlabs/parity-runner"
+
+version: "2"

Makefile

@@ -112,14 +112,14 @@ lint: ## Run linting with options (make lint VERBOSE=true CONFIG=custom.yml OUTP
 	@if [ -n "$(CONFIG)" ]; then \
 		FINAL_LINT_FLAGS="$(FINAL_LINT_FLAGS) --config=$(CONFIG)"; \
 	else \
-		FINAL_LINT_FLAGS="$(FINAL_LINT_FLAGS) --config=$(LINT_CONFIG)"; \
+		FINAL_LINT_FLAGS="$(FINAL_LINT_FLAGS)"; \
 	fi
 	@if [ -n "$(OUTPUT)" ]; then \
 		FINAL_LINT_FLAGS="$(FINAL_LINT_FLAGS) --out-format=$(OUTPUT)"; \
 	else \
 		FINAL_LINT_FLAGS="$(FINAL_LINT_FLAGS) --out-format=$(LINT_OUTPUT_FORMAT)"; \
 	fi
-	$(GOLANGCI_LINT) run $(FINAL_LINT_FLAGS)
+	golangci-lint run $(FINAL_LINT_FLAGS)
 
 format-lint: format lint ## Format code and run linters in one step