Priority Queue for TEE Worker Jobs #91
Description
Feature Request: Priority Queue for TEE Worker Jobs
Overview
Implement a priority queue system that allows users to specify which TEE worker IDs should receive priority in job processing. This will help in managing workload distribution, preventing abuse, and ensuring fair scoring.
Acceptance Criteria
- The system must allow users to specify a list of worker IDs that should be prioritized.
- The prioritization must be enforced at the job submission stage, ensuring that jobs from prioritized workers are processed first.
- The prioritized worker IDs must be fetched from the TEE API (https://github.com/masa-finance/masa-tee-api https://tee-api.masa.ai ) endpoint specified in the binary (configurable during build-time).
- The worker must have an internal priority queue mechanism to handle prioritized jobs.
- The feature must be configurable via build-time variables, with clear defaults provided.
- Documentation must be updated to reflect the new feature and its configuration options.
- Unit tests must be added to verify the prioritization logic.
Background
The current system does not have a mechanism to prioritize jobs based on worker IDs. This can lead to potential abuse, such as miners spamming each other's workers to rate limit them. By introducing a priority queue, we can ensure fairer distribution of work and improve the overall system integrity.
References
Additional Notes
- The prioritized worker list should be hardcoded to prevent tampering by workers.
- The system should handle cases where the TEE API is unavailable by falling back to a default prioritization strategy.
- Consider implementing a mechanism to dynamically update the prioritized worker list without restarting the worker.