feat(config): Centralize configuration and improve modularity

[SyedaAnshrahGillani]

This PR introduces a centralized configuration management system and significantly enhances the modularity of the application. Previously, configuration settings were scattered across environment variables, command-line arguments, and hardcoded defaults, resulting in a codebase that was difficult to maintain and extend.

Key Changes:

• config.py Module: Introduces a new config.py module containing a Config dataclass that encapsulates all application settings. This serves as a single source of truth, loading configuration values intelligently from environment variables and command-line arguments with a well-defined precedence.
• Refactored main.py: Updated to utilize the new Config object, removing scattered os.getenv calls and simplifying argument parsing. This results in a cleaner and more readable application entry point.
• Decoupled LLM Interaction: The llm.py module no longer relies on a global state (GLOBAL_LLM). Instead, an LLM instance is explicitly created in main.py and passed as a dependency to functions that require it (e.g., rerank_paper in recommender.py, render_email in construct_email.py). This promotes explicit dependency injection, improving component independence and testability.
• Improved Function Signatures: Functions that previously accepted numerous individual configuration parameters (such as set_global_llm) now accept the centralized Config object or relevant subsets, leading to clearer and more maintainable function signatures.

Benefits:

• Centralized Configuration: Simplifies understanding, modification, and debugging by consolidating all application settings in one place.
• Enhanced Modularity: Decoupling from global state and employing dependency injection enables components to be developed, tested, and maintained independently.
• Increased Testability: Explicit dependency passing and the removal of global state greatly facilitate unit testing of individual functions and classes.
• Improved Readability and Maintainability: Cleaner code with a clear separation of concerns reduces cognitive load and eases future development and maintenance.
• Better Extensibility: The system is designed to accommodate new configuration options or services with minimal impact on existing code.

This refactoring marks a foundational improvement to the project’s architecture, establishing a robust framework for scalable and maintainable future development.

[TideDra]

Thanks for your contribution! I'm trying to reconstruct this project recently, and the new version will adapt hydra as the centralized configuration system. To avoid conflict, this PR will not be merged currently. But I will take some good points of this PR for reference. Thanks again!

[SyedaAnshrahGillani]

No worries at all, glad it was helpful in some way. Appreciate the note!