Implement configurable logging system to replace print statements

[Copilot]

Overview

This PR implements a unified, configurable logging system for Stream DaQ that replaces scattered print() statements with proper logging management. The solution provides sensible defaults while allowing users to customize verbosity levels and output formatting.

Motivation

Previously, Stream DaQ used a mix of print() statements and logging calls across different modules:

• StreamDaQ.py used print statements for user feedback
• SchemaValidator.py used logging for validation events
• utils.py configured logging independently

This inconsistency made it difficult for users to control output verbosity and resulted in a fragmented logging experience.

Changes

1. Centralized Logging Configuration

Created a new streamdaq/logging_config.py module that provides:

from streamdaq import configure_logging, set_level, get_current_level
import logging

# Quick level change
set_level(logging.DEBUG)

# Full customization
configure_logging(
    level=logging.INFO,
    format_string="%(asctime)s - %(levelname)s - %(message)s",
    date_format="%Y-%m-%d %H:%M:%S"
)

2. Unified Print Statements

Replaced print statements in StreamDaQ.py with appropriate logging calls:

• print("Data set to artificial...") → logger.info(...)
• print("Violations sink is being set!") → logger.debug(...)

3. Updated Existing Logging

• utils.py: Removed manual logging.basicConfig() configuration, now uses centralized logging
• SchemaValidator.py: Updated to use centralized logger initialization

All modules now follow a consistent pattern:

from streamdaq.logging_config import get_logger
logger = get_logger(__name__)

4. Public API

Exported logging configuration functions in streamdaq/__init__.py:

• configure_logging() - Full configuration control
• set_level() - Quick verbosity changes
• get_current_level() - Query current level

5. Documentation & Examples

• New Documentation: docs/concepts/logging.rst - Comprehensive guide covering quick start, common scenarios, API reference, and best practices
• New Example: examples/logging_configuration.py - Demonstrates 6 different logging configurations
• Updated: README.md and docs/concepts/index.rst to reference logging feature

Usage Examples

Default Behavior (INFO level)

from streamdaq import StreamDaQ, DaQMeasures as dqm, Windows

# INFO level shows important operational messages
daq = StreamDaQ().configure(
    window=Windows.tumbling(3),
    instance="user_id",
    time_column="timestamp"
)
# Output: [INFO] streamdaq.StreamDaQ: Data set to artificial and data representation to native.

Development - Verbose Output

from streamdaq import set_level
import logging

# Show everything for debugging
set_level(logging.DEBUG)

Production - Minimal Output

from streamdaq import set_level
import logging

# Only show warnings and errors
set_level(logging.WARNING)

Custom Formatting

from streamdaq import configure_logging
import logging

configure_logging(
    level=logging.INFO,
    format_string="[%(asctime)s] %(levelname)s - %(message)s",
    date_format="%H:%M:%S"
)

Benefits

• ✅ Unified approach: Single logging system across all modules
• ✅ Configurable: Users control verbosity (DEBUG, INFO, WARNING, ERROR, CRITICAL)
• ✅ Sensible defaults: INFO level provides balanced visibility
• ✅ Production ready: Easy to reduce output with set_level(logging.WARNING)
• ✅ Developer friendly: Easy to increase output with set_level(logging.DEBUG)
• ✅ Standard library: Uses Python's built-in logging module
• ✅ Well documented: Comprehensive docs and examples
• ✅ Backward compatible: No breaking changes, existing code works without modification

Testing

• Created comprehensive integration tests validating all components
• Verified StreamDaQ, SchemaValidator, and utils integration
• Tested level hierarchy, custom formatting, and no duplicate logs
• Applied black, isort, and fixed all flake8 issues
• All existing functionality preserved

Migration Guide

For Users

No action required! The default INFO level provides output similar to previous versions. To customize:

# Reduce output for production
from streamdaq import set_level
import logging
set_level(logging.WARNING)

# Increase output for debugging
set_level(logging.DEBUG)

For Contributors

Use the centralized logger in new code:

from streamdaq.logging_config import get_logger
logger = get_logger(__name__)

logger.debug("Detailed diagnostic info")
logger.info("Normal operation confirmation")
logger.warning("Unexpected but handled event")
logger.error("Serious problem")

Closes #[issue_number]

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[FEATURE] Better logging</issue_title>
<issue_description>## Is your feature request related to a problem? Please describe
Not related to a problem.

Describe the solution you'd like

Now there are a lot of print() statements. We need a better user experience, using enhanced logging management. The approach should unify schema validation (which uses logging) and the rest of the code base (which primarily uses print statements).

Describe alternatives you've considered

No outstanding alternatives.

Additional context

Let's focus this PR only on implementing a configurable logging mechanism with sensible default values and verbosity level.</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes #27

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.