CLI: use library collectors (#339)

* CLI: use library collectors

use metrics_utility.library.collectors, via cli_* wrappers

cli collectors get since, until, output params,
passed to library, and used to specify collector output

exposed all exisiting collectors in the cli
limit_slicer becomes until_slicer, using until over now()
full_sync removed, unused
register description removed, unused

Issue: AAP-66077

* collector output - DictOutput, DataframeOutput, CollectorOutput

library collectors now get an output= param,
defaulting to either `DataframeOutput`, where `output.sql(db, query)` returns a pandas dataframe,
or to `DictOutput`, where `output.dict({...})` returns a dict.

for the CLI calls, `CollectorOutput` gets passed instead, initialized with a temporary path for files,
and outputs a filelist from output.sql, and a dict from output.dict.

this allows preserving the csv streaming for the cli, while keeping the
collector output a dataframe for anonymized

* review fixups

* claude fix tests

* more fixups

* unify on bool_from_env; fix 0 check

* pytest approx - ensure no rel=1e-6 for timestamps (was a 28 minutes tolerance)

* simplify the vcpu not started error flow

* test fixups

* make sure until puts this into the right day, but also always within the since/until interval

* claude up test coverage

* test_gather_ranges - update for until_slicer review changes

Author: himdel

metrics_utility/automation_controller_billing/collector.py

@@ -9,6 +9,7 @@
 
 from metrics_utility.automation_controller_billing.helpers import get_last_entries_from_db
 from metrics_utility.automation_controller_billing.package.factory import Factory as PackageFactory
+from metrics_utility.base.utils import bool_from_env
 from metrics_utility.library.lock import lock
 from metrics_utility.logger import logger
 
@@ -28,6 +29,7 @@ def __init__(self, collection_type=base.Collector.SCHEDULED_COLLECTION, collecto
     # TODO: extract advisory lock name in the superclass and log message, so we can change it here and then use
     # this method from superclass
     # TODO: extract to superclass ability to push extra params into config.json
+    # FIXME: subset is only used for tests, mock registered collectors instead?
     def gather(self, dest=None, subset=None, since=None, until=None, billing_provider_params=None):
         """Entry point for gathering
 
@@ -95,21 +97,20 @@ def _load_last_gathered_entries(self):
 
     def _gather_finalize(self):
         """Persisting timestamps (manual/schedule mode only)"""
-
-        disabled_str = os.getenv('METRICS_UTILITY_DISABLE_SAVE_LAST_GATHERED_ENTRIES', 'false')
-        disabled = False
-        if disabled_str and (disabled_str.lower() == 'true'):
-            disabled = True
-
-        if self.ship and not disabled:
-            # We need to wait on analytics lock, to update the last collected timestamp settings
-            # so we don't clash with analytics job collection.
-            with lock('gather_analytics_lock', wait=True, db=connection):
-                # We need to load fresh settings again as we're obtaning the lock, since
-                # Analytics job could have changed this on the background and we'd be resetting
-                # the Analytics values here.
-                self._load_last_gathered_entries()
-                self._update_last_gathered_entries()
+        if not self.ship:
+            return
+
+        if bool_from_env('METRICS_UTILITY_DISABLE_SAVE_LAST_GATHERED_ENTRIES'):
+            return
+
+        # We need to wait on analytics lock, to update the last collected timestamp settings
+        # so we don't clash with analytics job collection.
+        with lock('gather_analytics_lock', wait=True, db=connection):
+            # We need to load fresh settings again as we're obtaning the lock, since
+            # Analytics job could have changed this on the background and we'd be resetting
+            # the Analytics values here.
+            self._load_last_gathered_entries()
+            self._update_last_gathered_entries()
 
     def _save_last_gathered_entries(self, last_gathered_entries):
         settings.AUTOMATION_ANALYTICS_LAST_ENTRIES = json.dumps(last_gathered_entries, cls=DjangoJSONEncoder)