feat(io): add Prometheus Pushgateway support

Author: AndreyMarkinPPC

docs/usage/writers.md

@@ -21,6 +21,7 @@
 | `pubsub`   | PubSubWriter     | `project` |
 | `mongo`    | MongoDbWriter     | `connection_string`, `db` |
 | `firestore`   | FirestoreWriter     | `project`, `db` |
+| `pushgateway`   | PushgatewayWriter     | `endpoint`, `namespace`, `job` |
 
 
 ## Installation
@@ -53,6 +54,7 @@ To install specific writers use:
 * `pip install garf-io[pubsub]` for PubSub support
 * `pip install garf-io[mongo]` for MongoDB support
 * `pip install garf-io[firestore]` for Firestore support
+* `pip install garf-io[pushgateway]` for Prometheus Pushgateway support
 
 
 ## Usage

docs/usage/writers/pushgateway-writer.md

@@ -0,0 +1,146 @@
+!!! important
+    To save data to Pushgateway install `garf-io` with Pushgateway support
+
+    ```bash
+    pip install garf-io[pushgateway]
+    ```
+
+
+`pushgateway` writer allows you to publish `GarfReport` to a Pushgateway endpoint.
+
+/// tab | cli
+```bash
+garf query.sql --source API_SOURCE \
+  --output pushgateway
+```
+///
+
+/// tab | python
+```python
+from garf.core import report
+from garf.io.writers import pushgateway_writer
+
+sample_report = report.GarfReport(results=[[1]], column_names=['one'])
+
+writer = pushgateway_writer.PushgatewayWriter()
+writer.write(sample_report, 'grouping_key')
+```
+///
+
+## Parameters
+### endpoint
+
+By default writer pushes data to `http://localhost:9091/`.
+You can overwrite it with `endpoint` parameter.
+
+/// tab | cli
+```bash hl_lines="3"
+garf query.sql --source API_SOURCE \
+  --output pushgateway \
+  --pushgateway.endpoint=http://pushgateway:9091
+```
+///
+
+/// tab | python
+```python hl_lines="7"
+from garf.core import report
+from garf.io.writers import pushgateway_writer
+
+sample_report = report.GarfReport(results=[[1]], column_names=['one'])
+
+writer = pushgateway_writer.PushgatewayWriter(endpoint="http://pushgateway:9091")
+writer.write(sample_report, 'grouping_key')
+```
+///
+
+### namespace
+
+Every metric pushes to Pushgateway is prefix with `garf_` prefix.
+You can overwrite it with `namespace` parameter.
+
+/// tab | cli
+```bash hl_lines="3"
+garf query.sql --source API_SOURCE \
+  --output pushgateway \
+  --pushgateway.namespace=my_garf
+```
+///
+
+/// tab | python
+```python hl_lines="7"
+from garf.core import report
+from garf.io.writers import pushgateway_writer
+
+sample_report = report.GarfReport(results=[[1]], column_names=['one'])
+
+writer = pushgateway_writer.PushgatewayWriter(namespace="my_garf")
+writer.write(sample_report, 'grouping_key')
+```
+///
+
+### job
+
+By default all metrics is pushed into `garf` job.
+You can overwrite it with `job` parameter.
+
+/// tab | cli
+```bash hl_lines="3"
+garf query.sql --source API_SOURCE \
+  --output pushgateway \
+  --pushgateway.job=garf_job
+```
+///
+
+/// tab | python
+```python hl_lines="7"
+from garf.core import report
+from garf.io.writers import pushgateway_writer
+
+sample_report = report.GarfReport(results=[[1]], column_names=['one'])
+
+writer = pushgateway_writer.PushgatewayWriter(job="garf_job")
+writer.write(sample_report, 'grouping_key')
+```
+///
+
+### expose_metrics_with_zero_values
+
+By default only metrics with non-zero values are pushed to Pushgateway.
+You can overwrite it with `expose_metrics_with_zero_values` parameter.
+
+/// tab | cli
+```bash hl_lines="3"
+garf query.sql --source API_SOURCE \
+  --output pushgateway \
+  --pushgateway.expose_metrics_with_zero_values
+```
+///
+
+/// tab | python
+```python hl_lines="7"
+from garf.core import report
+from garf.io.writers import pushgateway_writer
+
+sample_report = report.GarfReport(results=[[1]], column_names=['one'])
+
+writer = pushgateway_writer.PushgatewayWriter(expose_metrics_with_zero_values=True)
+writer.write(sample_report, 'grouping_key')
+```
+///
+
+
+## Query syntax
+
+In order to expose metrics to Pushgateway you need to ensure that either field name
+or alias should contain `metric` in it.
+So given the query
+
+```sql
+SELECT
+  dimension.name AS name,
+  metric.name AS field1,
+  name AS metric_field2
+FROM resource
+```
+
+two metrics will be pushed to Pushgateway - `<namespace>_<job>_field1` and `<namespace>_<job>_field2`.

libs/io/README.md

@@ -5,36 +5,26 @@
 
 `garf-io` handles reading queries and writing `GarfReport` to various local/remote storages.
 
-Currently it supports writing data to the following destination:
-
-| identifier | Writer           | Options  |
-|------------| ---------------- | -------- |
-| `console`  | ConsoleWriter    | `page-size=10`,`format=table\|json\|jsonl`|
-| `csv`      | CsvWriter        | `destination-folder` |
-| `json`     | JsonWriter       | `destination-folder`,`format=json\|jsonl`|
-| `bq`       | BigQueryWriter   | `project`, `dataset`, `location`, `write-disposition` |
-| `sqldb`    | SqlAlchemyWriter | `connection-string`, `if-exists=fail\|replace\|append` |
-| `sheets`   | SheetsWriter     | `share-with`, `credentials-file`, `spreadsheet-url`, `is_append=True\|False`|
-
-Each of writer also support two options for dealing with arrays:
-
-* `WRITER.array-handling` - arrays handling method: "strings" (default)  - store arrays as strings (items combined via a separator, e.g. "item1|item2"), "arrays" - store arrays as arrays.
-* `WRITER.array-separator` - a separator symbol for joining arrays as strings, by default '|'.
-
 ## Installation
 
-`pip install garf-io`
+```
+pip install garf-io
+```
+By default  `garf-io` has only support for `console`, `csv` and `json` writers -
+explore what [additional writers are available](https://google.github.io/garf/usage/writers).
 
-By default  `garf-io` has only support for `console`, `csv` and `json` writers.\
-To install all writers use the following command `pip install garf-io[all]`.\
-To install specific writers use:
-* `pip install garf-io[bq]` for BigQuery support
-* `pip install garf-io[sheets]` for Google spreadsheets support
-* `pip install garf-io[sqlalchemy]` for SqlAlchemy support
 
 ## Usage
 
+### CLI
+
+```
+garf query.sql --source API_SOURCE \
+  --output csv --csv.destination-folder=/tmp/
 ```
+### Python
+
+```python
 import garf.core import report
 from garf.io import writer
 
@@ -47,3 +37,7 @@ concrete_writer = writer.create_writer('csv', destination_folder='/tmp/')
 # Write data to /tmp/sample.csv
 concrete_writer.write(sample_report, 'sample')
 ```
+
+## Documentation
+
+[Learn more](https://google.github.io/garf/usage/writers) about using `garf` writers.

libs/io/garf/io/__init__.py

@@ -13,4 +13,4 @@
 # limitations under the License.
 """Write GarfReport to anywhere."""
 
-__version__ = '1.3.1'
+__version__ = '1.3.2'

libs/io/garf/io/formatter.py

@@ -49,6 +49,7 @@ def apply_transformations(
       results=formatted_rows,
       column_names=report.column_names,
       results_placeholder=formatted_placeholders,
+      query_specification=report.query_specification,
     )
 
   def _format_rows(