Merge pull request #90 from bqbooster/doc-improv

Improve the documentation regarding combining audit logs with informa…

Author: Kayrnt

.changes/unreleased/Docs-20241215-170028.yaml

@@ -0,0 +1,6 @@
+kind: Docs
+body: Improve the documentation regarding combining audit logs with information schema jobs data
+time: 2024-12-15T17:00:28.333756+01:00
+custom:
+    Author: Kayrnt
+    Issue: ""

docs/audit-logs-vs-information-schema.md

@@ -9,7 +9,10 @@ There are two ways to monitor BigQuery jobs:
 - Using the BigQuery audit logs
 - Using the `INFORMATION_SCHEMA.JOBS` table
 
-## How to choose
+dbt-bigquery-monitoring supports both methods and goes further by providing a unified way to monitor both by offering a configuration that will combine the two sources.
+See `should_combine_audit_logs_and_information_schema` in the [configuration](/configuration) if you want to combine the two sources.
+
+## What's in there?
 
 Each of the solution has its advantages and disadvantages. Here is a comparison table to help you choose the right one for your use case:
 
@@ -18,9 +21,7 @@ Each of the solution has its advantages and disadvantages. Here is a comparison
 | Max retention | User defined | 6 months |
 | Detailed User information | ✅ | ❌ |
 | BI Engine | ❌ | ✅ |
-| Insights | ❌ | ✅ |
-
-At some point, a mode to combine both solutions could be implemented.
+| Jobs insights | ❌ | ✅ |
 
 ## Audit logs
 

docs/running-the-package.md

@@ -6,12 +6,16 @@ slug: /running-the-package
 # Running the package
 
 The package is designed to be run as a daily or hourly job.
-To do so, you can use the following dbt command:
+It leverages incremental modelisations to reduce the amount of data to process and to optimize the performance of the queries. Practically it won't reread data that has already been processed (and not needed anymore).
+
+If you plan to run all models, the simplest way to run the job is to run the following dbt command:
 
 ```
 dbt run -s tag:dbt-bigquery-monitoring
 ```
 
+The granularity of the data partitioning is hourly so the most cost efficient way to process the data is run it every hour but you may run it more frequently if you need more "real-time" data.
+
 ## Tags
 
 The package provides the following tags that can be used to filter the models: