Merge pull request #16 from autometrics-dev/fp-3168-autometrics-py-feature-add-slo-support

Add slo support

Author: brettimus

.github/workflows/build.yml

@@ -20,6 +20,9 @@ jobs:
           cache: "poetry"
       - name: Install dependencies
         run: poetry install --no-interaction --no-root --with dev
-      - uses: psf/black@stable
-      - name: Run pyright
+      - name: Check code formatting
+        run: poetry run black .
+      - name: Lint code
         run: poetry run pyright
+      - name: Run tests
+        run: poetry run pytest

README.md

@@ -1,6 +1,8 @@
+![GitHub_headerImage](https://user-images.githubusercontent.com/3262610/221191767-73b8a8d9-9f8b-440e-8ab6-75cb3c82f2bc.png)
+
 # autometrics-py
 
-A Python decorator that makes it easy to understand the error rate, response time, and production usage of any function in your code. Jump straight from your IDE to live Prometheus charts for each HTTP/RPC handler, database method, or other piece of application logic.
+A Python library that exports a decorator that makes it easy to understand the error rate, response time, and production usage of any function in your code. Jump straight from your IDE to live Prometheus charts for each HTTP/RPC handler, database method, or other piece of application logic.
 
 Autometrics for Python provides:
 
@@ -18,8 +20,7 @@ See [Why Autometrics?](https://github.com/autometrics-dev#why-autometrics) for m
 - 🔗 Create links to live Prometheus charts directly into each functions docstrings (with tooltips coming soon!)
 - 📊 (Coming Soon!) Grafana dashboard showing the performance of all
   instrumented functions
-- 🚨 (Coming Soon!) Generates Prometheus alerting rules using SLO best practices
-  from simple annotations in your code
+- 🚨 Enable Prometheus alerts using SLO best practices from simple annotations in your code
 - ⚡ Minimal runtime overhead
 
 ## Using autometrics-py
@@ -45,15 +46,96 @@ def sayHello:
 
 > Note that we cannot support tooltips without a VSCode extension due to behavior of the [static analyzer](https://github.com/davidhalter/jedi/issues/1921) used in VSCode.
 
+## Alerts / SLOs
+
+Autometrics makes it easy to add Prometheus alerts using Service-Level Objectives (SLOs) to a function or group of functions.
+
+In order to receive alerts you need to add a set of rules to your Prometheus set up. You can find out more about those rules here: [Prometheus alerting rules](https://github.com/autometrics-dev/autometrics-shared#prometheus-recording--alerting-rules). Once added, most of the recording rules are dormant. They are enabled by specific metric labels that can be automatically attached by autometrics.
+
+To use autometrics SLOs and alerts, create one or multiple `Objective`s based on the function(s) success rate and/or latency, as shown below. The `Objective` can be passed as an argument to the `autometrics` macro to include the given function in that objective.
+
+```python
+from autometrics import autometrics
+from autometrics.objectives import Objective, ObjectiveLatency, ObjectivePercentile
+
+API_SLO = Objective(
+    "random",
+    success_rate=ObjectivePercentile.P99_9,
+    latency=(ObjectiveLatency.Ms250, ObjectivePercentile.P99),
+)
+
+@autometrics(objective=API_SLO)
+def api_handler():
+  # ...
+```
+
+Autometrics by default will try to store information on which function calls a decorated function. As such you may want to place the autometrics in the top/first decorator, as otherwise you may get `inner` or `wrapper` as the caller function.
+
+So instead of writing:
+
+```py
+from functools import wraps
+from typing import Any, TypeVar, Callable
+
+R = TypeVar("R")
+
+def noop(func: Callable[..., R]) -> Callable[..., R]:
+    """A noop decorator that does nothing."""
+
+    @wraps(func)
+    def inner(*args: Any, **kwargs: Any) -> Any:
+        return func(*args, **kwargs)
+
+    return inner
+
+@noop
+@autometrics
+def api_handler():
+  # ...
+```
+
+You may want to switch the order of the decorator
+
+```py
+@autometrics
+@noop
+def api_handler():
+  # ...
+```
+
 ## Development of the package
 
 This package uses [poetry](https://python-poetry.org) as a package manager, with all dependencies separated into three groups:
- - root level dependencies, required
- - `dev`, everything that is needed for development or in ci
- - `examples`, dependencies of everything in `examples/` directory
+
+- root level dependencies, required
+- `dev`, everything that is needed for development or in ci
+- `examples`, dependencies of everything in `examples/` directory
 
 By default, poetry will only install required dependencies, if you want to run examples, install using this command:
 
-`poetry install --with examples`
+```sh
+poetry install --with examples
+```
+
+Code in this repository is:
+
+- formatted using [black](https://black.readthedocs.io/en/stable/).
+- contains type definitions (which are linted by [pyright](https://microsoft.github.io/pyright/))
+- tested using [pytest](https://docs.pytest.org/)
+
+In order to run these tools locally you have to install them, you can install them using poetry:
 
-Code in this repository is formatted using [black](https://black.readthedocs.io/en/stable/) and contains type definitions (which are linted by [pyright](https://microsoft.github.io/pyright/))
+```sh
+poetry install --with dev
+```
+
+After that you can run the tools individually
+
+```sh
+# Formatting using black
+poetry run black .
+# Lint using pyright
+poetry run pyright
+# Run the tests using pytest
+poetry run pytest
+```

examples/example.py

@@ -1,7 +1,8 @@
-from prometheus_client import start_http_server
-from autometrics import autometrics
 import time
 import random
+from prometheus_client import start_http_server
+from autometrics import autometrics
+from autometrics.objectives import Objective, ObjectiveLatency, ObjectivePercentile
 
 
 # Defines a class called `Operations`` that has two methods:
@@ -36,6 +37,24 @@ def div_unhandled(num1, num2):
     return result
 
 
+RANDOM_SLO = Objective(
+    "random",
+    success_rate=ObjectivePercentile.P99_9,
+    latency=(ObjectiveLatency.Ms250, ObjectivePercentile.P99),
+)
+
+
+@autometrics(objective=RANDOM_SLO)
+def random_error():
+    """This function will randomly return an error or ok."""
+
+    result = random.choice(["ok", "error"])
+    if result == "error":
+        time.sleep(1)
+        raise RuntimeError("random error")
+    return result
+
+
 ops = Operations()
 
 # Show the docstring (with links to prometheus metrics) for the `add` method
@@ -59,3 +78,4 @@ def div_unhandled(num1, num2):
     time.sleep(2)
     # Call `div_unhandled` such that it raises an error
     div_unhandled(2, 0)
+    random_error()