Update PyTorch profiler

Author: dmelikyan

README.md

@@ -1,15 +1,15 @@
-# Graphsignal: LLM Performance Analytics
+# Graphsignal: AI Performance Profiler
 
 [![License](http://img.shields.io/github/license/graphsignal/graphsignal-python)](https://github.com/graphsignal/graphsignal-python/blob/main/LICENSE)
 [![Version](https://img.shields.io/github/v/tag/graphsignal/graphsignal-python?label=version)](https://github.com/graphsignal/graphsignal-python)
 
 
-Graphsignal is an observability platform for AI applications. It helps developers ensure AI applications run efficiently and provide the best user experience. With Graphsignal, developers can:
+Graphsignal is an AI optimization platform, enabling developers to achieve faster, more efficient models and applications. With Graphsignal, developers can:
 
-* Measure and optimize LLM performance for APIs and hosted models.
-* Analyze model API costs for deployments, models, or any custom tags.
-* View and search LLM generations with full context.
-* Detect errors and monitor APIs, compute, and GPU utilization.
+* Identify and optimize the most significant contributors to latency.
+* Analyze model API costs for deployments, models, sessions, or any custom tags.
+* Ensure optimal model configuration and inference performance for hosted models.
+* Track errors and monitor APIs, compute, and GPU utilization.
 
 
 [![Dashboards](https://graphsignal.com/external/screenshot-dashboard.png)](https://graphsignal.com/)
@@ -19,7 +19,7 @@ Learn more at [graphsignal.com](https://graphsignal.com).
 
 ## Install
 
-Install Graphsignal library.
+Install the Graphsignal library.
 
 ```bash
 pip install --upgrade graphsignal
@@ -38,7 +38,7 @@ graphsignal.configure(api_key='my-api-key', deployment='my-app')
 
 To get an API key, sign up for a free account at [graphsignal.com](https://graphsignal.com). The key can then be found in your account's [Settings / API Keys](https://app.graphsignal.com/settings/api-keys) page.
 
-Alternatively, you can add Graphsignal tracer at command line, when running your module or script. Environment variables `GRAPHSIGNAL_API_KEY` and `GRAPHSIGNAL_DEPLOYMENT` must be set.
+Alternatively, you can add Graphsignal tracer from the command line, when running your module or script. Environment variables `GRAPHSIGNAL_API_KEY` and `GRAPHSIGNAL_DEPLOYMENT` must be set.
 
 ```bash
 python -m graphsignal <script>
@@ -56,13 +56,14 @@ Graphsignal **auto-instruments** and traces libraries and frameworks, such as [O
 Refer to the guides below for detailed information on:
 
 * [Manual Tracing](https://graphsignal.com/docs/guides/manual-tracing/)
+* [Inference Profiling](https://graphsignal.com/docs/guides/inference-profiling/)
 * [Session Tracking](https://graphsignal.com/docs/guides/session-tracking/)
 * [Cost and Usage Monitoring](https://graphsignal.com/docs/guides/cost-and-usage-monitoring/)
 * [Scores and Feedback](https://graphsignal.com/docs/guides/scores-and-feedback/)
 
 See [API reference](https://graphsignal.com/docs/reference/python-api/) for full documentation.
 
-Some integration examples are available in [examples](https://github.com/graphsignal/examples) repo.
+Integration examples are available in [examples](https://github.com/graphsignal/examples) repository.
 
 
 ## Analyze
@@ -72,18 +73,18 @@ Some integration examples are available in [examples](https://github.com/graphsi
 
 ## Overhead
 
-Graphsignal tracer is very lightweight. The overhead per trace is measured to be less than 100 microseconds.
+Graphsignal tracer is highly lightweight. The overhead per trace is measured to be less than 100 microseconds.
 
 
 ## Security and Privacy
 
-Graphsignal tracer can only open outbound connections to `api.graphsignal.com` and send data, no inbound connections or commands are possible.
+The Graphsignal tracer only establishes outbound connections to `api.graphsignal.com` to send data; inbound connections or commands are not possible.
 
 Payloads, such as prompts and completions, are recorded by default in case of automatic tracing. To disable, set `record_payloads=False` in `graphsignal.configure`.
 
 
 ## Troubleshooting
 
-To enable debug logging, add `debug_mode=True` to `configure()`. If the debug log doesn't give you any hints on how to fix a problem, please report it to our support team via your account.
+To enable debug logging, add `debug_mode=True` to `configure()`. If the debug log doesn’t provide hints for resolving the issue, report it to our support team via your account.
 
 In case of connection issues, please make sure outgoing connections to `https://api.graphsignal.com` are allowed.

graphsignal/__init__.py

@@ -59,6 +59,7 @@ def configure(
         tags: Optional[Dict[str, str]] = None,
         auto_instrument: Optional[bool] = True,
         record_payloads: Optional[bool] = True,
+        profiling_rate: Optional[float] = 0.1,
         upload_on_shutdown: Optional[bool] = True,
         debug_mode: Optional[bool] = False) -> None:
     global _tracer
@@ -80,6 +81,7 @@ def configure(
         tags=tags,
         auto_instrument=auto_instrument,
         record_payloads=record_payloads,
+        profiling_rate=profiling_rate,
         upload_on_shutdown=upload_on_shutdown,
         debug_mode=debug_mode)
     _tracer.setup()
@@ -121,18 +123,20 @@ def remove_context_tag(key: str):
 
 def trace(
         operation: str,
-        tags: Optional[Dict[str, str]] = None) -> 'Span':
+        tags: Optional[Dict[str, str]] = None,
+        with_profile: Optional[bool] = False) -> 'Span':
     _check_configured()
 
-    return _tracer.trace(operation=operation, tags=tags)
+    return _tracer.trace(operation=operation, tags=tags, with_profile=with_profile)
 
 
 def trace_function(
         func=None, 
         *,
         operation: Optional[str] = None,
-        tags: Optional[Dict[str, str]] = None):
-    return _tracer.trace_function(func, operation=operation, tags=tags)
+        tags: Optional[Dict[str, str]] = None,
+        with_profile: Optional[bool] = False) -> Any:
+    return _tracer.trace_function(func, operation=operation, tags=tags, with_profile=with_profile)
 
 
 def score(

graphsignal/client/__init__.py

@@ -38,6 +38,7 @@
 from graphsignal.client.models.metric import Metric
 from graphsignal.client.models.metric_type import MetricType
 from graphsignal.client.models.payload import Payload
+from graphsignal.client.models.profile import Profile
 from graphsignal.client.models.rate import Rate
 from graphsignal.client.models.score import Score
 from graphsignal.client.models.score_query_result import ScoreQueryResult

graphsignal/client/api/default_api.py

@@ -18,7 +18,7 @@
 
 from datetime import datetime
 from pydantic import Field, StrictInt, StrictStr
-from typing import List, Optional
+from typing import Any, Dict, List, Optional
 from typing_extensions import Annotated
 from graphsignal.client.models.log_entry import LogEntry
 from graphsignal.client.models.metric import Metric
@@ -354,7 +354,9 @@ def _query_scores_serialize(
         _query_params: List[Tuple[str, str]] = []
         _header_params: Dict[str, Optional[str]] = _headers or {}
         _form_params: List[Tuple[str, str]] = []
-        _files: Dict[str, str] = {}
+        _files: Dict[
+            str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]]
+        ] = {}
         _body_params: Optional[bytes] = None
 
         # process the path parameters
@@ -415,11 +417,12 @@ def _query_scores_serialize(
 
 
         # set the HTTP header `Accept`
-        _header_params['Accept'] = self.api_client.select_header_accept(
-            [
-                'application/json'
-            ]
-        )
+        if 'Accept' not in _header_params:
+            _header_params['Accept'] = self.api_client.select_header_accept(
+                [
+                    'application/json'
+                ]
+            )
 
 
         # authentication setting
@@ -728,7 +731,9 @@ def _query_spans_serialize(
         _query_params: List[Tuple[str, str]] = []
         _header_params: Dict[str, Optional[str]] = _headers or {}
         _form_params: List[Tuple[str, str]] = []
-        _files: Dict[str, str] = {}
+        _files: Dict[
+            str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]]
+        ] = {}
         _body_params: Optional[bytes] = None
 
         # process the path parameters
@@ -781,11 +786,12 @@ def _query_spans_serialize(
 
 
         # set the HTTP header `Accept`
-        _header_params['Accept'] = self.api_client.select_header_accept(
-            [
-                'application/json'
-            ]
-        )
+        if 'Accept' not in _header_params:
+            _header_params['Accept'] = self.api_client.select_header_accept(
+                [
+                    'application/json'
+                ]
+            )
 
 
         # authentication setting
@@ -1030,7 +1036,9 @@ def _upload_logs_serialize(
         _query_params: List[Tuple[str, str]] = []
         _header_params: Dict[str, Optional[str]] = _headers or {}
         _form_params: List[Tuple[str, str]] = []
-        _files: Dict[str, str] = {}
+        _files: Dict[
+            str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]]
+        ] = {}
         _body_params: Optional[bytes] = None
 
         # process the path parameters
@@ -1043,11 +1051,12 @@ def _upload_logs_serialize(
 
 
         # set the HTTP header `Accept`
-        _header_params['Accept'] = self.api_client.select_header_accept(
-            [
-                'application/json'
-            ]
-        )
+        if 'Accept' not in _header_params:
+            _header_params['Accept'] = self.api_client.select_header_accept(
+                [
+                    'application/json'
+                ]
+            )
 
         # set the HTTP header `Content-Type`
         if _content_type:
@@ -1305,7 +1314,9 @@ def _upload_metrics_serialize(
         _query_params: List[Tuple[str, str]] = []
         _header_params: Dict[str, Optional[str]] = _headers or {}
         _form_params: List[Tuple[str, str]] = []
-        _files: Dict[str, str] = {}
+        _files: Dict[
+            str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]]
+        ] = {}
         _body_params: Optional[bytes] = None
 
         # process the path parameters
@@ -1318,11 +1329,12 @@ def _upload_metrics_serialize(
 
 
         # set the HTTP header `Accept`
-        _header_params['Accept'] = self.api_client.select_header_accept(
-            [
-                'application/json'
-            ]
-        )
+        if 'Accept' not in _header_params:
+            _header_params['Accept'] = self.api_client.select_header_accept(
+                [
+                    'application/json'
+                ]
+            )
 
         # set the HTTP header `Content-Type`
         if _content_type:
@@ -1580,7 +1592,9 @@ def _upload_scores_serialize(
         _query_params: List[Tuple[str, str]] = []
         _header_params: Dict[str, Optional[str]] = _headers or {}
         _form_params: List[Tuple[str, str]] = []
-        _files: Dict[str, str] = {}
+        _files: Dict[
+            str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]]
+        ] = {}
         _body_params: Optional[bytes] = None
 
         # process the path parameters
@@ -1593,11 +1607,12 @@ def _upload_scores_serialize(
 
 
         # set the HTTP header `Accept`
-        _header_params['Accept'] = self.api_client.select_header_accept(
-            [
-                'application/json'
-            ]
-        )
+        if 'Accept' not in _header_params:
+            _header_params['Accept'] = self.api_client.select_header_accept(
+                [
+                    'application/json'
+                ]
+            )
 
         # set the HTTP header `Content-Type`
         if _content_type:
@@ -1855,7 +1870,9 @@ def _upload_spans_serialize(
         _query_params: List[Tuple[str, str]] = []
         _header_params: Dict[str, Optional[str]] = _headers or {}
         _form_params: List[Tuple[str, str]] = []
-        _files: Dict[str, str] = {}
+        _files: Dict[
+            str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]]
+        ] = {}
         _body_params: Optional[bytes] = None
 
         # process the path parameters
@@ -1868,11 +1885,12 @@ def _upload_spans_serialize(
 
 
         # set the HTTP header `Accept`
-        _header_params['Accept'] = self.api_client.select_header_accept(
-            [
-                'application/json'
-            ]
-        )
+        if 'Accept' not in _header_params:
+            _header_params['Accept'] = self.api_client.select_header_accept(
+                [
+                    'application/json'
+                ]
+            )
 
         # set the HTTP header `Content-Type`
         if _content_type: