You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
1. Push your release branch to the llm-d-router remote.
@@ -79,13 +79,13 @@ This document defines the process for releasing llm-d-router.
79
79
For a release candidate:
80
80
81
81
```shell
82
-
git tag -s -a v${MAJOR}.${MINOR}.${PATCH}-rc.${RC} -m 'llm-d-router v${MAJOR}.${MINOR}.${PATCH}-rc.${RC} Release Candidate'
82
+
git tag -s -a v${MAJOR}.${MINOR}.${PATCH}-rc.${RC} -m "llm-d-router v${MAJOR}.${MINOR}.${PATCH}-rc.${RC} Release Candidate"
83
83
```
84
84
85
85
For a major, minor or patch release:
86
86
87
87
```shell
88
-
git tag -s -a v${MAJOR}.${MINOR}.${PATCH} -m 'llm-d-router v${MAJOR}.${MINOR}.${PATCH} Release'
88
+
git tag -s -a v${MAJOR}.${MINOR}.${PATCH} -m "llm-d-router v${MAJOR}.${MINOR}.${PATCH} Release"
89
89
```
90
90
91
91
1. Push the tag to the llm-d-router repo.
@@ -102,16 +102,17 @@ This document defines the process for releasing llm-d-router.
102
102
git push ${REMOTE} v${MAJOR}.${MINOR}.${PATCH}
103
103
```
104
104
105
-
1. Pushing the tag triggers CI action to build and publish the [EPP image] and [sidecar image] to the [ghcr registry].
106
-
1. Test the steps in the tagged quickstart guide after the PR merges. TODO add e2e tests!<!-- link to an e2e tests once we have such one -->
105
+
1. Pushing the tag triggers CI action to build and publish the EPP image (`ghcr.io/llm-d/llm-d-router-endpoint-picker`) and sidecar image (`ghcr.io/llm-d/llm-d-router-disagg-sidecar`) to the [ghcr registry].
106
+
1. Verify the [CI release workflow] completed successfully before proceeding.
107
+
1. Test the steps in the tagged quickstart guide after the PR merges.
107
108
108
109
### Create the release!
109
110
110
111
1. Create a [new release]:
111
112
1. Choose the tag that you created for the release.
112
-
1. Use the tag as the release title, i.e.`v0.1.0` refer to previous release for the content of the release body.
113
+
1. Use the tag as the release title, e.g.`v0.1.0`.
113
114
1. Click "Generate release notes" and preview the release body.
114
-
1. Go to Gateway Inference Extension latest release and make sure to include the highlights in llm-d-router as well.
115
+
1. Ensure the release body includes: highlights, breaking changes (if any), known issues, and upgrade steps.
115
116
1. If this is a release candidate, selectthe"This is a pre-release" checkbox.
116
117
1. If you find any bugs in this process, create an [issue].
117
118
@@ -131,7 +132,6 @@ Use the following steps to announce the release.
Copy file name to clipboardExpand all lines: AGENTS.md
+29Lines changed: 29 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -39,6 +39,35 @@ llm-d Router. Go service that routes inference requests to model-serving pods vi
39
39
- State each fact once, in its canonical location. Do not duplicate across struct docs, prose, tables, inline comments, and examples.
40
40
- Do not use Unicode symbols or special characters in general, unless explicitly requested.
41
41
42
+
### Logging
43
+
44
+
The codebase uses `go-logr` via controller-runtime. Verbosity constants are defined in `pkg/common/observability/logging` (`DEFAULT=2`, `VERBOSE=3`, `DEBUG=4`, `TRACE=5`).
45
+
46
+
**Level conventions:**
47
+
48
+
-`logger.Info(...)` for once-per-request operational signals.
49
+
-`logger.V(logging.DEBUG).Info(...)` for per-item or per-loop signals that fire multiple times per request.
50
+
-`logger.V(logging.TRACE).Info(...)` for detailed state transitions (cache operations, index updates).
51
+
-`logger.Error(err, "msg", ...)` for recoverable errors that carry an underlying `error` value.
For more details, see the Gateway API Inference Extension
878
883
[getting started guide](https://gateway-api-inference-extension.sigs.k8s.io/guides/).
879
884
885
+
## Logging
886
+
887
+
We use `logr.Logger` interface for logging everywhere.
888
+
The logger instance is loaded from `context.Context` or passed around as an argument directly.
889
+
This is aligned with contextual logging as explained in [k8s instrumentation logging guidelines](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md).
890
+
891
+
In other words, we explicitly don't use `klog` global logging calls.
892
+
Using `klog` log value helpers like `klog.KObj` is just fine.
893
+
894
+
### Change log verbosity
895
+
896
+
We generally follow the [k8s instrumentation logging guidelines](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md), which states "the practical default level is V(2). Developers and QE environments may wish to run at V(3) or V(4)".
897
+
898
+
To configure logging verbosity, specify the `v` flag such as `--v=2`.
899
+
900
+
If `--v` is not set explicitly, the default verbosity is V(2) (`DEFAULT`).
901
+
### Add logs
902
+
903
+
The [k8s instrumentation logging guidelines](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md) have the following definitions:
904
+
905
+
-`logger.V(0).Info` = `logger.Info` - Generally useful for this to **always** be visible to a cluster operator
906
+
-`logger.V(1).Info` - A reasonable default log level if you don't want verbosity.
907
+
-`logger.V(2).Info` - Useful steady state information about the service and important log messages that may correlate to significant changes in the system. This is the recommended default log level for most systems.
908
+
-`logger.V(3).Info` - Extended information about changes
909
+
-`logger.V(4).Info` - Debug level verbosity
910
+
-`logger.V(5).Info` - Trace level verbosity
911
+
912
+
We choose to simplify to the following 4 common levels.
913
+
914
+
```go
915
+
const (
916
+
DEFAULT = 2
917
+
VERBOSE = 3
918
+
DEBUG = 4
919
+
TRACE = 5
920
+
)
921
+
```
922
+
923
+
The guidelines are written in the context of a k8s controller. Our [epp](pkg/epp/) does more things such as handling requests and scraping metrics, therefore we adapt the guidelines as follows:
924
+
925
+
1. The server startup process and configuration.
926
+
927
+
-`logger.Info` Logging at the `V(0)` verbosity level is generally welcome here as this is only logged once at startup, and provides useful info for debugging.
928
+
929
+
2. Reconciler loops. The reconciler loops watch for CR changes such as the `InferenceObjective` CR. And given changes in these CRs significantly affect the behavior of the extension, we recommend using `V(DEFAULT)` verbosity level as default, and sparsely use higher verbosity levels.
930
+
931
+
-`logger.V(DEFAULT)`
932
+
- Default log level in the reconcilers.
933
+
- Information about config (listening on X, watching Y)
934
+
- Errors that repeat frequently that relate to conditions that can be corrected (e.g., inference model not initialized yet)
935
+
- System state changing (adding/removing objects in the data store)
936
+
-`logger.V(VERBOSE)` and above: Use your best judgement.
937
+
938
+
3. Inference request handling. These requests are expected to be much higher volume than the control flow in the reconcilers and therefore we should be mindful of log spamming. We recommend using v=2 to log important info about a request, such as the HTTP response code, and higher verbosity levels for less important info.
939
+
940
+
-`logger.V(DEFAULT)`
941
+
- Logging the status code of an HTTP request
942
+
- Important decision making such as picking the target model, target pod
943
+
-`logger.V(VERBOSE)`
944
+
- Detailed request scheduling algorithm operations, such as running the filtering logic
945
+
-`logger.V(DEBUG)` and above: Use your best judgement.
946
+
947
+
4. Metric scraping loops. These loops run at a very high frequency, and logs can be very spammy if not handled properly.
948
+
949
+
-`logger.V(TRACE)`
950
+
- Transient errors/warnings, such as failure to get response from a pod.
951
+
- Important state changes, such as updating a metric.
952
+
953
+
5. Misc
954
+
1. Periodic (every 5s) debug loop which prints the current pods and metrics.
955
+
-`logger.V(DEFAULT).Error` If the metrics are not fresh enough, which indicates an error occurred during the metric scraping loop.
956
+
-`logger.V(DEBUG)`
957
+
- This is very important to debug the request scheduling algorithm, and yet not spammy compared to the metric scraping loop logs.
958
+
959
+
### Passing Logger Around
960
+
961
+
You can pass around a `context.Context` that contains a logger or a `logr.Logger` instance directly.
962
+
You need to make the call which one to use. Passing a `context.Context` is more standard, on the other hand you then need to call `log.FromContext` everywhere.
963
+
964
+
As `logger.V` calls are cumulative, i.e. `logger.V(2).V(3)` results in `logger.V(5)`, a logger should be passed around with no verbosity level set so that `logger.V(DEFAULT)` actually uses `DEFAULT` verbosity level.
965
+
880
966
## Submitting Changes
881
967
882
968
Read the [llm-d organization contributing guide](https://github.com/llm-d/llm-d/blob/main/CONTRIBUTING.md)
0 commit comments