Configure extra log fields for SDK workflow and activity logs

[timofurrer]

Is your feature request related to a problem? Please describe.

I want to be able to easily attach business-related log fields to the log messages the SDK logs internally for workflows and activities.

One of such logs is

sdk-go/internal/internal_task_handlers.go

Line 2271 in 6e75f6d

ath.logger.Error("Activity error.",

The business-related values I want to attach are only known during workflow and activity creation and NOT during worker initialization.

Describe the solution you'd like

Configuring them via StartWorkflowOptions and ActivityOptions seems desirable.

Describe alternatives you've considered

I've looked into using Interceptors, but the problem with them is that I have to register them at worker initialization and don't know the values at that point. I've looked into using custom search attributes or memos - which both would work but only exist of Workflows and not Activities. I also haven't found a way to easily retrieve the search attributes / memos of the workflow given an activity context.

The elaborate on the search attributes / memos approach: I basically would register an interceptor that reads the search attributes / memos from the workflow context and create a new logger with them that is returned.

What I want to avoid is having to query that data from within the activity interceptor from an "external" system.

Additional context
Add any other context or screenshots about the feature request here.

[cretz]

but the problem with them is that I have to register them at worker initialization and don't know the values at that point

You don't need to know the values at that point, both activity and workflow outbound interceptors have GetLogger you can customize when called. https://github.com/temporalio/samples-go/tree/main/interceptor shows how to write the interceptor, but you can use the workflow/activity context in the interceptor to get workflow/activity-specific things like memo (or values via context propagation which may make more sense).

Does this help?

[timofurrer]

You don't need to know the values at that point, both activity and workflow outbound interceptors have `GetLogger` you can customize when called. https://github.com/temporalio/samples-go/tree/main/interceptor shows how to write the interceptor, but you can use the workflow/activity context in the interceptor to get workflow/activity-specific things like memo (or values via context propagation which may make more sense).

Are context values being persisted across activity / workflow retries and such?

[timofurrer]

@cretz while working on tracing for the SDK I implemented a context propagator - is that would I can use to persist the context values and use it for the logger?

[timofurrer]

@cretz I'm still struggling to get this working with what you suggested. I've implemented a context propagator that records my "values" into the contexts. I've also implemented a worker interceptor that, through the chain of the other interceptor implements GetLogger(). Something along the lines of this:

package server

import (
	"context"

	"gitlab.com/gitlab-org/cluster-integration/gitlab-agent/v17/internal/tool/logz"
	"go.temporal.io/sdk/interceptor"
	tlog "go.temporal.io/sdk/log"
	"go.temporal.io/sdk/workflow"
)

type loggerInterceptor struct {
	interceptor.WorkerInterceptorBase
}

func newLoggerInterceptor() *loggerInterceptor {
	return &loggerInterceptor{}
}

func (w *loggerInterceptor) InterceptActivity(ctx context.Context, next interceptor.ActivityInboundInterceptor) interceptor.ActivityInboundInterceptor {
	i := &loggerActivityInboundInterceptor{root: w}
	i.Next = next
	return i
}

type loggerActivityInboundInterceptor struct {
	interceptor.ActivityInboundInterceptorBase
	root *loggerInterceptor
}

func (a *loggerActivityInboundInterceptor) Init(outbound interceptor.ActivityOutboundInterceptor) error {
	i := &loggerActivityOutboundInterceptor{root: a.root}
	i.Next = outbound
	return a.Next.Init(i)
}

type loggerActivityOutboundInterceptor struct {
	interceptor.ActivityOutboundInterceptorBase
	root *loggerInterceptor
}

func (a *loggerActivityOutboundInterceptor) GetLogger(ctx context.Context) tlog.Logger {
	logger := a.Next.GetLogger(ctx)

	values, ok := ctx.Value(contextPropagatorKey).(contextPropagatorValues)
	if !ok {
		logger.Warn("the passed context to the interceptor didn't contain the expected context propagator values")
		return logger
	}

	logger = tlog.With(logger,
		logz.ProjectIDN(values.ProjectID), logz.EventID(values.EventID), logz.EventType(values.EventType))
	return logger
}

func (w *loggerInterceptor) InterceptWorkflow(ctx workflow.Context, next interceptor.WorkflowInboundInterceptor) interceptor.WorkflowInboundInterceptor {
	i := &loggerWorkflowInboundInterceptor{root: w}
	i.Next = next
	return i
}

type loggerWorkflowInboundInterceptor struct {
	interceptor.WorkflowInboundInterceptorBase
	root *loggerInterceptor
}

func (w *loggerWorkflowInboundInterceptor) Init(outbound interceptor.WorkflowOutboundInterceptor) error {
	i := &loggerWorkflowOutboundInterceptor{root: w.root}
	i.Next = outbound
	return w.Next.Init(i)
}

type loggerWorkflowOutboundInterceptor struct {
	interceptor.WorkflowOutboundInterceptorBase
	root *loggerInterceptor
}

func (w *loggerWorkflowOutboundInterceptor) GetLogger(ctx workflow.Context) tlog.Logger {
	logger := w.Next.GetLogger(ctx)

	values, ok := ctx.Value(contextPropagatorKey).(contextPropagatorValues)
	if !ok {
		logger.Warn("the passed context to the interceptor didn't contain the expected context propagator values")
		return logger
	}

	logger = tlog.With(logger,
		logz.ProjectIDN(values.ProjectID), logz.EventID(values.EventID), logz.EventType(values.EventType))
	return logger
}

However, I'm only seeing "my" logger, with my fields for the logs I'm doing in the workflows and activities. That's desired, but all SDK internal logs are missing these fields - it's seems like it's not getting the logger through my GetLogger() implementations.

Some example log calls I expect to have my fields:

• sdk-go/internal/internal_event_handlers.go

  Line 775 in 4fb50dc

  wc.logger.Debug("ExecuteActivity",

• sdk-go/internal/internal_event_handlers.go

  Line 625 in 4fb50dc

  wc.logger.Debug("ExecuteChildWorkflow",

@cretz are we facing some variation of #829 ?

cc @ktenzer

[cretz]

Ah, the interceptor and customizations are for when you call GetLogger in your activity or workflow. Yes, this is a duplicate of #829 if you're trying to customize internal logger use on a per-workflow basis. For those two log situations, you should consider using your own log statements in your own interceptor instead of relying on debug logs from the internals of the SDK.

[timofurrer]

For those two log situations, you should consider using your own log statements in your own interceptor instead of relying on debug logs from the internals of the SDK.

@cretz the two last examples I've referenced were indeed Debug logs, but the one I've referenced initially is an error log. (

sdk-go/internal/internal_task_handlers.go

Line 2271 in 6e75f6d

ath.logger.Error("Activity error.",

) Those are especially useful if they would have the proper log fields. Yes, I may be able to use an interceptor to also log my own message on errors or maybe even just log the errors in the Workflows / Activities before returning one - but then we always have duplicate error logs which is not optimal. In addition, you may just introduce more logs to the SDK with different levels and "pollute" the app logs.

Having a solution for #829 would help I suppose.