Persist and restore trace headers in workqueue

[mattmoor]

I left a TODO here tagging @tcnghia because he worked this magic for Pub/Sub:

terraform-infra-common/pkg/workqueue/gcs/gcs.go

Lines 70 to 72 in c79371c

	writer.Metadata = map[string]string{
	// TODO(nghia): Extract and persist things like trace headers here.
	}

I'm guessing it's not a ton of work, but we need to extract the relevant headers and encode them above.

Then restore them here:

terraform-infra-common/pkg/workqueue/gcs/gcs.go

Lines 250 to 253 in c79371c

	// Context implements workqueue.OwnedInProgressKey.
	func (o *inProgressKey) Context() context.Context {
	return o.ownerCtx
	}

I also need to fix the dispatcher to properly use this Context() method, which is an oversight on my part, which I'll correct now.

[tcnghia]

For pubsub, this was the PR #374 . Newer PubSub client sets a message attribute named googclient-traceparent which becomes a header with the same name. In our metrics handler we just restore it like this https://github.com/chainguard-dev/terraform-infra-common/blob/main/pkg/httpmetrics/metrics.go#L135

For work queue, since we control the request dispatching, we don't need to do stuffs in a metrics handler, but just set the traceparent header directly.

[mattmoor]

Does that do what we expect transitively? It feels like that would have a relatively shallow effect? 🤔