[appo11y/spanmetrics] Allow customization of span_metrics_prefilter

[a-abella]

Hello,

It would be useful to allow users to add additional filter statements to the SpanMetrics prefilter. The following filter scenarios are ones I'm particularly interested in:

• Drop client/producer spans; i.e., only generate metrics for server/consumer spans
• Drop spans with duration > N; useful for unterminated spans caused by upstream application span close failure--in such scenarios you can get spans with a false duration of hours/days/weeks that negatively affect aggregations like avg

Don't want to set these filters at the applicationObservability.traces.filters.span level because we still want to capture client traces to Tempo, just don't want the cardinality hit of millions of client span metrics.

Current Workaround:

Dropping client/consumer spans with relabeling in the prometheus destination config. This works to prevent ingestion but receiver still takes the compute and memory hit of generating all those dropped metrics.

Using applicationObservability.traces.transforms.span to clamp the duration on spans to a hardcoded maximum--not ideal as it affects the spans sent to Tempo and still negatively affects generated metric values with nonsense values, just smaller nonsense values.

Thanks for considering