[ENHANCEMENT]: Field masking

[dpolac]

Is your feature request related to a problem? Please describe.

I have secrets (passwords, tokens, api keys) and personal data (names, phone numbers) in my spans. They are deep inside of input and output objects.

Phoenix offers two types of span masking:

• masking span input/output: https://arize.com/docs/phoenix/tracing/how-to-tracing/advanced/masking-span-attributes
• filtering entire span: https://arize.com/docs/phoenix/tracing/how-to-tracing/advanced/modifying-spans

However, neither of them achieve my goal of only redacting selected input/output keys.

Describe the solution you'd like

I'd like to have masking configuration that I can apply on the self-hosted server configuration.
Example:

mask_keys:
  given_name: [REDACTED]
  address: [REDACTED]
  api_key: xxxxxxxx
  password: *hiden*
mask_values:
  "[0-9\-]{12,15}": [REDACTED ID NUMBER]

So that the span input:

{
  "user_info": {
    "id": "241",
    "given_name": "John"
  },
  "api_key": "275619824jh2416aa612="
  "value": "1234-6789-123",
  "result": "success"
}

would result in:

{
  "user_info": {
    "id": "241",
    "given_name": "[REDACTED]"
  },
  "api_key": "xxxxxxxx"
  "value": "[REDACTED ID NUMBER]",
  "result": "success"
}

[hawat]

It's possible routing thru collector and making changes them, but sometimes additional step is overcolplication.
As @dpolac mention - it should be easy "out-of-the-box" functionality.

[mikeldking]

hey @dpolac @hawat - thanks for raising this important issue. I think in general it's pretty difficult to predict where and when fields should be redacted. In general I'd probably advise using OTEL building blocks like SpanProcessors or custom exporters to redact the values before export.

Did you have any particular ergonomic changes on the OI side?

[dpolac]

I'd probably advise using OTEL building blocks like SpanProcessors or custom exporters to redact the values before export.

@mikeldking I tried that following phoenix documentation. However, there is no public api for span object to modify the span fields. I can only remove entire span, apparently.

If there is a way of masking a single input/output field by name, then I would appreciate an example implementation of such SpanProcessor. The documentation does not provide that.

For now, we are using otel collector to achieve masking. But I'd rather have it as few lines of code rather than adding a new collector server that needs to be maintained.