Obfuscating sensitive attribute values with an Instrumentation extension

[lalitmathwani]

Hello all,

Please do excuse a few noobie questions below :) - really grateful for any guidance!

We are looking at creating a custom extension that would allow us to look for sensitive token data in span attributes (by a match against certain regexes) and mask them upfront before exporting to the collector.

We are definitely going to evaluate the collector side redaction processor (https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/redactionprocessor) which is in the works - if and when it does get released.

Going by https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/examples/extension#i-want-to-edit-some-attributes-that-dont-depend-on-any-db-connection-instance - it seems like for editing attributes in a span, without injecting any special advice and irrespective of the source, one should evaluate creating a custom SpanProcessor. Is that understanding correct?

If that's the case, then I was expecting that the regex-match-and-obfuscate would be implemented on span end ..with onEnd we seem to receive a handle to a ReadableSpan that's immutable (

opentelemetry-java-instrumentation/examples/extension/src/main/java/com/example/javaagent/DemoSpanProcessor.java

Line 35 in 902748c

public void onEnd(ReadableSpan span) {}

). We wouldnt be able to modify the ReadableSpan and set any attribute values at this point, unless I am missing something :(

Is there any other approach to modifying any of the attributes within the current span with the intent of masking sensitive parts of the attribute value?

Thanks a lot in advance!

[trask]

hi @lalitmathwani!

you'll want to write a AutoConfigurationCustomizerProvider that registers a span exporter customizer using AutoConfigurationCustomizer.addSpanExporterCustomizer(), and your span exporter customizer can wrap the real exporter with your own SpanExporter class that delegates to the real exporter.

and then in your SpanExporter you can wrap the SpanData (since it's immutable) in your own subclass of DelegatingSpanData which does the obfuscation, and pass your subclass down to the real exporter

[lalitmathwani]

Thanks a bunch @trask - that does help a lot :) ! So these obfuscations would apply during the export and the overall approach would be I guess somewhat similar to: https://github.com/open-telemetry/opentelemetry-java/blob/1e073fcff20697fd5f2eb39bd6246d06a1231089/sdk-extensions/autoconfigure/src/testFullConfig/java/io/opentelemetry/sdk/autoconfigure/SpanExporterCustomizer.java#L30 ..

I do see a reference here to the SPI extension option - https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/agent-config.md#sdk-autoconfiguration.

Trying to understand where the extension itself will live and become available to the instrumentation agent..does the SPI extension need to be a part of https://github.com/open-telemetry/opentelemetry-java/blob/main/sdk-extensions/autoconfigure or can it live within an opentelemetry-java-instrumentation agent extension ? Again, apologies if the answer is already documented in a ReadME that I may have missed ..

[trask]

Trying to understand where the extension itself will live and become available to the instrumentation agent.. can it live within an opentelemetry-java-instrumentation agent extension

yes, your AutoConfigurationCustomizerProvider implementation will live in your agent extension and the autoconfiguration mechanism will find it via SPI (META-INF/services/... file)

[lalitmathwani]

Got it 👍 Thanks again @trask!

[pajohri]

Take a look into this project https://github.com/pajohri/jdotm