[Otel] Support for capturing trace.Span events

[vtfr]

Summary

Currently, adding events (via AddEvent) and errors (via RecordError) using Open Telemetry is a NOOP using the SentrySpanProcessor. To properly add them, it's necessary to resort to sentry's specific abstractions (such as CaptureMessage and CaptureException) passing an EventHint with a context containing a span, making the underlying code dependent on both OTel and sentry's API.

This becomes more evident when handling panics, since the default spanRecorder automatically recovers from panics, adds them as an event and re-panics. Making the use of sentry's Recover not only redundant but misleading, as the reported stacktrace will start at the repanic location, inside the spanRecorder, not the place where the panic itself happened. See #582.

Motivation

To improve the current integration, my feature request is to enhance the current SentrySpanProcessor to iterate over the OTel Span events and send them to sentry.

Current challenges

• For errors, Open Telemetry's default spanRecorder records all errors' metadata (name, kind, stacktrace strings), but keeps no record of the error itself. For reporting Stacktraces, it would be necessary to parse the stacktrace string into a []sentry.Exception, which can be challenging. There are libraries that already do the parsing, but I will keep this open for discussion.

Additional Context

Possible implementation:

func (ssp *sentrySpanProcessor) OnEnd(s otelSdkTrace.ReadOnlySpan) {
	...
	processEvents(sentrySpan, s)
	...
}

func processEvents(sentrySpan *sentry.Span, s otelSdkTrace.ReadOnlySpan) {
	ctx := sentrySpan.Context()

	hub := sentry.GetHubFromContext(ctx)
	if hub == nil {
		hub = sentry.CurrentHub()
	}

	for _, event := range s.Events() {
		sentryEvent := sentry.NewEvent()
		sentryEvent.Timestamp = event.Time

		switch event.Name {
		case semconv.ExceptionEventName:
			var exceptionType string
			var exceptionMessage string
			var exceptionStacktrace string

			for _, kv := range event.Attributes {
				switch kv.Key {
				case semconv.ExceptionTypeKey:
					exceptionType = kv.Value.Emit()
				case semconv.ExceptionMessageKey:
					exceptionMessage = kv.Value.Emit()
				case semconv.ExceptionStacktraceKey:
					exceptionStacktrace = kv.Value.Emit()
				}
			}

			// todo: Parse the exceptionStacktrace into an []sentry.Exception
			sentryEvent.Level = sentry.LevelError			
			sentryEvent.Message = exceptionMessage
			sentryEvent.Exception = []sentry.Exception{
				{
					Type:  exceptionType,
					Value: exceptionMessage,
				},
			}
		default:
			// todo: Check for attributes for setting the level
			sentryEvent.Level = sentry.LevelInfo
			sentryEvent.Message = event.Name
		}

		hub.Client().CaptureEvent(sentryEvent, &sentry.EventHint{
			Context: ctx,
		}, nil)
	}
}