You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When using the Langfuse integration, time to first token metrics are not available (the report I received mentioned Azure OpenAI but this might apply to all LLMs).
Langfuse supports time to first token via the “completionStartedAt” attribute on generations.
To Reproduce
Use Langfuse integration with AzureOpenAI in Haystack.
Describe your environment (please complete the following information):
Reported by user. Can try to get details on this if helpful.
The text was updated successfully, but these errors were encountered:
This is a bit vague as it was reported by a user but I assume the attribute is not set at all and thus it could be sufficiently detailed.
I think this would be a great addition to the integration.
marcklingen
changed the title
Langfuse integration: Missing time to first token for AzureOpenAI
Langfuse integration: completionStartedAt is not logged, required for streaming metrics like TTFT and TPM/TPS
Dec 24, 2024
Describe the bug
When using the Langfuse integration, time to first token metrics are not available (the report I received mentioned Azure OpenAI but this might apply to all LLMs).
Langfuse supports time to first token via the “completionStartedAt” attribute on generations.
To Reproduce
Use Langfuse integration with AzureOpenAI in Haystack.
Describe your environment (please complete the following information):
Reported by user. Can try to get details on this if helpful.
The text was updated successfully, but these errors were encountered: