OpenLLMetry

OpenLLMetry is an open-source set of OpenTelemetry extensions that automatically trace calls to OpenAI, Anthropic, vector databases and agent frameworks. With a couple of lines of code you get spans for prompts, tokens and latency.

The data can go to Grafana, Datadog, Honeycomb or any compatible tool. It contributed ideas to the official OpenTelemetry generative AI conventions.

Use it if you want LLM tracing without adopting a new dashboard.