# LLMs Observability with Traceloop + qryn

[![](https://assets-global.website-files.com/643c87010b62a2009a1f57bf/655cc4769701a8661bb0fe69_illustration-p-1600.png align="center")](https://www.traceloop.com/openllmetry)

[OpenLLMetry](https://www.traceloop.com/openllmetry) is a set of extensions built on top of [OpenTelemetry](https://opentelemetry.io/) that gives you complete observability over your LLM application with minimal complexity.

Because it uses OpenTelemetry under the hood it can be connected to existing observability solutions such as our **polyglot stack**[qryn](https://qryn.dev) and [qryn.cloud](https://qryn.cloud) ⭐⭐⭐

## Step 1: Traceloop SDK Setup

[OpenLLMetry](https://www.traceloop.com/openllmetry) lets you easily trace prompts and embedding calls of **OpenAI** and can provide a complete view of your OpenAI application using traces and spans.

To get started, Install the [Traceloop SDK](https://pypi.org/project/traceloop-sdk/) and initialize it within your code.

### 🧠 OpenAI Example

Automatically log all calls to OpenAI, with prompts and completions

```python
import openai
from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import workflow

Traceloop.init(app_name="joke_generation_service")

@workflow(name="joke_creation")
def create_joke():
    completion = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": "Tell me a joke about opentelemetry"}],
    )

    return completion.choices[0].message.content
```

### 🦙 LLAMA Example

Automatically log all calls to LLAMA models, with prompts and completions

```python
import chromadb
import os
import openai

from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
from llama_index.vector_stores import ChromaVectorStore
from llama_index.storage.storage_context import StorageContext
from llama_index.embeddings import HuggingFaceEmbedding
from traceloop.sdk import Traceloop

openai.api_key = os.environ["OPENAI_API_KEY"]

# Initialize Traceloop
Traceloop.init()

chroma_client = chromadb.EphemeralClient()
chroma_collection = chroma_client.create_collection("quickstart")

# define embedding function
embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-base-en-v1.5")

# load documents
documents = SimpleDirectoryReader("./data/my_docs/").load_data()

# set up ChromaVectorStore and load in data
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
service_context = ServiceContext.from_defaults(embed_model=embed_model)
index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context, service_context=service_context
)

# Query Data
query_engine = index.as_query_engine()
response = query_engine.query("Summarize the documents in context")
```

For more information refer to the [Traceloop SDK Documentation](https://www.traceloop.com/docs/openllmetry/introduction)

## Step 2: Grafana Agent Sender

Configure a [Grafana Agent](https://grafana.com/docs/agent/latest/static/set-up/install/) instance to feed **Traceloop** traces into **qryn** / **qryn.cloud**

```yaml
traces:
  configs:
    - name: default
      remote_write:
        - endpoint: <Gigapipe qryn.cloud endpoint>:443
          basic_auth
            username: <Gigapipe qryn X-API-Key>
            password: <Gigapipe qryn X-API-Secret>
      receivers:
        otlp:
          protocols:
            grpc:
            
/* Environment Variable for your local app with Traceloop */
TRACELOOP_BASE_URL=http://<grafana-agent-hostname>:4318
```

**<mark>That's it! You're now ready to explore your LLMs activity using qryn</mark>**

You can immediately get started with some popular examples:

#### 👉 [Trace prompts and completions](https://github.com/traceloop/openllmetry/blob/main/packages/sample-app/sample_app/openai_streaming.py)

Call OpenAI and see prompts, completions, and token usage for your call.

👉 [Trace your RAG retrieval pipeline](https://github.com/traceloop/openllmetry/blob/main/packages/sample-app/sample_app/chroma_app.py)

Build a RAG pipeline with Chroma and OpenAI. See vectors returned from Chroma, full prompt in OpenAI and responses

### **Are you Ready?**

Signup for a free account on [**qryn.cloud**](https://qryn.cloud/) or install our [**oss stack**](https://qryn.dev/) on-premise ⭐

[![](https://user-images.githubusercontent.com/1423657/218818279-3efff74f-0191-498a-bdc4-f2650c9d3b49.gif align="left")](https://qryn.cloud/)