# 🦟 Fly.io Observability with qryn

[![](https://cdn.hashnode.com/res/hashnode/image/upload/v1705710706485/97499114-600f-4912-a883-69774267b4b4.png align="left")](https://fly.io)

> Fly is a platform for running full stack apps and databases close to your users. We’ve been hammering on this thing since 2017, and we think it’s pretty great.

*Lots of smart people love Fly.io and run their apps on this great platform!*

Each **Fly Apps** will produce logs, helpful if not essential for a variety of use cases - *debugging, tracking, collating, correlating, coalescing, and condensing* the happenings of your running code into useful bits of human-parsable information.

But since [Fly.io](http://Fly.io) *doesn’t keep your logs around forever* to make the best of them we can store and explore data using services such as [qryn](https://qryn.dev) and [qryn.cloud](https://qryn.cloud), without loosing any of the **Grafana** compatibility and Dashboards Fly.io offers by default.

*<mark>How can we do that</mark>?* Luckily for us fly.io is always super elegant and very generously [routes all Firecracker VM logs through NATS](https://fly.io/blog/shipping-logs/), ready to be picked up and processed.

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">👊 Thanks to our good friend <a target="_blank" rel="noopener noreferrer nofollow" href="https://twitter.com/dan_the_goodman?lang=en" style="pointer-events: none">Dan The Goodman</a> from <a target="_blank" rel="noopener noreferrer nofollow" href="https://www.tangia.co/" style="pointer-events: none">Tangia</a> for suggesting this smart approach and for inspiring this guide! Checkout his projects!</div>
</div>

### 🎈 Fly higher with qryn

To get logs all you need is an app that acts as a NATS client, reads the logs, and ships them somewhere. **Vector** can do just that and since it supports **qryn** natively through the **Loki** and **Prometheus** sinks, *it works out of the box with our example.*

## **Using the Log Shipper**

For our example we'll use use the [fly-log-shipper example](https://github.com/superfly/fly-log-shipper), extended for **qryn**.

The fly **NATS** log stream is scoped to your user organization, which means the Fly Log Shipper collects logs from *all* your applications at once. *Very practical.*

Here's a quick example using a custom **qryn** sink for shipping **logs** and **metrics**:

```bash
# Make a directory for qryn log shipper
mkdir logshippper
cd logshippper

# I chose not to deploy yet
fly launch --image qxip/fly-log-shipper:latest

# Set secrets to enable qryn integration
fly secrets set ORG=personal
fly secrets set ACCESS_TOKEN=$(fly auth token)
fly secrets set QRYN_URL=<qryn API URL>
fly secrets set QRYN_USERNAME=<qryn API user>
fly secrets set QRYN_PASSWORD=<qryn API token>
```

Before launching your application, you should edit the generated `fly.toml` file and delete the entire `[[services]]` section. Replace it with this:

```ini
[[services]]
  http_checks = []
  internal_port = 8686
```

Once ready deploy the **qryn log shipper** application to your stack:

```bash
fly deploy
```

You’ll soon start to see **logs** appear from all of your apps:

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1705672682151/3f34fbda-4250-4610-8fdb-dde5fcb3336a.png align="left")

> *NOTE: shipping logs to qryn does* ***not*** *interrupt the native logging feature!*

***<mark>What's happening under the hood?</mark>***

* **Vector** receives all the organization logs through **NATS**
    
* **NATS** sourced logs are parsed and labeled for the Loki sink
    
* **Vector** ships parsed logs to a Loki sink pointed at **qryn** or **qryn.cloud**
    

### Fly Metrics

Fly.io produces granular metrics for all running applications using an internal shared Prometheus and Grafana service. Those seems to be long-retention but you might still want to pull them into your own stack for further processing and analysis.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1705749285513/1f25ac41-4625-47b8-a150-4678f7ea0ed3.png align="center")

To scrape those **metrics into our custom solution** we can use an extension to the existing Vector configuration to scrape all our organization timeseries using the same Fly.io access token used for our NATS connections, query matching all apps.

```ini
[sources.flyio_metrics]
  type = "prometheus_scrape"
  endpoints = [ "https://api.fly.io/prometheus/personal/federate" ]
  auth.strategy = 'bearer'
  auth.token = "${ACCESS_TOKEN?}"
  query.match = ['{app=~".+"}']
```

The resulting metrics are shipped to **qryn** using a standard **remote\_write** API. Easy!

*To reproduce the full experience, export & recycle Fly's Grafana dashboards.*

### OLAP-in-One

**That's it!** You're now ready to query your logs using **Loki/LogQL** and your metrics using **Prometheus/PromQL** and ready to receive Traces and Continuous Profiling events from within your App code using **Tempo** and **Pyroscope** compatibility.

*Using the same process you can ship logs, metrics or traces from any stack.*

### Get Polyglot

Tired of unnecessary complexity? Join our **polyglot observability** journey today and regain control of your data with [qryn oss](https://qryn.dev) or [qryn.cloud](https://qryn.cloud) for all of your **Logs, Metrics, Traces** and **Profiling** while saving a ton in cost and resources and without forcing your teams to learn new query languages and without having to change your tools.

Which stack would you like to see next? Let us know in the comments.

[![](https://github.com/metrico/qryn-docs/assets/1423657/a5164f98-d3ed-4638-afe5-c87d252c74af align="left")](https://qryn.cloud)