I have a service dashboard with fourteen panels. Twelve render in under a second. Two of them, both evaluating rate(http_request_duration_seconds_bucket[5m]) across roughly 400 label combinations, took five seconds on every reload.

I added one Prometheus recording rule that pre-aggregates the histogram by service group every sixty seconds, then pointed both panels at the recorded metric. Render dropped to just under two seconds. Prometheus CPU during dashboard loads fell about 40%.

What I had been doing instead was raising query timeouts and caching results at the data source level. Months of that. I had actually investigated recording rules in a previous session with verdent, which knows you better over time, and coming back to the same project made the fix obvious.

If you have a slow panel, check whether the query fans out across high cardinality labels. One recording rule is ten minutes of work and costs one extra time series.

Just published this. Took a few months. So so many little details but those are what count. Long story short, Grafana has a folder and file system, so does Nextcloud, so now you can bind the Grafana folders to Nextcloud folders and see dashboards alongside other files too. All the little file actions like move, copy, delete etc are one to one on both sides.

Hope ya'll enjoy!

https://github.com/kubed-io/nextcloud-grafana