adaptive-metrics

v2026.09.24

Cut Grafana Cloud Metrics cost by shrinking active-series count with Adaptive Metrics aggregation rules — auto-recommendations from query history, custom exact/regex rules, label-drop config, unused-metric detection, and Alloy remote_write fallback. Use when investigating a high Mimir/Grafana Cloud bill, hunting high-cardinality labels (`pod_uid`, `service_instance_id`, `version`), pre-aggregating counters/gauges, dropping unused metrics, or measuring `grafanacloud_instance_active_series` before/after — even when the user says "reduce cardinality", "too many series", "metrics spend", "active series count is exploding", or "drop the version label" without naming Adaptive Metrics.

GitHub
安装命令
npx skhub add grafana/adaptive-metrics
Markdown
SKILL.md

Grafana Cloud Adaptive Metrics

Docs: https://grafana.com/docs/grafana-cloud/cost-management-and-billing/reduce-costs/metrics-costs/adaptive-metrics.md

Aggregation rules that pre-shrink high-cardinality metrics before storage — directly reduces active-series billing.

Prerequisites

  • Grafana Cloud Metrics plan (any paid tier)
  • API key with metrics:write (for the Adaptive Metrics API — adaptive-metrics.grafana.net, Bearer auth)
  • For the verification queries: the metrics query endpoint (prometheus-prod-XX.grafana.net) uses HTTP basic auth — <metrics_user> (numeric stack/instance ID) plus a token with metrics:read — not the Bearer key
  • Access to Home → Adaptive Metrics in the Cloud portal

Common Workflows

1. Review + apply auto-recommendations

# 1. Pull the recommendation list (sorted by series-reduction impact)
curl -s -H "Authorization: Bearer <KEY>" \
  "https://adaptive-metrics.grafana.net/api/v1/recommendations" \
  | jq '.recommendations[] | {metric_name, current_series, projected_series, estimated_reduction_percent}'

# 2. Capture the baseline series count for the target metric
#    (metrics query endpoint = basic auth, not the Bearer key)
curl -s -u "<metrics_user>:<metrics_token>" \
  "https://prometheus-prod-XX.grafana.net/api/prom/api/v1/query?query=count({__name__=\"process_cpu_seconds_total\"})" \
  | jq '.data.result[0].value[1]'   # → e.g. "12480"

# 3. Apply the recommendation (or click Apply in the UI)
curl -s -X POST -H "Authorization: Bearer <KEY>" \
  "https://adaptive-metrics.grafana.net/api/v1/recommendations/<ID>/apply"

# 4. Wait ~5 min. Verify — re-run the count query; expect a large drop.
#    Also check the saving metric:
#      grafanacloud_instance_active_series_dropped_by_aggregation_rules

Rollback — delete the rule:

curl -s -H "Authorization: Bearer <KEY>" \
  "https://adaptive-metrics.grafana.net/api/v1/rules" | jq '.rules[] | {id, metric_name}'
curl -s -X DELETE -H "Authorization: Bearer <KEY>" \
  "https://adaptive-metrics.grafana.net/api/v1/rules/<RULE_ID>"
# Or in the UI: Rules → row → Disable

2. Hand-write a custom rule

# 1. Sanity-check the metric is not used WITH that label in dashboards/alerts
grep -r 'process_cpu_seconds_total' dashboards/ alerts/ | grep -E 'version|go_version'
# Expect no hits → safe to drop.

# 2. Create the rule
curl -s -X POST -H "Authorization: Bearer <KEY>" -H "Content-Type: application/json" \
  "https://adaptive-metrics.grafana.net/api/v1/rules" \
  -d '{"rules":[{"metric_name":"process_cpu_seconds_total","match_type":"MATCH_TYPE_EXACT",
                 "drop_labels":["version","go_version"],
                 "aggregations":[{"type":"AGGREGATION_TYPE_SUM"}]}]}'

# 3. Verify — same count() query as above; series count should drop within 5 min.

Full payloads (regex match, aggregation types, all caveats): references/api.md.

3. Drop unused metrics entirely

# 1. List unused metrics
curl -s -H "Authorization: Bearer <KEY>" \
  "https://adaptive-metrics.grafana.net/api/v1/usage-analysis?filter=unused" | \
  jq '.metrics[] | {metric_name, series_count, last_queried}'

# 2. Confirm not referenced in dashboards / alerts / recording rules
grep -r '<METRIC_NAME>' dashboards/ alerts/ recording-rules/

# 3. Add a write_relabel_config drop in Alloy (full block in references/api.md)
#    Reload Alloy: curl -X POST http://localhost:12345/-/reload

# 4. Verify — the metric should no longer appear in series counts after ~10 min
curl -s -u "<metrics_user>:<metrics_token>" \
  'https://prometheus-prod-XX.grafana.net/api/prom/api/v1/label/__name__/values' | jq '.data | index("<METRIC_NAME>")'  # → null

Measure the impact

# Total active series (billed unit)
grafanacloud_instance_active_series

# Series specifically dropped by Adaptive Metrics rules
grafanacloud_instance_active_series_dropped_by_aggregation_rules

Rules take effect within ~5 minutes; full billing impact appears within an hour. The original high-cardinality samples keep flowing but the dropped labels no longer count toward billing.

Resources

发现
标签

此技能尚未发布标签。

版本
最新版本元数据

版本

v2026.09.24

发布时间

2026年9月24日

分类

未分类

许可证

Apache-2.0

源路径

skills/grafana-cloud/adaptive-metrics

默认分支

main

最新提交

1ccacf2

Tree SHA

f3d82e4