Prometheus: relabel your scrape_config
Prometheus labels every data point — the most well-known example of a label is (probably)
Take a look at this query result (query:
So what does this tell me?
I queried for the "up" metric and filtered it for "prometheus" — yay. The "1" says, my service is alive. So far so gut.
Since we are in the process of running a few Prometheus servers (in federation), each of those metrics will report back with
instance="127.0.0.1:9090" (along with other labels of course).
While this works, I'm not a computer. If the "instance" reported an FQDN or some other readable name, it would make any dashboard or alert more approachable. Or readable, if you will.
The instance label
instance is a standard field used in various Grafana dashboards out there. Dashboards often use the value in
instance to provide you with a dropdown list of (well) instances (or nodes) to select from.
To not end up with a dropdown full of
127.0.0.1:9000, here is a snippet on how to work with labels to make life a little easier.
Consider the following
- job_name: "prometheus" metrics_path: "/metrics" static_configs: - targets: - "127.0.0.1:9090"
It produces the result above.
Now, extend it slightly to include a
name and relabel the
instance field with it:
- job_name: "prometheus" metrics_path: "/metrics" relabel_configs: - source_labels: [name] target_label: instance static_configs: - targets: - "127.0.0.1:9090" labels: name: my-prometheus.example.org
Now "instance" is set to something I can grok by glancing over it. Which makes me happy.
Thanks for following along!
The author does not allow comments to this entry