Monitor Server using victoriametrics and Grafana

Simple setup to monitor my machine using

node_exporter: Exposes my machine metrics at http://localhost:9100/metrics

vmagent: Scrapes the metrics exposed at /metrics at defined interval and sends it to victoriametrics

victoriametrics: It is a single node set up, a tsdb storage.

grafana: Visual representation of metrics used for analysing and alerting. It is queried using PromQl.

Setup

Node exporter

• wget https://github.com/prometheus/node_exporter/releases/download/v1.3.1/node_exporter-1.3.1.linux-amd64.tar.gz
• start: ./node_exporter
• verify: curl http://localhost:9100/metrics

VictoriaMetrics:

• start:

sudo docker run -it --net=host --rm -v `pwd`/victoria-metrics-data:/victoria-metrics-data -p 8428:8428 victoriametrics/victoria-metrics:latest

Vmagent

• download and make it executable binary:

wget https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.81.2/vmutils-linux-amd64-v1.81.2.tar.gz

• start scraping:

vmagent-prod -promscrape.config=prometheus.yml  -remoteWrite.url="https://localhost:8428/api/v1/write" -promscrape.config.strictParse=false

prometheus.yml

  #The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'local_prometheus'
    #Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s
    static_configs:
      - targets: ['127.0.0.1:9100']

Grafana

Here is how you can monitor by exploring your system metrics.

• start:

docker run -d -p 3000:3000 grafana/grafana-oss:latest

• Login to dashboard at http://localhost:3000.Add endpooint in data source i.e http://localhost:8428, because VM and Grafana are running on the same server