Docs
Node
Node Monitoring
Loki Log Manager

Loki Log Management

The following is a guide outlining the steps to setup Loki for log management of a Tangle node. If you do not have Tangle node setup yet, please review the Tangle Node Quickstart setup guide here (opens in a new tab).

In this guide we will configure the following modules to scrape metrics from the running Tangle node.

Let's first start by downloading the latest releases of the above mentioned modules (Loki, Promtail download pages).

This guide assumes the user has root access to the machine running the Tangle node, and following the below steps inside that machine.

1. Download Loki

AMD version:

AMD
curl -O -L "https://github.com/grafana/loki/releases/download/v2.7.0/loki-darwin-amd64.zip"

ARM version:

ARM
curl -O -L "https://github.com/grafana/loki/releases/download/v2.7.0/loki-darwin-arm64.zip"

2. Download Promtail

AMD version:

AMD
curl -O -L "https://github.com/grafana/loki/releases/download/v2.7.0/promtail-darwin-amd64.zip"

ARM version:

ARM
curl -O -L "https://github.com/grafana/loki/releases/download/v2.7.0/promtail-darwin-arm64.zip"

3. Extract the Downloaded Files:

unzip
unzip "loki-linux-amd64.zip" &&
unzip "promtail-linux-amd64.zip"

4. Copy the Extracted Files into /usr/local/bin:

cp
sudo cp loki-linux-amd64 /usr/local/bin/ &&
sudo cp promtail-linux-amd64 /usr/local/bin/

5. Create Dedicated Users:

Now we want to create dedicated users for each of the modules we have installed:

useradd
sudo useradd --no-create-home --shell /usr/sbin/nologin loki &&
sudo useradd --no-create-home --shell /usr/sbin/nologin promtail

6. Create Directories for loki, and promtail:

mkdir
sudo mkdir /etc/loki &&
sudo mkdir /etc/promtail

7. Change the Ownership for all Directories:

We need to give our user permissions to access these directories:

chown
sudo chown loki:loki /usr/local/bin/loki-linux-amd64 &&
sudo chown promtail:promtail /usr/local/bin/promtail-linux-amd64

9. Finally, let's clean up these directories:

rm
rm -rf ./loki-linux-amd64* &&
rm -rf ./promtail-linux-amd64*

The next series of steps will be configuring each service.

Configuration

For configuration examples, see Monitoring (opens in a new tab).

Loki

Loki's configuration details what ports to listen to, how to store the logs, and other configuration options. There are many other config options for Loki, read more. (opens in a new tab)

Let’s create the file:

nano
sudo touch /etc/loki/config.yml
sudo nano /etc/loki/config.yml
config.yaml
auth_enabled: false
 
server:
  http_listen_port: 3100
  grpc_listen_port: 9096
 
ingester:
  lifecycler:
    address: 127.0.0.1
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
    final_sleep: 0s
  chunk_idle_period: 5m
  chunk_retain_period: 30s
  max_transfer_retries: 0
 
schema_config:
  configs:
    - from: 2020-10-24
      store: boltdb-shipper
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 168h
 
 
storage_config:
  boltdb:
    directory: /data/loki/index
 
  filesystem:
    directory: /data/loki/chunks
 
limits_config:
  enforce_metric_name: false
  reject_old_samples: true
  reject_old_samples_max_age: 168h
 
chunk_store_config:
  max_look_back_period: 0s
 
table_manager:
  retention_deletes_enabled: false
  retention_period: 0

Promtail

The Promtail configuration details what logs to send to Loki. In the below configuration we are indicating to send the logs to Loki from the /var/log/dkg directory. This directory can be changed based on what logs you want to pick up. There are many other config options for Promtail, refer to the Promtail documentation (opens in a new tab)

Let’s create the file:

nano
sudo touch /etc/promtail/config.yml
sudo nano /etc/promtail/config.yml
config.yaml
server:
  http_listen_port: 9080
  grpc_listen_port: 0
 
positions:
  filename: /data/loki/positions.yaml
 
clients:
  - url: http://localhost:3100/loki/api/v1/push
 
scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs
      __path__: /var/log/dkg/*log

Service Setup

Loki

Create and open the Loki service file:

loki.service
sudo tee /etc/systemd/system/loki.service > /dev/null << EOF
[Unit]
  Description=Loki Service
  Wants=network-online.target
  After=network-online.target
 
[Service]
  User=loki
  Group=loki
  Type=simple
  ExecStart=/usr/local/bin/loki-linux-amd64 -config.file /etc/loki/config.yml
 
[Install]
WantedBy=multi-user.target
EOF

Promtail

Create and open the Promtail service file:

promtail.service
sudo tee /etc/systemd/system/promtail.service > /dev/null << EOF
[Unit]
  Description=Promtail Service
  Wants=network-online.target
  After=network-online.target
 
[Service]
  User=promtail
  Group=promtail
  Type=simple
  ExecStart=/usr/local/bin/promtail-linux-amd64 -config.file /etc/promtail/config.yml
 
[Install]
WantedBy=multi-user.target
EOF

Great! You have now configured all the services needed to run Loki.

Starting the Services

Launch a daemon reload to take the services into account in systemd:

daemon-reload
sudo systemctl daemon-reload

Next, we will want to start each service:

start service
sudo systemctl start loki.service &&
sudo systemctl start promtail.service

And check that they are working fine, one by one:

loki:

status
systemctl status loki.service

promtail:

status
systemctl status promtail.service

If everything is working adequately, activate the services!

enable
sudo systemctl enable loki.service &&
sudo systemctl enable promtail.service

Amazing! You have now successfully configured Loki for log management. Check out the Grafana documentation to create a Loki log dashboard!

Grafana DashboardStart Validating