Imagine you are attempting to decide why certainly one of your programs isn’t appearing smartly. The utility logs don’t display a selected factor and by the point you checked the server, the issue is long gone.

Most most probably, you possibly can wish to file efficiency for a given time period to look what metrics deteriorated at the device. But you wouldn’t have a devoted agent to care for that.

Some gear are ideal for appearing the server’s well being in genuine time, like HTOP and Glances. However, what differentiates Glances from different utilities is that it could file metrics to a far off agent whilst appearing them in genuine time.

You additionally want to proportion the ideas you be told with workforce contributors so that you could undergo those metrics as a time collection. That’s the place a database resolution like InfluxDB, which integrates smartly with many different methods, comes into play.

[ Related reading: How to analyze time-series data with Python and InfluxDB ]

Set up a snappy observability resolution

The objective of this instructional is to arrange an observability resolution as briefly as imaginable to seize information about the issue because it occurs.

You will learn to:

• Install Glances the use of a Python digital setting
• Install InfluxDB the use of a Podman container
• Connect InfluxDB and Glances to file historic metrics

Start by way of putting in those gear.

Install Glances and InfluxDB

Begin by way of putting in Glances at the device the place you need to seize the metrics.

At the time of writing this text, Glances has a minor factor that reasons it to crash when exporting metrics to InfluxDB. The downside has already been mounted within the upstream undertaking, but it surely has no longer been launched but. To make certain you’ll apply in conjunction with this case, set up Glances 3.2.5 (or more moderen) the use of a Python digital setting.

Create a brand new Python digital setting and turn on it:

$ sudo dnf set up python3-devel
$ python3 -m venv ~/virtualenv/glances
$ . ~/virtualenv/glances/bin/turn on
(glances) $ pip set up wheel

Install Glances into the digital setting:

(glances) $ pip set up --upgrade glances[all]==3.2.5
Collecting glances
  Downloading Glances-3.3.1.1-py3-none-any.whl (708 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 708.4/708.4 kB 6.7 MB/s eta 0:00:00
Collecting psutil>=5.6.7
  Downloading psutil-5.9.4-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (280 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 280.2/280.2 kB 9.7 MB/s eta 0:00:00
Collecting defusedxml
  Downloading defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)
Collecting packaging
  Using cached packaging-23.0-py3-none-any.whl (42 kB)
Collecting ujson>=5.4.0
  Downloading ujson-5.7.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (52 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 52.8/52.8 kB 6.7 MB/s eta 0:00:00
Installing gathered applications: ujson, psutil, packaging, defusedxml, glances
Successfully put in defusedxml-0.7.1 glances-3.3.1.1 packaging-23.0 psutil-5.9.4 ujson-5.7.0
...

Then set up the InfluxDB Python Jstomer, which permits Glances to export the metrics without delay to InfluxDB:

(glances) $ pip set up influxdb-client
Requirement already glad: six>=1.5 in ./virtualenv/glances/lib64/python3.11/site-packages (from python-dateutil>=2.5.3->influxdb-client) (1.16.0)
Collecting typing-extensions<5.0.0,>=4.1.1
  Using cached typing_extensions-4.5.0-py3-none-any.whl (27 kB)
Installing gathered applications: typing-extensions, reactivex, influxdb-client
Successfully put in influxdb-client-1.36.1 reactivex-4.0.4 typing-extensions-4.5.0

Now, pick out a device to retailer Glances metrics for later research and run InfluxDB the use of a container:

josevnz@server2:~$ mkdir -p information/influxdb
josevnz@server2:~$ podman run --name mydb --detach 
  --publish 8086:8086 
  --volume $PWD/information/influxdb:/var/lib/influxdb2:Z 
  influxdb:newest --reporting-disabled

87cd8aae2b9af65a1c10be530e699b5f54026d8ca2b66bd0890a49a30c643853

josevnz@server2:~$ podman logs mydb
Command "print-config" is deprecated, use the influx-cli command server-config to show the configuration values from the operating server
Command "print-config" is deprecated, use the influx-cli command server-config to show the configuration values from the operating server
Command "print-config" is deprecated, use the influx-cli command server-config to show the configuration values from the operating server
2023-03-18T21:57:19.388287863Z  warn    boltdb no longer discovered at configured trail, however DOCKER_INFLUXDB_INIT_MODE no longer specified, skipping setup wrapper    {"system": "docker", "bolt_path": ""}
...

If you run this container on Fedora or Red Hat Enterprise Linux (RHEL), use possibility Z when mounting a quantity to verify it receives the proper SELinux labels.

[ Get the SELinux cheat sheet. ]

The subsequent step is initializing the database. I wish to apply the wizard:

josevnz@server2:~$ podman exec --tty --interactive mydb /bin/bash
root@cd378ef1f5c3:/# inflow setup
> Welcome to InfluxDB 2.0!
? Please sort your number one username josevnz
? Please sort your password *********
? Please sort your password once more *********
? Please sort your number one group call KodeGeek
? Please sort your number one bucket call glances
? Please sort your retention length in hours, or 0 for countless 0
? Setup with those parameters?
  Username:   josevnz
  Organization:KodeGeek
  Bucket:     glances
  Retention Period:  countless
 Yes
User    Organization    Bucket
josevnz KodeGeek    glances
root@cd378ef1f5c3:/# 

While attached to the container, create an all-access token that you are going to use to offer entry in your database:

root@cd378ef1f5c3:/# inflow auth create 
  --org KodeGeek 
  --read-authorizations 
  --write-authorizations 
  --read-buckets 
  --write-buckets 
  --read-dashboards 
  --write-dashboards 
  --read-tasks 
  --write-tasks 
  --read-telegrafs 
  --write-telegrafs 
  --read-users 
  --write-users 
  --read-variables 
  --write-variables 
  --read-secrets 
  --write-secrets 
  --read-labels 
  --write-labels 
  --read-views 
  --write-views 
  --read-documents 
  --write-documents 
  --read-notificationRules 
  --write-notificationRules 
  --read-notificationEndpoints 
  --write-notificationEndpoints 
  --read-checks 
  --write-checks 
  --read-dbrp 
  --write-dbrp 
  --read-annotations 
  --write-annotations 
  --read-sources 
  --write-sources 
  --read-scrapers 
  --write-scrapers 
  --read-notebooks 
  --write-notebooks 
  --read-remotes 
  --write-remotes 
  --read-replications 
  --write-replications
ID          Description Token                                               User Name   User ID         Permissions
0ae9b2f1ea468000            F8y7eoaPX5gMkWvpxZ-b2LOnJjMO6gdH1ba1HfQV0dXmJm6oBekA7WsPiPk-3zhOxL8Y55_aJB1Ii-kRBDsH6w==    josevnz     0ae9b2131d868000    [read:orgs/b38ef4c091e3eca2/authorizations write:orgs/b38ef4c091e3eca2/authorizations read:orgs/b38ef4c091e3eca2/buckets write:orgs/b38ef4c091e3eca2/buckets read:orgs/b38ef4c091e3eca2/dashboards write:orgs/b38ef4c091e3eca2/dashboards read:orgs/b38ef4c091e3eca2/tasks write:orgs/b38ef4c091e3eca2/tasks read:orgs/b38ef4c091e3eca2/telegrafs write:orgs/b38ef4c091e3eca2/telegrafs read:/users write:/users read:orgs/b38ef4c091e3eca2/variables write:orgs/b38ef4c091e3eca2/variables read:orgs/b38ef4c091e3eca2/secrets write:orgs/b38ef4c091e3eca2/secrets read:orgs/b38ef4c091e3eca2/labels write:orgs/b38ef4c091e3eca2/labels read:orgs/b38ef4c091e3eca2/views write:orgs/b38ef4c091e3eca2/views read:orgs/b38ef4c091e3eca2/documents write:orgs/b38ef4c091e3eca2/documents read:orgs/b38ef4c091e3eca2/notificationRules write:orgs/b38ef4c091e3eca2/notificationRules read:orgs/b38ef4c091e3eca2/notificationEndpoints write:orgs/b38ef4c091e3eca2/notificationEndpoints read:orgs/b38ef4c091e3eca2/checks write:orgs/b38ef4c091e3eca2/checks read:orgs/b38ef4c091e3eca2/dbrp write:orgs/b38ef4c091e3eca2/dbrp read:orgs/b38ef4c091e3eca2/annotations write:orgs/b38ef4c091e3eca2/annotations read:orgs/b38ef4c091e3eca2/sources write:orgs/b38ef4c091e3eca2/sources read:orgs/b38ef4c091e3eca2/scrapers write:orgs/b38ef4c091e3eca2/scrapers read:orgs/b38ef4c091e3eca2/notebooks write:orgs/b38ef4c091e3eca2/notebooks read:orgs/b38ef4c091e3eca2/remotes write:orgs/b38ef4c091e3eca2/remotes read:orgs/b38ef4c091e3eca2/replications write:orgs/b38ef4c091e3eca2/replications]

root@cd378ef1f5c3:/# go out

The vital piece is the contemporary new token: F8y7eoaPX5gMkWvpxZ-b2LOnJjMO6gdH1ba1HfQV0dXmJm6oBekA7WsPiPk-3zhOxL8Y55_aJB1Ii-kRBDsH6w==. Keep it secure. You will use it in a while to permit Glances to hook up with InfluxDB.

[ Want to test your sysadmin skills? Take a skills assessment today. ]

Connect InfluxDB and Glances

Glances expects to have its configuration in a number of places. For this case, I will be able to use ~/.config/glances/glances.conf with InfluxDB settings on server1 (the device with issues):

$ mkdir ~/.config/glances/
$ /bin/cat<<EOF>~/.config/glances/glances.conf
[global]
refresh=2
check_update=false
history_size=28800
[influxdb2]
# server2 is the place InfluxDB is operating
host=server2
port=8086
protocol=http
org=KodeGeek
bucket=glances
# And right here you set the tocket we generated at the earlier step
token=F8y7eoaPX5gMkWvpxZ-b2LOnJjMO6gdH1ba1HfQV0dXmJm6oBekA7WsPiPk-3zhOxL8Y55_aJB1Ii-kRBDsH6w==
EOF

Capture the metrics

It’s time to seize the ones metrics. Activate your digital setting and get started Glances:

$ . ~/virtualenv/glances/bin/turn on
(glances) $ glances -t 5 --export influxdb2

Run the sysbench utility to simulate task:

$ sudo dnf set up -y sysbench
$ for i in $(seq 10); do sysbench --test=cpu run; carried out

Watch the consequences

Glances is superb at appearing real-time metrics, however now you’ll see how the ideas appears to be like as a time collection. Using your browser, navigate to the IP cope with or hostname of the device operating InfluxDB on port 8086 and log in with the consumer you created when putting in place the database. You can use the Data Explorer view to look the metrics gathered from Glances.

See the next video for an instance of the way it seems like on my device:

As you’ll see, Glances created a number of metrics and tags within the Glances bucket you outlined within the configuration.

Wrap up

• Traditional gear like System Activity Report (sar) are helpful for taking pictures metrics, however more moderen utilities have extra flexibility and choices. Glances is written in Python and helps a server mode, which means that you’ll write your personal gear on best of it.
• Glances and InfluxDB are a super aggregate to watch and troubleshoot device problems. They are simple to put in, paintings smartly in combination, and are simple to make use of, which is helping you focal point on fixing your efficiency factor.
• Both Glances and InfluxDB have a colourful group. If you run into hassle with them, you’re going to in finding lend a hand briefly.
• I in short discussed Sysbench. You can run different benchmark gear that can assist you in finding device bottlenecks.

[ Keep your most commonly used commands handy with the Linux commands cheat sheet. ]