Getting Started With ELK Stack By Building Uptime Monitor.

What is ELK? long story short,ELK stand for Elasticsearch, Logstash, Kibana.

Elasticsearch is a full-text, distributed NoSQL database.

Logstash is a tool for collecting, parsing, and storing logs for future use.

Kibana is an open source analytics and visualization platform designed to work with Elasticsearch.

There is one more tool, or actually tools i want to mention:

Beats all kinds of shippers for all kinds of data (later on we are going to use Heartbeat from the beats family).

ELK  stack can be overwhelming so i suggest we start with setup an environment and continue from that.

Quick setup with docker:

git clone https://github.com/niradler/docker-elk
cd docker-elk
docker-compose -f "docker-compose.yml" up

lets see what we got:

docker ps

After docker compose finish we get 4 containers up, Kibana, Elasticsearch, Logstash and Heartbeat.

The All propose of the Elk stack is to store, analyze, and visualise data, so let's create data to work with, for that i'm going to use Heartbeat (i choose this one only because it is simple to understand and very useful).

Heartbeat monitor services for their availability with active probing.

in the heartbeat folder you can find the heartbeat.yml

in this example i'm going to use heartbeat to call Elasticsearch root every 20s, google every 40s, and a random website every 40s, and of-course to report the result to Elasticsearch.

now let's open Kibana http://localhost:5601 and go to the management tab, choose Index Patterns and enter the index: heartbeat-* , press next and choose timestamp.

now we can go to the discover tab and get amazed.

Notes:

  • press the options button and turn on query features.
  • press the auto refresh button and configure it to keep update the data.
  • you can choose the field to show by hover them in the side menu and press add.
  • you can search with ┬ámonitor.status:up.

let's move on to the Visualize tab, press the plus button and choose pie, on the left choose the index we created earlier, now lets visualize the statuses we get from heartbeat, press Split Slices and fill:

  • Aggregation -> Terms
  • Field -> monitor.status

press the play button and you get:

you can hover it to see the count, when you satisfied with the result press the save button on the top and enter a name for the chart.

Notes and resources:

  • watch this official elastic videos: Elasticsearch, Kibana, Logstash
  • logz.io have lots of good examples and they are free to experiment up to 3gb.
  • to experiment with Elasticsearch you can use the Dev Tools in Kibana.

Nir Adler

Nir Adler