Install Elasticsearch, Logstash, and Kibana (ELK Stack) on Ubuntu

28/12/2020
Chưa phân loại
In this lesson, we will see how we can get our ELK Stack up and running on our Ubuntu machines. ELK consist of Elasticsearch 2.2.x, Logstash 2.2.x, and Kibana 4.5.x. We will show how we can configure this stack and use Kibana to visualise the logs which our applications and systems create in a centralized location, using Filebeat 1.1.x. To understand these tools, let’s look at a basic definition for each of them:

  • Logstash: It is an Open Source tool which is used to collect, parse and store logs for an application or a system which can be used later for metrics of APIs, errors faced in a system and many more use-cases which are beyond the scope of this lesson
  • Kibana: This is a dashboard interface on the web which is an excellent dashboard used to search and view the logs that Logstash has indexed into the Elasticsearch index
  • Filebeat: This is installed on the client-server who want to send their logs to Logstash. Filebeat acts as a log shipping agent and communicates with Logstash.

Let us also visualise how things will work:

ELK Setup on Ubuntu

Prerequisites

For this lesson and all installations it needs, you should have root access to the machine. We will be using a machine with this configuration:

  • Ubuntu 16.04
  • RAM: 4 GB
  • CPU: 2

Few application servers from where you want to gather data from would also be a good to have.

Install Java

To install Elasticsearch on Ubuntu, we must install Java first. Java might not be installed by default. We can verify it by using this command:

Checking Java version

java -version

Here is what we get back with this command:

Checking Java version

We will now install Java on our system. Use this command to do so:

Installing Java

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer

Once these commands are done running, we can again verify that Java is now installed by using the same version command.

Install Elasticsearch

Next step for the ELK Stack setup is installing Elasticsearch on Ubuntu Machine which will store the logs generated by systems and applications. Before we can install Elasticsearch, we need to import its public GPG keys to the rpm package manager:

GPG Keys

rpm –import http://packages.elastic.co/GPG-KEY-elasticsearch

Now, insert the mentioned lines to the configuration file for the repository ‘elasticsearch.repo’:
Repository Config

[elasticsearch]
name=Elasticsearch repository
baseurl=http://packages.elastic.co/elasticsearch/2.x/centos
gpgcheck=1
gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

Now, read the lesson Install ElasticSearch on Ubuntu for installation process. Once ES is up and running, make sure it responds normally to this curl command:

ES Status

curl -X GET ‘http://localhost:9200’

The normal output will be:

Install Logstash

Installing Logstash is very easy using the apt package manager and is available with the same repository and public key as Elasticsearch, so we don’t have to do that again. Let’s create the source list to start:

Create Source list

echo ‘deb http://packages.elastic.co/logstash/2.2/debian stable main’ | sudo tee /etc/apt/sources.list.d/logstash-2.2.x.list

Update the apt package list:

Updating Packages

sudo apt-get update

Install Logstash with a single command:

Install Logstash

sudo apt-get install logstash

Logstash is installed but it is not configured yet. We will configure Logstash in coming sections.

Install Kibana

Kibana is very easy to install. We can start by creating the Kibana source list:

Create Kibana source list

echo "deb http://packages.elastic.co/kibana/4.5/debian stable main" | sudo tee -a /etc/apt/sources.list.d/kibana-4.5.x.list

Now, we will update the apt package list:

Updating Packages

sudo apt-get update

We are ready to install Kibana now:

Install Kibana

sudo apt-get -y install kibana

Once Kibana is installed, we can run it:

Start Kibana Service

sudo update-rc.d kibana defaults 96 9
sudo service kibana start

Before we show you the Kibana Dashboard, we need to setup the Filebeat Log shipping agent as well.

Setup Filebeat

We are ready to install Filebeat now:

Install Filebeat

sudo apt-get -y install filebeat

Before we can start the Filebeat service, we need to configure it for the input type and document type. Because we’re using system logs only as of now, let’s mention this in the configuration file in ‘/etc/filebeat/filebeat.yml’:

Configure Filebeat


input_type: log
document_type: syslog

We can also start filebeat now:

Start Filebeat Service

sudo update-rc.d kibana defaults 96 9
sudo service filebeat start

Once filebeat is up and running, we can check that it is OK by issuing the following curl command:

Testing Filebeat

curl -XGET ‘http://localhost:9200/filebeat-*/_search?pretty’

We should receive a similar result as we got in the ES installation.

Connecting to Kibana

We are now ready to connect to Kibana. As we already started the Kibana service, its dashboard should be visible at:

Kibana Dashoboard URL

http://localhost:5601

Once you’re up on Kibana, create an index on Kibana with name ‘filebeat-*’. Now based on the logs available, you can see the metrics and logs in your Kibana Dashboard:

Conclusion

In this lesson, we looked at how we can install and start using the ELK Stack for log visualisation and support an excellent Dashboard for business teams.

ONET IDC thành lập vào năm 2012, là công ty chuyên nghiệp tại Việt Nam trong lĩnh vực cung cấp dịch vụ Hosting, VPS, máy chủ vật lý, dịch vụ Firewall Anti DDoS, SSL… Với 10 năm xây dựng và phát triển, ứng dụng nhiều công nghệ hiện đại, ONET IDC đã giúp hàng ngàn khách hàng tin tưởng lựa chọn, mang lại sự ổn định tuyệt đối cho website của khách hàng để thúc đẩy việc kinh doanh đạt được hiệu quả và thành công.
Bài viết liên quan

Mẹo #3 : Cập nhật WordPress không cần FTP

Đôi khi, khi bạn cập nhật WordPress, các plugin hoặc theme. Một cửa sổ có thể xuất hiện và yêu...
30/12/2020

Linux File Managers and NextCloud Integration

In this article, I am going to show you how to integrate your NextCloud account with your Linux desktop and access NextCloud...
29/12/2020

Configuring ZFS Cache for High Speed IO

Configuring Cache on your ZFS pool If you have been through our previous posts on ZFS basics you know by now that this...
28/12/2020
Bài Viết

Bài Viết Mới Cập Nhật

Mua Proxy V6 Nuôi Facebook Spam Hiệu Quả Tại Onetcomvn
03/06/2024

Hướng dẫn cách sử dụng ProxyDroid để duyệt web ẩn danh
03/06/2024

Mua proxy Onet uy tín tại Onet.com.vn
03/06/2024

Thuê mua IPv4 giá rẻ, tốc độ nhanh, uy tín #1
28/05/2024

Thuê địa chỉ IPv4 IPv6 trọn gói ở đâu chất lượng, giá RẺ nhất?
27/05/2024