Filebeat configuration for ELK

Filebeat Configuration for ELK Stack

Filebeat is a log shipper which can be used for linux and windows both. It is generally use for sending the logs to the server. As I am using filebeat for my ELK stack (ubuntu server), So filebeat sends logs to the logstash server. While communicating with filebeat, logstash use lumberjack protocol.

Installation steps : 

first you have to add the beats repository for APT :

             Run the following commands:

$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -   


$ echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list


After this install the filebeat package

$ sudo apt-get update
$ sudo apt-get install filebeat

After installation next step is configuration of filebeat

Here we are configuring filebeat to sends logs to the logstash server. The configuration file of filebeat is in the /etc/filebeat directory, for open it

$ sudo nano /etc/filebeat/filbeat.yml

before editing the file you should take the backup of the same file, and one thing more as this is a .yml file so you should take care of extra spaces and extra characters one extra space can spoil your configuration.


So, in the very starting you'll get one section called prospectors in which you'll have to define the log file path that mean which log file you want to send to the logstash, add the file path like if I want to sends the system logs to logstash then it looks like

 prospectors:

    -
      paths:
        - /var/log/syslog
        - /var/log/mail.log
        - /var/log/auth.log
      input_type: log
      document_type: syslog
      fields_under_root: true


if you want to send the apache logs

    -
      paths:
        - /var/log/apache2/access.log
      input_type: log
      document_type: apache
      fields_under_root: true

Now the second part of the configuration which is output section

In this section we'll have to configure the output of filebeat,  I choose logstash as output so we'll configure according to logstash

In filebeat.yml file find the output section, under this section you'll get two thing

elasticsearch as output
&
logstash as output

so you'll have to comment or remove full elasticseach section, and come to logstash section says

#Logstash as output

here also two things are there that if you want to send the logs to the logstash server which is on local machine then do not edit the host section or make it localhost as shown below

output:

  ### Logstash as output
  logstash:
    # The Logstash hosts
    hosts: ["localhost:5044"]


And if you want to send the logs to the logstash server which is on different machine then in the host section write the ip address of the same machine, here 5044 is logstash port number.

output:

  ### Logstash as output
  logstash:
    # The Logstash hosts
    hosts: ["Logstash_server_ip_address:5044"]


save the file and restart the filebeat service

$ service filebeat restart 

for every new changes in the filebeat configuration file you should restart the filebeat service with the above command.




if you face any issue in the configuration then please go through this below link :

filebeat configuration

And if you face any issue then kindly comment on it.

reference : https://www.elastic.co, https://github.com