Installation

On Ubuntu (picking the right version https://docs.fluentd.org/installation/install-by-deb)

curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-trusty-td-agent3.sh | sh
sudo systemctl start td-agent.service

and then it should show a postive status with

sudo systemctl status td-agent.service

Fluentd and Elasticsearch

Setup an elastic shipper and a secure login plugin

$ sudo /usr/sbin/td-agent-gem install fluent-plugin-secure-forward
$ sudo /usr/sbin/td-agent-gem install fluent-plugin-elasticsearch

Configure the configure Fluentd by editing etc/td-agent/td-agent.conf

# Listen to incoming data over SSL
<source>
  type secure_forward
  shared_key FLUENTD_SECRET
  self_hostname logs.example.com
  cert_auto_generate yes
</source>

# Store Data in Elasticsearch and S3
<match *.**>
  type copy
  <store>
    type elasticsearch
    host localhost
    port 9200
    include_tag_key true
    
    tag_key @log_name
    logstash_format true
    flush_interval 10s
  </store>

Restart the agent

$ sudo service td-agent restart

Fluentd and Java/Log4j

Add a forwarder to forward data from log4j

<source>
  @type forward
  port 24224
</source>
<match fluentd.test.**>
  @type stdout
</match>

add the gradle

dependencies {
    compile 'org.fluentd:fluent-logger:'+loggerVersion
}

High Availability

 
fluentd.txt · Last modified: 2019/09/13 07:11 by root
 
RSS - 200 © CrosswireDigitialMedia Ltd