Reputation: 3
I am learning to use FluentD. I am trying to use it to send logs to coralogix. I have installed the td-agent and I got it to work in a basic manner by using tail plugin to tail a logfile and output plugin to write it to another directory. So I know my install works but I cannot make the http plugin work for sending data to coralogix. I do not even get the td-agent.service daemon to run and I get no error information in the td-agent logfile. All I get is the following information from systemctl status:
[ec2-user@ip-172-31-8-39 td-agent]$ sudo systemctl status -l td-agent.service ● td-agent.service - td-agent: Fluentd based data collector for Treasure Data Loaded: loaded (/usr/lib/systemd/system/td-agent.service; enabled; vendor preset: disabled) Active: failed (Result: start-limit) since Mon 2022-10-03 20:33:54 UTC; 4s ago Docs: https://docs.treasuredata.com/display/public/PD/About+Treasure+Data%27s+Server-Side+Agent Process: 11419 ExecStop=/bin/kill -TERM ${MAINPID} (code=exited, status=0/SUCCESS) Process: 11467 ExecStart=/opt/td-agent/bin/fluentd --log $TD_AGENT_LOG_FILE --daemon /var/run/td-agent/td-agent.pid $TD_AGENT_OPTIONS (code=exited, status=1/FAILURE) Main PID: 11203 (code=exited, status=0/SUCCESS)
Oct 03 20:33:54 ip-172-31-8-39.us-west-1.compute.internal systemd[1]: td-agent.service: control process exited, code=exited status=1 Oct 03 20:33:54 ip-172-31-8-39.us-west-1.compute.internal systemd[1]: Failed to start td-agent: Fluentd based data collector for Treasure Data. Oct 03 20:33:54 ip-172-31-8-39.us-west-1.compute.internal systemd[1]: Unit td-agent.service entered failed state. Oct 03 20:33:54 ip-172-31-8-39.us-west-1.compute.internal systemd[1]: td-agent.service failed. Oct 03 20:33:54 ip-172-31-8-39.us-west-1.compute.internal systemd[1]: td-agent.service holdoff time over, scheduling restart. Oct 03 20:33:54 ip-172-31-8-39.us-west-1.compute.internal systemd[1]: Stopped td-agent: Fluentd based data collector for Treasure Data. Oct 03 20:33:54 ip-172-31-8-39.us-west-1.compute.internal systemd[1]: start request repeated too quickly for td-agent.service Oct 03 20:33:54 ip-172-31-8-39.us-west-1.compute.internal systemd[1]: Failed to start td-agent: Fluentd based data collector for Treasure Data. Oct 03 20:33:54 ip-172-31-8-39.us-west-1.compute.internal systemd[1]: Unit td-agent.service entered failed state. Oct 03 20:33:54 ip-172-31-8-39.us-west-1.compute.internal systemd[1]: td-agent.service failed. [ec2-user@ip-172-31-8-39 td-agent]$ sudo systemctl status -l td-agent.service
My td-agent.conf file is:
@type tail
format none
read_from_head true
tag coralogix
path /var/log/td-agent/coralogix/external/*.log
pos_file /var/log/td-agent/coralogix/external/logfile.pos
</source>
<match coralogix>
@type http
endpoint "https://api.coralogix.com/logs/rest/singles"
headers {"private_key":15156465465 -Private Key numbers -928989023c762e}
retryable_response_codes 503
error_response_as_unrecoverable false
<buffer>
@type memory
chunk_limit_size 5MB
compress gzip
flush_interval 1s
overflow_action block
retry_max_times 5
retry_type periodic
retry_wait 2
</buffer>
<secondary>
#If any messages fail to send they will be send to STDOUT for debug.
@type stdout
</secondary>
</match>
Upvotes: 0
Views: 2103
Reputation: 16
Seems that you are missing the at the top of the configuration. Don't know if its a copy&paste typo or that's the error.
Never the less, here ( https://coralogix.com/docs/fluentd/ ) you can find an example on how to configure fluentd.
Also --dry-run will log more about the start error. Cheers.
Upvotes: 0