All Questions
13 questions
3
votes
1
answer
1k
views
Flatten nested JSON using fluentd
I have a program that writes structured logs, and the following example applies:
{
"time": "time_val",
"log": "{
\"field1\": \"value1\&...
- 343
0
votes
1
answer
1k
views
Fluentd doesn't parse json log from docker
I have some problems parsing json logs that were received from docker container. I know this question is probably a duplicate but none of the solutions found, including the documentation (https://docs....
- 51
0
votes
1
answer
118
views
Fluentd change output encapsulation
I have a fluentd record
{"key": "value"}
but i wish to emit this record encapsulated inside an array (even if only one), like so:
[{"key": "value"}]
Is there ...
- 129
2
votes
1
answer
857
views
Multiple time formats using fluentd JSON parser
I'm currently reading container logs as my source in fluentd, and I'm parsing all of our log files which is in JSON format. The JSON parser is working as expected based on our configuration, but the ...
- 520
2
votes
1
answer
3k
views
Nginx json logs are incorrectly parsed by Fluentd in Elasticsearch (+ Kibana)
I have an nginx 1.16.1 running in docker in vanilla K8s (bare metal).
Logs are forwarded to stdout by ln -sf /dev/stdout /var/log/nginx/access.log & then are transferred to Elasticsearch docker....
- 374
2
votes
1
answer
6k
views
How do I get fluentd / elasticsearch to parse the "log" key_name as json from a kubernetes pod?
I am having issues trying to get logs into elasticsearch from fluentd in a k8s cluster.
I have several web applications which output their logs as json. Using a simple setup locally with docker ...
- 393
1
vote
1
answer
4k
views
Process multi-level nested escaped JSON strings inside JSON with fluentd
I'm new to fluentd and I would like to parse a multi-level nested escaped JSON strings inside JSON.
My messages look like:
{"log":"HELLO WORLD\n","stream":"stdout","time":"2019-05-23T15:40:54....
- 203
3
votes
1
answer
7k
views
How to tail multiple files in fluentd
I have setup fluentd logger and I am able to monitor a file by using fluentd tail input plugin. All the data is received by fluentd is later published to elasticsearch cluster. Below is the ...
- 7,078
2
votes
1
answer
693
views
How to get data from a file in fluentd logger
I have a python code running as a service in the background. It generates a data.json file which contains a json data, something like below
{
"Type":"Data"
"Name":"XYZ"
"Level":32
}
This ...
- 7,078
2
votes
1
answer
1k
views
Logging with Fluentd - why is the output of json log file appearing as textpayload (not jsonpayload)?
I am new to Fluentd. I'm using stackdriver in GKE and I am customizing the Fluentd configuration in GKE to make some changes to the logs.
In my configuration file for the logs of my containers I have:...
- 141
2
votes
0
answers
1k
views
How to store logs in MongoDB
I am using Fluentd data collector for storing Apache httpd logs in MongoDB. I made the necessary changes in td-agent configuration files like:
<source>
@type tail
format apache2
path C:\...
- 163
6
votes
1
answer
3k
views
Kubernetes save JSON logs to file with escaped quotes. Why?
I'm using Fluentd with Elasticsearch for logs from Kubernetes but I noticed that some JSON logs cannot be correctly indexed because JSON is stored as string.
Logs from kubectl logs look like:
{"...
- 61
1
vote
2
answers
5k
views
How to get container and image name when using fluentd for docker logging?
I am using a fluentd container that mounts /var/lib/docker/containers directory and tails all of the logs for each container. The logs are stored in /var/lib/docker/containers/$container_id/$...
- 260