Logs in LogDNA do not match logs in render dashboard

#1

There are various lines in the logs tab on the render dashboard that are not showing up on LogDNA. There are no exclusion rules set, and I am below the LogDNA quota. There also appears to be no pattern to what logs do and don’t get forwarded on.

A specific instance of this happened at 2020-12-23T20:16:15.516152 on srv-bho3afi0gk6pimeflgug, where the render logs show the following entry:
Dec 23 01:16:15 PM I, [2020-12-23T20:16:15.516152 #55] INFO -- : [9aa2d5a4-f7f5-460d-90af-02680a9c25d3] Started POST "/thermistors/567c3e45-4288-4bbf-b423-4ac37a304be2/attach_pictures" for 69.254.134.100 at 2020-12-23 20:16:15 +0000
but the logdna logs show nothing.

This is a big issue for us: we need to be able to trust our logs.

#2

Hi Kai- thanks for including so much info in this post. This definitely is a big issue, and we’re passing this along to LogDNA to help prevent this from happening in the future.

I’ll comment here when we find something conclusive; in the meantime, if you notice this happening again, could you continue to post any information here. I’ll continue investigating on our end.

#3

Any progress here?

I seem to have gotten this again. I have a record in my database that says it was created at 2021-02-06T03:54:19.163Z; however, there is nothing in the LogDNA logs at this time despite all creations being logged. The render logs only go back about 6 minutes, so I was unable to confirm that it was logged in render, but this has all the same symptoms as before.

It’s unknown how many other logs should be there but are missing without me knowing about it.

#4

We are exploring other logging options. Which provider would you like to integrate with?

#5

No strong preferences. Back in my heroku days I would use LogEntries because they had a generous free tier at the time, but I have no idea what the logging space looks like right now.

1 Like