I’m currently setting up DataDog integrations into my team’s Render deployment. As the documentation suggested, I added a LogStream pointing to DataDog’s ingestion endpoint with my API key.
By doing this and no more, I noticed that my team’s PostgresQL DB health check logs are making their way to DataDog as well in high volumes. Specifically, the health checks that it seems the Render team is running (Postgres logs filled with constant connection logs) creates logs every 10 seconds and these are being automatically streamed to DataDog which will affect our message quota.
Is there a way I can either disable these health check logs or otherwise prevent them from being sent to DataDog?
@John_B thank you for the response I appreciate it.
Just to confirm, there’s no further advice or information Render can provide at this time on options to mitigate the huge volume of logs from our DB? Is there a way my team can configure how frequently these logs are emitted if we cannot filter them out completely?
I also wanted to confirm, there are only two other individuals that have run into this issue on the Render and DataDog integration? Since my team has not done a ton of additional work, it would appear that anyone that uses the LogStream with DataDog would be seeing this huge volume of logs.
That’s right - I’ve definitely spoken with the team about this and it’s all or nothing. We have some work items noted for tracking around allowing you to exclude services from streams - however, at the moment all services emit into the logstream.
I wonder if there’s something you could do Datadog side to drop the logs from the indexes?
We’re using logtail after moving away from papertrail, and we’re seeing the same noise now in logtail. I didn’t see it before when using papertrail.
Sadly, while papertrail has a way to filter things out at their end, logtail does not have a similar feature yet. I’d love a way to stop the postgres noise at the source as well.