Hi! I followed the LogDNA migration guide last month when LogDNA was removed as a Render add-on. So now my Render logs are forwarded to LogDNA via the Syslog Endpoint setting.
Unfortunately, I frequently receive errors:
Log endpoint failure:
We could not connect to your logs endpoint and log streaming has been stopped. We will continue to retry the connection every 10 minutes.
I get about 5-10 of these a day. The logging interruption isn’t so bad since it seems to recover quickly. What is annoying is that it clogs up our Slack channel (we have Render Slack notifications connected) and obfuscates important messages from our channel’s users and Render slack notifications related to deploys. I can’t find any setting to dismiss Render “Log endpoint failure” Slack messages.
Is there any way these errors can be hidden or fixed?
I hear you on issue of notifications being more disruptive than the intermittent connection failures. We can’t disable these slack notifications right now, but are planning to shift away from these failure notifications in favor of resume emails.
As for the failures, this seems to be an issue on LogDNA’s side that they’ve been unable to provide a root cause or mitigation for. There’s unfortunately not much we can do on our end. We haven’t seen similar reliability issues with other providers. I’d suggest switching to PaperTrail to resolve these errors. If you have any issues migrating, please let us know.
hey @Aaron_Gruberg thanks for the coherent reply, that is all useful information. Also we haven’t had any log failure messages in 2 days, so maybe LogDNA has done something about it.
We certainly hear this. It’s a tough balance, for most other logging providers this notification rarely fires so when it does it’s usually a sign of a problem - LogDNA seems to be particularly noisy and at https://render.com/docs/log-streams#logdna we actually suggest not using LogDNA at the moment.
