DNS not correctly resolving?

1

Hey Render team,

I have a two services both deployed with docker and sharing a single database

  1. A public server exposing an API
  2. A private service used by the public server
  3. Postgres database accessed by both

The problem I’m having is that service 1 sometimes crashes because it cannot reach service 2 or the database with errors like:

HTTP connection error: URL: http://rudderstack-transformer:9090/v0/... Error: Post http://rudderstack-transformer:9090/v0/...: dial tcp: lookup rudderstack-transformer on 10.131.0.10:53: no such host

The common line to the error logs is:

dial tcp: lookup rudderstack-transformer on 10.131.0.10:53: no such host
dial tcp: lookup dpg-..... on 10.131.0.10:53: no such host

It seems like the DNS isn’t resolving, I’m really not sure if this is a problem with how I’ve deployed or if this is due to containers being created/destroyed.

Any help would be appreciated.

3

Is this only happening after a new deploy? I took a look at your /etc/resolve.conf file and didn’t see the domain names from your error messages there, but will need to look into whether that is expected or not.

4

We had an issue with our proxy that could have impacted DNS for services on 2021-08-12 06:53. This was resolved at 2021-08-13 13:57, was this for the https://staging.lightdash.com/ service?

5

This was for analytics.lightdash.com

8

Hi @Oliver_Laslett ,

After taking a look at your service, there doesn’t seem to be any more DNS errors occurring and the DNS errors that occurred were during our incident. Everything looks like it’s configured correctly, so if you see more of these errors in the future, definitely let us know!

9

Thanks so much! I didn’t realise there was the outage. So that makes sense.

At least logging was working!

1 Like