Health check not working with custom domain

I have a GraphQL API set up at a custom domain, but when I enter /graphql as my Health Check Path, render tries to make a request to (for example) http://mycustomdomain.com:10000/graphql which fails. Making that request also fails from the shell via curl. My Apollo Server welcome page is indeed up and working and returning a 200 OK at (for example) https://mycustomdomain/graphql any idea what’s up?

Hey Chris,

Our health checks make a GET request to the configured healthcheck path, does your /graphql endpoint respond to GET requests, in my experience they typically are a POST endpoint?

Regards,

John B

Hi John,

Good question, however that /graphql endpoint is an Apollo Server welcome screen that does respond to a GET request (it’s just a web page).

Could the problem be that the custom domain used by the health check is resolving to an external IP address, that doesn’t work from inside Render’s network?

Chris,

I’m still suspicious of that endpoint - most people I’ve seen configure a lightweight /health endpoint that checks the external dependencies are ok.

One thing you can do to test - from the dashboard, drop into the shell tab and then execute

$ curl -I http://<internal address>/graphql

and see what it returns, where is the internal address of the service shown at the top of the page. Checking the external address with curl certainly fails:

$ curl -I https://api.staging.local328.org/graphqlHTTP/2 400date: Mon, 25 Apr 2022 18:00:56 GMTcontent-type: text/html; charset=utf-8access-control-allow-origin: *etag: W/"12-7JEJwpG8g89ii7CR/6hhfN27Q+k"vary: Accept-Encodingx-powered-by: Expresscf-cache-status: MISSexpect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"server: cloudflarecf-ray: 7018fb260b2a1887-MANalt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

John B

John,

Thanks for helping me investigate this, I really appreciate it! Looks like that /graphql address does indeed return a 400, I found out Apollo Server’s built-in health check URL is actually /.well-known/apollo/server-health which does work. Deploy fixed!

I did notice in your “Events” log, the health check URL appears to be dynamically generated so old log entries with errors show the new URL currently in Settings, instead of the old URL used for the request.

Previously I tried running curl from the dashboard shell, but it hung with no response because I was using the address shown in that Events error message, which actually doesn’t work from anywhere (internally or externally) becasue it shows the external domain with the internal port.

I’d suggest showing the internal domain and port in that error message, so the URL shown in the error message is what’s actually being used by the health check and works from the shell. Offering that up as a bug fix for the sake of people who might be troubleshooting health check URLs in the future.

I’m glad we got things sorted - I’ll send your feedback over to the product team!

John B