Dockerized RoR API deploy failing when using health check path

[StarSheriff2]

I have successfully deployed a dockerized Ruby on Rails API in a production environment on render. I am using a free Postgres service to persist my data. Everything works as expected until I add a health check path on my settings.

I have tried several things. One thing I did was point my ‘’/healthcheck’ route to a 200 OK response all the time; that is, my DB is not even touched by the health check path. That means that the problem has nothing to do with the DB connection not being active.

My hunch is that this is something to do with the port used in the internal url for the health check. In my case it’s something like my-render-generated-url:1000/healthcheck.

Render logs are usually not telling me anything as to why the deploys are failing. The server just shuts down after Puma boots up:

Mar 8 12:50:57 AM  => Booting Puma
Mar 8 12:50:57 AM  => Rails 6.1.4.4 application starting in production
Mar 8 12:50:57 AM  => Run `bin/rails server --help` for more startup options
Mar 8 12:51:31 AM  Puma starting in single mode...
Mar 8 12:51:31 AM  * Puma version: 5.5.2 (ruby 3.0.2-p107) ("Zawgyi")
Mar 8 12:51:31 AM  *  Min threads: 5
Mar 8 12:51:31 AM  *  Max threads: 5
Mar 8 12:51:31 AM  *  Environment: production
Mar 8 12:51:31 AM  *          PID: 1
Mar 8 12:51:31 AM  * Listening on http://0.0.0.0:10000
Mar 8 12:51:31 AM  Use Ctrl-C to stop
Mar 8 12:55:06 AM  - Gracefully stopping, waiting for requests to finish
Mar 8 12:55:06 AM  Exiting

However, occasionally I do get these errors:

Mar 8 11:44:57 AM  => Booting Puma
Mar 8 11:44:57 AM  => Rails 6.1.4.4 application starting in production
Mar 8 11:44:57 AM  => Run `bin/rails server --help` for more startup options
Mar 8 11:45:32 AM  [1] Puma starting in cluster mode...
Mar 8 11:45:32 AM  [1] * Puma version: 5.5.2 (ruby 3.0.2-p107) ("Zawgyi")
Mar 8 11:45:32 AM  [1] *  Min threads: 5
Mar 8 11:45:32 AM  [1] *  Max threads: 5
Mar 8 11:45:32 AM  [1] *  Environment: production
Mar 8 11:45:32 AM  [1] *   Master PID: 1
Mar 8 11:45:32 AM  [1] *      Workers: 4
Mar 8 11:45:32 AM  [1] *     Restarts: (✔) hot (✖) phased
Mar 8 11:45:32 AM  [1] * Preloading application
Mar 8 11:45:32 AM  [1] * Listening on http://0.0.0.0:10000
Mar 8 11:45:32 AM  [1] Use Ctrl-C to stop
Mar 8 11:45:32 AM  [1] - Worker 0 (PID: 8) booted in 0.19s, phase: 0
Mar 8 11:45:32 AM  [1] - Worker 1 (PID: 10) booted in 0.21s, phase: 0
Mar 8 11:45:32 AM  [1] - Worker 2 (PID: 15) booted in 0.2s, phase: 0
Mar 8 11:45:32 AM  [1] - Worker 3 (PID: 27) booted in 0.1s, phase: 0
Mar 8 11:49:34 AM  [1] === puma shutdown: 2023-03-08 17:49:34 +0000 ===
Mar 8 11:49:34 AM  [1] - Goodbye!
Mar 8 11:49:34 AM  [1] - Gracefully shutting down workers...
Mar 8 11:49:39 AM  [BUG] Segmentation fault at 0x0000000000000440SEGV received in SEGV handler
Mar 8 11:49:39 AM  SEGV received in SEGV handler
Mar 8 11:49:39 AM  [BUG] Segmentation fault at 0x0000000000000440
Mar 8 11:49:39 AM  ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux]
Mar 8 11:49:39 AM  
Mar 8 11:49:39 AM  SEGV received in SEGV handler
Mar 8 11:49:39 AM  SEGV received in SEGV handler
Mar 8 11:49:39 AM  -- Machine register context ------------------------------------------------
Mar 8 11:49:39 AM   RIP: 0x00007f07d2f813daSEGV received in SEGV handler
Mar 8 11:49:39 AM  SEGV received in SEGV handler
Mar 8 11:49:39 AM  SEGV received in SEGV handler
Mar 8 11:49:39 AM  SEGV received in SEGV handler
Mar 8 11:49:39 AM   RBP: 0x00007f07cd0cde60 RSP: 0x00007f07cd0cddf0
Mar 8 11:49:39 AM   RAX: 0x0000000000000000 RBX: 0x0000000000000000 RCX: 0x0000000000000006
Mar 8 11:49:39 AM   RDX: 0x0000000000000002 RDI: 0x0000562a2cfaa010 RSI: 0x0000000000000000
Mar 8 11:49:39 AM    R8: 0x00000000000000ca  R9: 0x0000000000000001 R10: 0x0000000000000000
Mar 8 11:49:39 AM   R11: 0x0000000000000286 R12: 0x0000562a2cfaa010 R13: 0x00007f07cd0ce700
Mar 8 11:49:39 AM   R14: 0x00007f07d3273e70 R15: 0x00007f07cd0cdea8 EFL: 0x0000000000010202
Mar 8 11:49:39 AM  
Mar 8 11:49:39 AM  -- C level backtrace information -------------------------------------------
Mar 8 11:49:40 AM  Exiting

There seems to be a memory leak.

I have tried the health checks locally through the Healthcheck property of the Dockerfile, and everything seems to work just fine locally.

I don’t know what I am missing here. Could it be that the health check is pinging my app before it actually spins up?

I am sharing here my current setup:

Note: Environmental files are loaded through Render’s settings.

production.Dockerfile

# syntax=docker/dockerfile:1

FROM ruby:3.0.2

RUN apt-get update -qq && apt-get install -y postgresql-client

ENV PORT=3001 \

RAILS_ENV=production \

RAILS_LOG_TO_STDOUT=true

# throw errors if Gemfile has been modified since Gemfile.lock

RUN bundle config --global frozen 1

WORKDIR /app

COPY Gemfile Gemfile.lock ./

RUN gem update --system && gem install bundler

RUN bundle config set --local without 'development test' \

&& bundle install \

&& bundle exec rails db:prepare

COPY . .

COPY production.entrypoint.sh /usr/bin/

RUN chmod +x /usr/bin/production.entrypoint.sh

ENTRYPOINT ["production.entrypoint.sh"]

EXPOSE 3001

CMD ["rails", "server", "-b", "0.0.0.0"]

production.entrypoint.sh

#!/bin/bash

set -e

if [ -f tmp/pids/server.pid ]; then
    rm tmp/pids/server.pid
fi

exec "$@"

config/puma.rb

max_threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 }

min_threads_count = ENV.fetch("RAILS_MIN_THREADS") { max_threads_count }

threads min_threads_count, max_threads_count

worker_timeout 3600 if ENV.fetch("RAILS_ENV", "development") == "development"

port ENV.fetch("PORT") { 3001 }

environment ENV.fetch("RAILS_ENV") { "development" }

pidfile ENV.fetch("PIDFILE") { "tmp/pids/server.pid" }

if ENV['RENDER'].present?

  workers ENV.fetch("WEB_CONCURRENCY") { 4 }
  
  preload_app!

end

plugin :tmp_restart

config/environments/production.rb

require "active_support/core_ext/integer/time"

Rails.application.configure do

  config.hosts << "my-render-generated-url.com"

  config.cache_classes = true

  config.eager_load = true
  config.consider_all_requests_local       = false

  config.require_master_key = true

  config.public_file_server.enabled = ENV['RAILS_SERVE_STATIC_FILES'].present? || ENV['RENDER'].present?

  config.active_storage.service = :local

  config.log_tags = [ :request_id ]

  config.action_mailer.perform_caching = false

  config.i18n.fallbacks = true

  config.active_support.deprecation = :notify

  config.active_support.disallowed_deprecation = :log

  config.active_support.disallowed_deprecation_warnings = []

  config.log_formatter = ::Logger::Formatter.new

  if ENV["RAILS_LOG_TO_STDOUT"].present?
    logger           = ActiveSupport::Logger.new(STDOUT)
    logger.formatter = config.log_formatter
    config.logger    = ActiveSupport::TaggedLogging.new(logger)
  end

  config.active_record.dump_schema_after_migration = false
ActiveRecord::Middleware::DatabaseSelector::Resolver::Session
end

config/database.yml

default: &default
  adapter: postgresql
  encoding: unicode
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  host: <%= ENV['DATABASE_HOSTNAME'] %>
  username: postgres
  password: secret
  port: 5432

development:
  <<: *default
  database: weather_app_calendar_db_development

test:
  <<: *default
  database: weather_app_calendar_db_test

production:
  <<: *default
  url: <%= ENV['DATABASE_URL'] %>

[StarSheriff2]

Finally solved it. The issue was extremely silly and simple, as usually.

I had this line in my config/environments/production.rb file:

config.hosts << ENV.fetch['RENDER_EXTERNAL_HOSTNAME']

I removed it, and all of a sudden my Healthcheck endpoint started working.

I spent ungodly amount of hours debugging this.