[Rails] Preventing 'ActiveRecord::ConcurrentMigrationError' on build

The docs and example code suggest running your Rails database migrations (rails db:migrate) as part of your build script.

I’ve been running the same build script for both my ruby web service and my (background) worker.

The idea here is that whichever service gets deployed first will take care of the migrations and the next one will see the database is already up-to-date and skip the migrations.

This has worked fine, but I now have a Rails app with slower database migrations. This leads to scenario where one service has already started migrating, but isn’t finished yet. The second service sees the database is out-of-date and also tries to migrate the database, which causes the ActiveRecord::ConcurrentMigrationError exception.

What’s the recommended approach here?

Running the migrations in the build script for only one service doesn’t cut it, because the other one might get deployed first.

Hi @marc ,

The approach I usually recommend is to have some locking mechanism on the database to prevent double-migration, so it’s great that you already have that in the form of the ActiveRecord::ConcurrentMigrationError exception.

When your second service builds and the migration lock is already held, what behavior do you want? Do you want your second service to wait for the lock to be released? If that’s the case, you can likely detect that exception and do a loop with a sleep that waits for the lock to be released.

Since this seems like a common setup (having a Rails app with a worker), I’d expect Render to handle it out-of-the-box somehow.

I’m not sure what the right architecture would be for this (perhaps a service can be configured to be ‘dependent’ on another service like in GitHub Actions) but having to add a loop seems a bit hacky.

Do you happen to have some example code you can share to implement this work around? Let’s say bundle exec rake db:migrate is part of my bash script and I want to retry every 10 seconds until it succeeds or after 5 failures.

We don’t have any sample code, unfortunately. I asked internally, and a colleague suggested another workaround: If you’re running into this because both services are auto-deploying, you can disable auto-deploy on your background worker, and have your rails service trigger a deploy of the background worker using a deploy hook (Deploy Hooks | Render · Cloud Hosting for Developers). I know this is also a bit hacky, but I wanted to share it in case it’s helpful for you.

I’ve also created a feature request for dependent services on feedback.render.com. If there’s any more detail you think would be helpful to include, I definitely encourage you to add it.

1 Like

Thanks Dan. I left some more comments on the Feature Request.

The Deploy Hooks are an interesting idea, but I don’t want my web service and worker to potentially run on different versions of the code base. My whole app is designed in such a way that assumes they are always running the same version.