Long running sidekiq jobs failing with ActiveRecord::StatementInvalid

We have a bunch of sidekiq jobs (background workers) that talk to an API for a long time (10 minutes+) and then save the results into the DB.

For some APIs/users the jobs are failing consistently with ActiveRecord::StatementInvalid and:

 PG::ConnectionBad: PQconsumeInput() server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.

Render and its postgress should not have any limitation on long-lived connections, yet we are seeing this happen on a daily basis, some jobs never finish untill a workaround is implemented. Right now I am testing something like this: retry for ActiveRecord::StatementInvalid · GitHub

Has anyone seen a similar issue and found a solution? Render’s support has not been able to help so far.

The only slighly related thing I found online is this thread: https://groups.google.com/g/sidekiq/c/dvJq8Sk6MuI

but enabling reaper has not helped

I’m experiencing the exact same thing, but with delayed rather than sidekiq. The only thing I’ve found is that it might be related to pooling?

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.