Long running sidekiq jobs failing with ActiveRecord::StatementInvalid

1

We have a bunch of sidekiq jobs (background workers) that talk to an API for a long time (10 minutes+) and then save the results into the DB.

For some APIs/users the jobs are failing consistently with ActiveRecord::StatementInvalid and:

 PG::ConnectionBad: PQconsumeInput() server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.

Render and its postgress should not have any limitation on long-lived connections, yet we are seeing this happen on a daily basis, some jobs never finish untill a workaround is implemented. Right now I am testing something like this: retry for ActiveRecord::StatementInvalid · GitHub

Has anyone seen a similar issue and found a solution? Render’s support has not been able to help so far.

2

The only slighly related thing I found online is this thread: https://groups.google.com/g/sidekiq/c/dvJq8Sk6MuI

but enabling reaper has not helped

3

I’m experiencing the exact same thing, but with delayed rather than sidekiq. The only thing I’ve found is that it might be related to pooling?

4

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.