Redis worker task not completing

1

I have worker class that uses Sidekiq in my Rails app that carries out a simple update of static content to a VPS.

Worked fine on Heroku with Redis plugin setup there but since I migrated to Render and now using Render’s redis service it’s not working.

I can connect fine to the redis instance via the web service and the worker appears to fire as normal. Using the redis cli and MONITOR tool I get the following when the worker is ran:

1650455531.038333 [0 10.201.28.160:52550] “multi”
1650455531.039255 [0 10.201.28.160:52550] “sadd” “queues” “default”
1650455531.039265 [0 10.201.28.160:52550] “lpush” “queue:default” “{“class”:“PublisherWorker”,“args”:[9,“ranchogranderoad.com”],“retry”:true,“queue”:“default”,“jid”:“3fa72b39afc76a348c6acbb5”,“created_at”:1650455531.0337899,“enqueued_at”:1650455531.0339687}”
1650455531.039294 [0 10.201.28.160:52550] “exec”

No error or indication though of what’s going wrong.

2

Hey Dan,

Do you have a background service running sidekiq and therefore processing your queued jobs?

John B

3

Hey John,

Sorry yes I did think of that just after and set that up.

But now when I run my job I get

Apr 20 02:11:15 PM 2022-04-20T13:11:15.159Z 66 TID-godg9t7oa PublisherWorker JID-58ce06818ccc19af74e647e6 INFO: start
Apr 20 02:11:15 PM 2022-04-20T13:11:15.160Z 66 TID-godg9t7oa PublisherWorker JID-58ce06818ccc19af74e647e6 INFO: fail: 0.0 sec
Apr 20 02:11:15 PM 2022-04-20T13:11:15.211Z 66 TID-godg9qfz2 WARN: {“context”:“Job raised exception”,“job”:{“class”:“PublisherWorker”,“args”:[9,“ranchogrander…”],“retry”:true,“queue”:“default”,“jid”:“ec3113923f68fac69e2216d4”,“created_at”:1650319467.6995518,“enqueued_at”:1650319467.699732},“jobstr”:"{“class”:“PublisherWorker”,“args”:[9,“ranchogranderoad.com”],“retry”:true,“queue”:“default”,“jid”:“ec3113923f68fac69e2216d4”,“created_at”:1650319467.6995518,“enqueued_at”:1650319467.699732}"}
Apr 20 02:11:15 PM 2022-04-20T13:11:15.212Z 66 TID-godg9qfz2 WARN: NameError: uninitialized constant PublisherWorker
Apr 20 02:11:15 PM 2022-04-20T13:11:15.212Z 66 TID-godg9qfz2 WARN: /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:274:in block in constantize' Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:271:in each’
Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:271:in inject' Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:271:in constantize’
Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:133:in block (5 levels) in dispatch' Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq.rb:37:in block in module:Sidekiq’
Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in block (4 levels) in dispatch' Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in stats’
Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in block (3 levels) in dispatch' Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/job_logger.rb:8:in call’
Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in block (2 levels) in dispatch' Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in global’
Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in block in dispatch' Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in with_context’
Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in with_job_hash_context' Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in dispatch’
Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in process' Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in process_one’
Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in run' Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in watchdog’
Apr 20 02:11:15 PM /opt/render/project/.gems/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in `block in safe_thread’

4

So I think I know what’s going on here - and one of the reasons why I’m writing a new guide for deploying Sidekiq with Rails on us as we speak in fact.

You’re correct in that you need to be running a background worker service, this should be running the same codebase as your webservice and have all the same configuration, environment variables etc, but the start command should be bundle exec sidekiq - your Rails service probably uses bundle exec puma or similar.

So the steps required:

  1. From the dashboard, New → Background worker
  2. Pick your main website repo
  3. Give the service a name
  4. Change the start command to bundle exec sidekiq
  5. Set any environment variable from the advanced settings
  6. wait.
  7. process jobs.

John B

5

That worked, thanks very much.

1 Like
6

Great, I’m glad you got it sorted!

John B

7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.