Background worker

1

Hi
I am trying to implement celery with my Django application (terradiem).
I have already activated a Redis server (redisgreen.com) and it works.
I would like to activate a worker together with the terradiem web service. Is it something I can do ? What start command should I enter in the Settings of render ?
I tried to create a background worker (terraworker) but I don’t think it can work as it cannot access to my Django views and to the database.
Thanks a lot for your help
Best
Julien

2

Hi Julien! Have you tried following the steps laid out in our guide to deploying celery on Render? The key step is creating a CELERY_BROKER_URL environment variable that holds the service address for your redis server. You can find your service address in the Render dashboard (it’ll be something like redis:6379).

3

Hi David ! Thank you for your answer. Yes, I created a REDIS_URL environment variable (pointing to redisgreen) which I use in my Django application. The connection is fine.

I know need to activate a worker. As my worker needs Django views and models, I guess it should be activated together with the Django application and cannot be separated ? This is the part I am struggling to manage…

With Heroku, I had a “Procfile” file to activate both a gunicorn and a worker.

Thanks a lot
Julien

4

Got it. You might benefit from using Render’s infrastructure as code offering. As you can see from the render.yaml spec, with IaC you can define and activate multiple services in the same file. It’s similar to a Procfile, but much more expressive.

Make sure you click YAML and then New from YAML when you go to apply your render.yaml file in the dashboard. And don’t hesitate to ask any follow-up questions!

5

Great David.
I was able to deploy an IAC, thank you for that. Unfortunately, the services do not seem to talk to each others. I am actually not very sure how I should do that.

  1. I deployed a web service (terradiem), based on Django, which contains all Celery views
  2. I deployed the same code with a worker and a --beat instruction
  3. I deployed a message broker on redisgreen.
    There is no error, but the worker does not seem to receive the messages.
    Any clue ?
    Please find the content of the render.yalm file below.
    Best,
    Julien

services:

Python web service

  • type: web
    name: terradiem
    env: python
    plan: starter
    repo: https://github.com/jpuaux/terradiem.git
    #healthCheckPath: /health
    region: frankfurt
    branch: master
    numInstances: 3
    buildCommand: pip install -r requirements.txt
    startCommand: gunicorn terradiem.wsgi:application
    domains:
    • www.capelight.com
      envVars:
    • fromGroup: terra-settings
    • key: DATABASE_URL
      fromDatabase:
      name: terradiem-db-new
      property: connectionString
      autoDeploy: true # optional

Background worker that consumes a queue

  • type: worker
    name: terra-queue
    env: python
    buildCommand: pip install -r requirements.txt # contains installation of requirements.txt
    startCommand: celery -A terradiem worker --beat -l info
    envVars:
    • fromGroup: terra-settings
      autoDeploy: true # optional
6

Hi @jpuaux, sorry I missed your response! Were you able to debug this? If not, I’ll be happy to assist.

7

Hi David.
Yes, it is working, thank you.

1 Like
8

please i want to deploy my golang application that has a whatsapp bot which needs a background worker and also i want to also deploy my webservice together with it in the same code but use a background service on render to host it is this possible cause i would create different go rutine for the different instance (whatsapp chat bot and the web service) is this possible?

9

Hi there,

You’ll want two separate services for this kind of setup: one for the bot as a Background Worker and one for the application as a Web Service.

Just let us know if you have any further questions.

Regards,
Mike

Render Support Engineer, MT (UTC-6, UTC-7 in Winter)

10

Hi Mike,

Thanks for the reply earlier i just hosted my web service on render and also paid and deployed my background worker which is a whatsapp bot but i want to also use this whatsapp bot to send otp codes so how do i make my web service to communicate with my background worker since there are no open ports?

you can find screenshot below…

Screen Shot 2024-04-12 at 6.50.41 PM
Screen Shot 2024-04-12 at 6.50.41 PM1293×140 11 KB

11

Hi there,

Background Workers cannot be communicated with, but can have outgoing traffic. If your bot needs to send out OTP codes, perhaps configure a Web Service API to be able to receive these codes at a specific endpoint, which can then be processed from there.

Beyond that, we can’t really provide much in the way of code/design advice.

Regards,
Mike

Render Support Engineer, MT (UTC-6, UTC-7 in Winter)

12

Okay thanks on your reply i would work on it to meet this requirements…