Hi,
I’m getting this error when i do an API request (for generating an image, so i need to wait a little bit longer):
Jul 13 03:50:11 PM [2023-07-13 13:50:11 +0000] [51] [CRITICAL] WORKER TIMEOUT (pid:66)
I already changed the gunicorn’s timeout but i keep having the same issue. I would like to ask you if there are some settings in Render that are related to this.
Render has a 100-minute timeout. The timeout here is the Gunicorn worker timeout, and using -t / --timeout sets this to a specific value. The default is 30 seconds.
If increasing this doesn’t help, it is because something is occurring within your application that is causing the request to be stuck, and the request is still taking longer than the timeout you set. The most basic way to debug this would be to add additional logging statements to your application to figure out where the request gets stuck.
Unfortunately, everytime my application returns error in differents points of the code so the print debuggers are completely useless. Is it maybe possible that the free plan resources are not enough for my application? (it makes various api calls, both for text generation and for image generation)
I looked at the documentation and around the dashboard but i couldn’t find a way to see if the system was overloaded, do you know if i can do this?
Thank you for helping me
Is there anything in common with the points in the code where you get a worker timeout? If you were hitting the resource limits of your service, our platform would restart your application. Are your API calls to 3rd parties? Do you set timeouts on those calls? What is the highest value you have set on the worker timeout? If you set it to say 30 minutes, even though it would be slow, does it eventually work?
Hi,
those are the logs:
Jul 18 07:24:31 PM [2023-07-18 17:24:31 +0000] [51] [CRITICAL] WORKER TIMEOUT (pid:66)
Jul 18 07:24:31 PM creazione Immagine
Jul 18 07:24:31 PM prima di response
Jul 18 07:24:31 PM dentro al try
Jul 18 07:24:31 PM dopo response
Jul 18 07:24:31 PM creazione titolo
Jul 18 07:24:31 PM prima di response
Jul 18 07:24:31 PM dentro al try
Jul 18 07:24:31 PM 127.0.0.1 - - [18/Jul/2023:17:24:31 +0000] “POST /index.html HTTP/1.1” 500 711 “https://ai-article.onrender.com/form.html” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
Jul 18 07:24:31 PM [2023-07-18 17:24:31 +0000] [66] [INFO] Worker exiting (pid: 66)
Jul 18 07:24:32 PM [2023-07-18 17:24:32 +0000] [95] [INFO] Booting worker with pid: 95
as you can see the application return the error, then it print some logs and after that it display my custom error page and some infos.
I’m not sure about what you mean with
but i make a call to openai and to stabilityai with their official API available in their site.
i’ve set this on the gunicorn procfile file:
web: gunicorn app:web_app -t 1800 --keep-alive 1800
Unfortunately even if i set the timeout to 30 minutes it doesn’t work.
Thank you for the help,
Francesco
It is hard to say without seeing your code. It could be that it is getting stuck in IO and when the worker hits the timeout, logs are then drained on getting the SIGTERM signal.
I would suggest putting timeouts on the calls to OpenAI and StabilityAI and see if that prevents the worker timeouts.