In my flask Application I allow users to upload a file so that the app can manipulate the file. The upload happens as expected and the celery task is run. But the celery task runs in a different place than where the files get uploaded to (~/uploads/user-id).
Do i need to programmatically tell celery to look in a different place or can i configure celery to run in the same location as the flask web service?
Hi,
Could you explain a little more about your setup?
At a guess, if users are uploading files, they’re doing so on a Web Service. And usually, Celery would be run on a different service (a Background Worker). Filesystems aren’t shared between services.
In addition, you didn’t mention using a Disk. Render service instances have ephemeral filesystems, meaning any file written to the instance after it has booted will be lost when it next restarts (e.g. spun down if on free instance type, next deploy, manual restart, etc.).
To persist uploaded/generated files, you’d need to use a permanent store, which could be a Render Disk, or some other service (e.g. AWS S3). If you do have the 2-service setup I initially mentioned, a disk also wouldn’t be accessible cross-service. In that case a service like S3 or some other shared store may be the way forward.
Alan
Gotcha, thanks for the help. Yes, I do have the two service setup going as you described. I didn’t realize the file system was ephemeral either 
It sounds like putting these files into an S3 buckets is probably my best solution. I’ve avoided AWS for a long time but looks like my time has come.
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.