Prisma Python Missing Binaries

[neos]

Hi guys,

So I have a python web service running quite alright except when I need it to use prisma to connect to a PG db. Here is the error log I get:

Expected /opt/render/project/src/prisma-query-engine-debian-openssl-1.1.x, /opt/render/.cache/prisma-python/binaries/4.15.0/8fbc245156db7124f997f4cecdd8d1219e360944/prisma-query-engine-debian-openssl-1.1.x or /opt/render/.cache/prisma-python/binaries/4.15.0/8fbc245156db7124f997f4cecdd8d1219e360944/node_modules/prisma/query-engine-debian-openssl-1.1.x to exist but none were found or could not be executed. Try running prisma py fetch

And my build command is:

pip install -r requirements.txt && pip install engines-0.4.0-cp38-cp38-linux_x86_64.whl && prisma py generate && prisma py fetch

The build log shows that prisma (0.10.0) has been installed, the prisma client has been generated, and the prisma binaries have been added. So… what could be the issue?

Build Logs

Oct 4 05:27:50 PM  Successfully installed MarkupSafe-2.1.3 anyio-4.0.0 astunparse-1.6.3 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.0 click-8.1.7 cryptography-41.0.2 exceptiongroup-1.1.3 fastapi-0.97.0 graphql-core-3.2.3 gunicorn-21.2.0 h11-0.14.0 httpcore-0.18.0 httpx-0.25.0 idna-3.4 jinja2-3.1.2 maturin-1.2.3 minecart-0.3.0 nodeenv-1.8.0 orjson-3.9.6 packaging-23.2 pdfminer3k-1.3.4 ply-3.11 prisma-0.10.0 pycparser-2.21 pydantic-1.10.13 pypdf-3.11.0 python-dateutil-2.8.2 python-dotenv-1.0.0 pytz-2023.3 qolpy-0.1.2 requests-2.31.0 six-1.16.0 sniffio-1.3.0 starlette-0.27.0 strawberry-graphql-0.199.0 tomli-2.0.1 tomlkit-0.12.1 typing-3.7.4.3 typing-extensions-4.8.0 urllib3-2.0.6 uvicorn-0.22.0 websockets-11.0.3 wheel-0.41.2
Oct 4 05:27:50 PM  WARNING: You are using pip version 21.1.1; however, version 23.2.1 is available.
Oct 4 05:27:50 PM  You should consider upgrading via the '/opt/render/project/src/.venv/bin/python3.8 -m pip install --upgrade pip' command.
Oct 4 05:27:51 PM  Processing ./engines-0.4.0-cp38-cp38-linux_x86_64.whl
Oct 4 05:27:51 PM  Installing collected packages: engines
Oct 4 05:27:51 PM  Successfully installed engines-0.4.0
Oct 4 05:27:51 PM  WARNING: You are using pip version 21.1.1; however, version 23.2.1 is available.
Oct 4 05:27:51 PM  You should consider upgrading via the '/opt/render/project/src/.venv/bin/python3.8 -m pip install --upgrade pip' command.
Oct 4 05:27:51 PM  WARNING: Unsupported version of pydantic installed, this command may not work as intended
Oct 4 05:27:51 PM  Please update pydantic to 1.8 or greater.
Oct 4 05:27:51 PM  
Oct 4 05:27:52 PM  Prisma schema loaded from prisma/schema.prisma
Oct 4 05:27:52 PM  
Oct 4 05:27:52 PM  Some types are disabled by default due to being incompatible with Mypy, it is highly recommended
Oct 4 05:27:52 PM  to use Pyright instead and configure Prisma Python to use recursive types. To re-enable certain types:
Oct 4 05:27:52 PM  
Oct 4 05:27:52 PM  generator client {
Oct 4 05:27:52 PM    provider             = "prisma-client-py"
Oct 4 05:27:52 PM    recursive_type_depth = -1
Oct 4 05:27:52 PM  }
Oct 4 05:27:52 PM  
Oct 4 05:27:52 PM  If you need to use Mypy, you can also disable this message by explicitly setting the default value:
Oct 4 05:27:52 PM  
Oct 4 05:27:52 PM  generator client {
Oct 4 05:27:52 PM    provider             = "prisma-client-py"
Oct 4 05:27:52 PM    recursive_type_depth = 5
Oct 4 05:27:52 PM  }
Oct 4 05:27:52 PM  
Oct 4 05:27:52 PM  For more information see: https://prisma-client-py.readthedocs.io/en/stable/reference/limitations/#default-type-limitations
Oct 4 05:27:52 PM  
Oct 4 05:27:52 PM  
Oct 4 05:27:52 PM  
Oct 4 05:27:52 PM  ✔ Generated Prisma Client Python (v0.10.0) to ./.venv/lib/python3.8/site-packages/prisma in 231ms
Oct 4 05:27:52 PM  
Oct 4 05:27:53 PM  Downloaded binaries to /opt/render/.cache/prisma-python/binaries/4.15.0/8fbc245156db7124f997f4cecdd8d1219e360944

Other Relevant Details

Python Version: 3.8.10
Node Version: 18.16.0

[neos]

Okay so, the solution I’ve found thus far was to change my build command to remove the prisma command from my build command and move it to my start command, as well as remove the py and completely remove the fetch command. What could be the underlying issue here? The py works well in development, however, my assumption is that without it, the prisma CLI is using it’s node counterpart which may be less “buggy”.

Anyways, now my start command looks like this:

prisma generate && gunicorn ...

[Keith]

Hi there,

I believe the main issue with using Prisma on Render is that the default location for where Prisma installs in binaries is not a location that ends up in your final image that runs on Render. The default location and how to configure it is defined here: https://prisma-client-py.readthedocs.io/en/stable/reference/config/#binary-cache-directory. The /{home}/.cache directory isn’t a path we include in the image (home on Render is /opt/render). However, this path is included in our build cache, which is why Prima works okay from builds, but then fails at runtime.

Overall, I’m not that familiar with Prisma, but I have played around with getting it working consistently. What I did was create a render-build.sh which copies the Prisma binaries between a cache location and runtime location. This script looks something like this:

#!/usr/bin/env bash# exit on errorset -o errexitpip install -r requirements.txtprisma generate# Store/pull Prisma cache with build cacheif [[! -d $PRISMA_BINARY_CACHE_DIR]]; then echo "...Copying Prisma Binary Cache from Build Cache" cp -R $XDG_CACHE_HOME/prisma/binaries $PRISMA_BINARY_CACHE_DIRelse echo "...Storing Prisma Binary Cache in Build Cache" cp -R $PRISMA_BINARY_CACHE_DIR $XDG_CACHE_HOMEfi

I use this script as my build command. In my service, I define PRISMA_BINARY_CACHE_DIR as an environment variable with the value /opt/render/project/prisma/binaries so it sets the Prisma cache to a location included in our runtime image. I then just start my service normally with gunicorn ..... with no Prisma commands, this is all done from the build like you originally tried.

Strictly speaking, I think you only “need” the last part with the environment variable, but this will result in Prisma downloading the binaries every build because they are not stored in a location our builds cache. This is what the render-build.sh helps with.

If you use this, please make sure after adding it, you run your first build with a clear build cache. Also, make sure you use at least version 0.10.0 of Prisma as older versions can’t be configured like this.

For completeness for anyone else reading this, if you have used Puppeteer and NodeJS, this is the issue of what can happen with Puppeteer and it downloading Chrome binaries.

Regards,

Keith
Render Support, UTC+10