Python build fails. No such file: Python.h

I cannot deploy my application because Python development header files are missing:

gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g /render/project/src/.venv/include -I/opt/python-installer/Python-3.10.7/include/python3.10 -c src/hiredis.c -o build/temp.linux-x86_64-cpython-310/src/hiredis.o
In file included from src/hiredis.c:1:
src/hiredis.h:4:10: fatal error: Python.h: No such file or directory
#include <Python.h>
         ^~~~~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1

The package in question is hiredis. It doesn’t have a compiled release (wheel, *.whl) for Python 3.10 and therefore pip wants to build it. How do I go about using it within my app on render.com?

In the documentation page about Native Environments you state that the python3-dev package is available. Shouldn’t it contain the required header files?

Hey there, I took a look at your services and see they’ve all deployed successfully now, did you figure it out?

Regards,

John B

Hi John.

No, unforunately not. I just temporarily removed the packages that needed compiling form my app. I’d like to introduce them again.

Are the Python C header files expected to be present in the Native Environment?

Hi there,
Yes, I’d expect them to be:

binutils \gdal-bin \libgdal-dev \libproj-dev \python3-dev \python3-setuptools \

are what we have installed.

John B

Actually I am facing the same issue as you, I’ve noticed that GDAL and Spatial databases with Python running in Ubuntu (or even locally) need some additional apt-get stuff to be run first before they can be installed

I’ve managed to get them running in docker both locally and in DO Ubuntu 22.0.4, by adding those to the dockerfile to run first before the pip install -r requirements. However I was stumped by the no-logs bug. Will share the fix once it’s working!

Hello @John_B

the problem seems to be that only the header files for Python 2.7 and Python 3.7 are installed while my application uses Python 3.10.7 (which is officially supported).

$ echo $PYTHON_VERSION 
3.10.7

$ lsb_release -d
Description:    Debian GNU/Linux 10 (buster)

$ apt list | grep libpython | grep dev
libpython-all-dev/now 2.7.16-1 amd64 [installed,local]
libpython-dev/now 2.7.16-1 amd64 [installed,local]
libpython2-dev/now 2.7.16-1 amd64 [installed,local]
libpython2.7-dev/now 2.7.16-2+deb10u1 amd64 [installed,local]
libpython3-dev/now 3.7.3-1 amd64 [installed,local]
libpython3.7-dev/now 3.7.3-2+deb10u3 amd64 [installed,local]

Installing a package from PyPI that requires compiling works fine for the system-level Python version (in this case Python 3.7):

$ /usr/local/bin/python --version
Python 3.7.10

$ /usr/local/bin/python -m pip install --no-binary :all: hiredis
Defaulting to user installation because normal site-packages is not writeable
Collecting hiredis
  Using cached hiredis-2.0.0.tar.gz (75 kB)
  Preparing metadata (setup.py) ... done
Skipping wheel build for hiredis, due to binaries being disabled for it.
Installing collected packages: hiredis
  Running setup.py install for hiredis ... done
Successfully installed hiredis-2.0.0

When installing a package using pip on Python 3.10, on the other hand, it fails:

$ which python
/opt/render/project/src/.venv/bin/python

$ python --version
Python 3.10.7

$ python -m pip install hiredis
  […] snip
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Ivendor -I/opt/render/project/src/.venv/include -I/opt/python-installer/Python-3.10.7/include/python3.10 -c src/hiredis.c -o build/temp.linux-x86_64-cpython-310/src/hiredis.o
  In file included from src/hiredis.c:1:
  src/hiredis.h:4:10: fatal error: Python.h: No such file or directory
  #include <Python.h>
           ^~~~~~~~~~
  compilation terminated.
  error: command '/usr/bin/gcc' failed with exit code 1

Is it possible to add the required header files for all supported Python versions?

Same as @jnns , building psycopg2 (from my requirements.txt) under Python 3.10.7 fails for me. I am using psycopg2-binary for now. Thanks!