Linked GitHub repo and requirements.txt

Hi,

I’m trying to deploy a test webservice which is a Dash App (so Flask) which is labelled OffshoreOptimalReg28. Which will have a dependency which is another package in my Github (packaged with a setup.py file).

In the requirements.txt file I have added the line.

topgun-optimiser @ git+https://github.com/multistrat/TopGun_Optimiser.git@main

which is the git repo and branch; I’ve tried leaving this blank and using a specific commit. Either way I get the following error when render tries to build

Apr 13 12:36:37 PM  ERROR: Could not find a version that satisfies the requirement topgun-optimiser (unavailable) (from versions: none)
Apr 13 12:36:37 PM  ERROR: No matching distribution found for topgun-optimiser (unavailable)

To test locally I have setup a new virtual environment project then run

pip install -r requirements.txt
python -m index

Which installed all dependencies without issue and is able to run the app locally.

In terms of obvious issues:

  • I’ve included PYTHON_VERSION in the environment variables which is fine (3.9.7)
  • Github is connected in render (auto deploy working fine on this and other projects)

Any help appreciated.
David

Hi David,

Thanks for reaching out.

Your Render account git credentials aren’t passed to the build container, if you need to access private git repositories as part of your build you’ll need to ensure git is authenticated to be able to access it. I’m not too familiar with private packages in Python, but we did have a similar case recently with private git-based packages in Go. The solution in that case ended up being along the lines of:

1. Add a GITHUB_USER & GITHUB_TOKEN to the service as environment variables (you may need to generate a personal access token in GitHub).

2. In the Build Command, reference those variables to set the git config in the build container:

git config --global url."https://${GITHUB_USER}:${GITHUB_TOKEN}@github.com".insteadOf "https://github.com"

It might be an idea to create a render_build.sh script in your code to reference as the Build Command, so that you can add any other steps required for your project setup, e.g. pip install -r requirements.txt.

If could give that a try and let us know how you get on.

Kind regards

Alan

Thanks Alan,

I’ve managed to get something working on a similar theme, using the personal access token. I’ve found that the syntax

package-name @ git+https://${GITHUB_TOKEN}@github.com/user/package-name.git@main

Has worked when I setup “GITHUB_TOKEN” as an environment variable in render. I would request git credentials being passed to the build container as a feature (if possible?). Think the PAT workaround is okay but does mean one either can’t use pip freeze > requirements.txt to build the requirements file (because of the need to manually fudge it) or need to maintain a personal access token (which seems to be considered bad practice).

Being more of a finance person than a dev is there an example available for how to implement a shell script in a build? Not something I know how to do, but would like to try; also think this maybe needed later in my project where I may want a dependency in conda forge.

Thanks,
David

Hi David,

Glad to hear you’ve got something working.

Having git credentials configured outside of the requirements.txt may help you with the “freezing”? It seems we already have a similar private package feature request on our feedback site, it’s related to NPM, but I think the principle is the same in requesting the Render account git creds are passed to the build (seems there’s the same suggestion on a workaround too). Please feel free to upvote/comment: https://feedback.render.com/features/p/allow-private-git-repos-for-npm

I don’t think we have any specific documentation on build scripts, they’re bash scripts, but I will certainly add this to the list of areas of documentation we can improve. Our Django example repo has a simple build script here: https://github.com/render-examples/django-quick-start/blob/master/build.sh

#!/usr/bin/env bashpip install -r requirements/production.txtpython manage.py migratepython manage.py collectstatic

Using that example above, the file is in the root of the project, so the service build command would be changed to ./build.sh and the shell commands in the script would be run at build time.

Hope that helps

Alan

1 Like

Thanks Alan, I’ll have a look and your help is much appreciated.

David

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.