Git nested submodules not syncing

1

I have a python app running in a background worker instance, that has a git structure that looks something like this : Main repository > Submodule 1 > Nested Submodule 1. I’ve committed and deployed the main repo to Render, submodule 1 is synced up automatically by Render, but Nested Submodule 1 does not sync up. This results in an error that is something like "ModuleNotFoundError: No module named ‘Submodule_1’ ". When debugging I found that an empty folder called Submodule_1 exists where the submodule should exist.

2

To explain my point a little better, what I mean by “synced up” is that Render checks out the referenced commit for Submodule 1, but for Nested Submodule 1 Render does not check out any commits, leaving just an empty folder in the background worker file system

3

Hi there,

Thanks for reaching out.

I don’t think we recursive down through submodules. I don’t have a nested submodules project example to test, but maybe you could update your Build Command to include a git command to get those nested modules? Maybe something like, git submodule update --init --recursive; your_build_commands?

Alan

4

Hey Alan,

Thanks for confirming, I wanted to make sure I wasn’t missing something before I went on to try and find a manual workaround. I’ll give that a shot

5

I tried the above, but looks like it still errors out. I tried it on my local machine and it works fine there though. Must just be whatever is different about the runtime instance on Render. Here is what was logged when I tried to run it

Cloning into ‘/opt/render/project/src/Submodule1/NestedSubmodule1’…
fatal: could not read Username for ‘https://github.com’: No such device or address
fatal: clone of ‘https://github.com/pathToNestedSubmodule1.git’ into submodule path ‘/opt/render/project/src/Submodule1/NestedSubmodule1’ failed
Failed to clone ‘NestedSubmodule1’ Retry scheduled

At this point, unless you have another solution, I think I’ll just try condensing the 2 submodules into a single submodule so I can still deploy to Render and get my app running. Less than ideal but I can’t think of another solution

6

Looks like you need to auth those requests with GitHub, maybe adding something like:

git config --global url."https://${GITHUB_USERNAME}:${GITHUB_ACCESS_TOKEN}@github.com".insteadOf "https://github.com"

to your Build Command (with GITHUB_USERNAME & GITHUB_ACCESS_TOKEN added as environment variables) may get you there?

Alan

7

That looks like it has resolved this issue. Thanks for you help!

8

How can I do this with Dockerfile?
I tried to do with startCommand, but turns out startCommand in not supported while using Dockerfile.

9

I don’t think this would be possible with Docker, from a previous community post:

… we remove the .git folder on Docker builds as we’ve seen scenarios where it caused issues with buildkit caching when present.

Alan

11

Thanks for reply.
So, How can I do this with docker?
If It’s not possible, Is render going to fix this issue? because it should be considered as a bug, Since Nested submodules are a thing, and not syncing them properly is a big issue

12

For now, you would have to workaround it some other way. Maybe vendor the submodule into the project or pull it down within the build, that would be down to what you’re trying to do and how it’s implemented.

I wouldn’t say this is a bug, just that Render doesn’t currently recursively clone through submodules. While, yes, nest submodules are a thing, their use does feel less common. This is only topic I’m aware of on customers having an issue with it.

That said, I agree that it should be possible to do without workarounds, so I’ve raised a feature request: https://feedback.render.com/features/p/nested-submodule-support Feel free to upvote/comment to be notified of any updates.

Alan

13

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.