Our API is deployed via Docker on Render. We’re seeing the downloading cache step take up to 9 minutes on some deploys which is really slowing things down. Here is a snippet of the logs from a couple days ago:
Jan 2 04:03:35 AM ==> Downloading cache…
Jan 2 04:12:14 AM #2 [internal] load .dockerignore
Jan 2 04:12:14 AM #2 transferring context: 2B done
Anything we can do to make this cache download not take so long?
I’m having the same issue, though I don’t think it has anything to do with Docker since it runs before the build.
Hi Mark and Michael,
We’ve made some initial changes to our build nodes to improve cache download speeds.
Let me know if you see an improvement in your next build.
Hi shantanu,
Unfortunately I’m still seeing 5m download times. This is from 15m ago:
Jan 6 05:39:13 PM ==> Downloading cache...
Jan 6 05:44:21 PM #1 [internal] load .dockerignore
What cache is being downloaded?
Yep, seeing 11 min on ours:
Jan 6 06:39:15 PM ==> Downloading cache...
Jan 6 06:50:29 PM #2 [internal] load .dockerignore
Jan 6 06:50:29 PM #2 transferring context: 2B done
Jan 6 06:50:29 PM #2 DONE 0.0sHi Michael,
We cache build dependencies and docker build steps where appropriate.
You could try resetting the cache by using the clear build cache and deploy option.
While the first build will be slow as the cache will have been cleared, you might see an improvement in subsequent builds.
We are also continuing to look into improving the cache download speeds.
My docker image is ~1GB. Is it expected for 1GB to take 5m to download? That’s quite slow for internal datacenter traffic. If that’s the bandwidth I can expect, I’ll have to move to a different provider.
The cache download was indeed much faster (~30s) after a cache reset and initial warming. Why would that be? It’s building the same docker image.
Hi Michael,
Overtime the cache can get much bigger than the original docker image. I tried looking up your cache size up but it had already been reset.
Resetting/clearing the cache allows us to build the cache up from scratch and does away with old info that is perhaps no longer required.
We’ve made improvements to our cache extraction and fetching processes and those steps should be much faster now.
The cache downloading felt a bit faster for a while, but seems to have slowed down again:
Feb 25 12:52:29 PM ==> Downloading cache…
Feb 25 12:59:22 PM ==> Downloaded 3.0GB in 360s. Extraction took 47s.
And from another service:
Feb 25 12:58:09 PM ==> Downloaded 3.0GB in 286s. Extraction took 54s.
Although not an issue for us, but seeing both having the exact same cache size, is the cache shared between services deploying from the same repository? And would that interfere with faster cache downloads?
Hey there- thanks for reporting this. It’s possible that today’s slowdowns are a result of an incident from this morning, which should be resolved now. You can view more details on this on our status page, if you’re interested.
We’re keeping an eye on this in general though, so please do continue to let us know if you’re seeing cache slowdowns.
In regards to your question, though, no- caches are service specific, so this should not be a factor on cache download speeds.
Additionally, if your cache size has grown, you might want to try doing a clear cache and deploy.
