Docker builds spend 9 minutes downloading cache

#1

Our API is deployed via Docker on Render. We’re seeing the downloading cache step take up to 9 minutes on some deploys which is really slowing things down. Here is a snippet of the logs from a couple days ago:

Jan 2 04:03:35 AM ==> Downloading cache…
Jan 2 04:12:14 AM #2 [internal] load .dockerignore
Jan 2 04:12:14 AM #2 transferring context: 2B done

Anything we can do to make this cache download not take so long?

1 Like
#2

I’m having the same issue, though I don’t think it has anything to do with Docker since it runs before the build.

#3

Hi Mark and Michael,

We’ve made some initial changes to our build nodes to improve cache download speeds.

Let me know if you see an improvement in your next build.

#4

Hi shantanu,

Unfortunately I’m still seeing 5m download times. This is from 15m ago:

Jan 6 05:39:13 PM  ==> Downloading cache...
Jan 6 05:44:21 PM  #1 [internal] load .dockerignore

What cache is being downloaded?

#5

Yep, seeing 11 min on ours:

Jan 6 06:39:15 PM  ==> Downloading cache...
Jan 6 06:50:29 PM  #2 [internal] load .dockerignore
Jan 6 06:50:29 PM  #2 transferring context: 2B done
Jan 6 06:50:29 PM  #2 DONE 0.0s
#6

Hi Michael,

We cache build dependencies and docker build steps where appropriate.

You could try resetting the cache by using the clear build cache and deploy option.

image
image868×354 27.7 KB

While the first build will be slow as the cache will have been cleared, you might see an improvement in subsequent builds.

We are also continuing to look into improving the cache download speeds.

#7

My docker image is ~1GB. Is it expected for 1GB to take 5m to download? That’s quite slow for internal datacenter traffic. If that’s the bandwidth I can expect, I’ll have to move to a different provider.

#8

The cache download was indeed much faster (~30s) after a cache reset and initial warming. Why would that be? It’s building the same docker image.

#9

Hi Michael,

Overtime the cache can get much bigger than the original docker image. I tried looking up your cache size up but it had already been reset.

Resetting/clearing the cache allows us to build the cache up from scratch and does away with old info that is perhaps no longer required.

#10

We’ve made improvements to our cache extraction and fetching processes and those steps should be much faster now.

2 Likes