The “Linking dependencies” step of our static build yarn install is very slow: 6 minutes for 1.3GB unpacked. I would expect that your build cache would include node_modules so I’m not sure why that would be the case.
When we run yarn on our Node server on the same yarn.lock it only takes seconds, implying that something is off about your frontend build cache and it not properly saving/restoring node_modules. Would love help identifying the problem. The static server is srv-bpj8kb08atneofi3geog and an example deploy is dep-c2ephcs64cknvv931kc0.
Is the node_modules folder at the root of your repository? If not, it won’t be cached by default. We have an open issue to address that. In the meantime, you can cache additional directories using $XDG_CACHE_HOME to avoid redownloading it every time.
If it is at the root of the repo, I’m happy to investigate further.
It’s surprising that you are seeing different caching behavior between the two environments since they both use the same caching process. This seems to indicate that the two services are doing something differently that is causing us to fail to correctly cache the artifacts for your static site between builds. I see that your cache download is around 850MB compressed, so it certainly seems to be caching artifacts.
Comparing the build commands between the service you shared and your other node servers, I see that the static site is missing a --frozen-lockfile flag. Is it possible that this is causing your static site to update it’s dependencies on each build and require additional time for the linking step?
Static sites run yarn on their own outside the build step, so I don’t actually have control over the params for the initial run. That said, I tested our backend both with and without frozen-lock file, and saw no difference, so I don’t think that param is a factor.
I also tried clearing the cache for frontend, thinking maybe it had a bum cache, and even after a newly generated and re-fetched cache we’re still seeing slow install times.
I’m having trouble tracking down what might be going wrong here. Having an example to test with would be super helpful. Are you able to share a reproducible example?
@jake can you provide a bit more guidance on what I should set this environment variable to? If I’m in a monorepo where my static site is in frontend/ and therefore my node_modules are at frontend/node_modules, what do I set it to?
I believe @jake’s suggestion is to put things you need cached in the directory already set in $XDG_CACHE_HOME, since we do save that cache. For example, if your node_modules directory isn’t at the root of your project (which we will cache automatically) and instead is in some subfolder, you can still use $XDG_CACHE_HOME to cache it.
An example would be doing something like this as a wrapper to your build script:
#!/usr/bin/env bash
# If we have a cache directory, use it. If not, create it
cache_dir="${XDG_CACHE_HOME}/node_modules"
main_dir="my-app/some-subfolder/node_modules"
mkdir -p "${cache_dir}"
mv "${cache_dir}" "${main_dir}"
# call your normal build script
my-build-script.sh
# put the updated node_modules directory back in the cache so the next build can use it
mv "${main_dir}" "${cache_dir}"