Hey Brian,
Is the node_modules folder at the root of your repository? If not, it won’t be cached by default. We have an open issue to address that. In the meantime, you can cache additional directories using $XDG_CACHE_HOME to avoid redownloading it every time.
If it is at the root of the repo, I’m happy to investigate further.