Is there a way to speedup npm ci using cache?

Question

for now npm ci is the most common way to install node modules when using CI. But it is honestly really slow. Is there a way to speedup npm ci using cache or do not fully remove existing packages (whole node_modules folder)?

Some interesting benchmarks here that can help [link](http://www.tiernok.com/posts/2019/faster-npm-installs-during-ci/). Also the way **[npm ci](https://docs.npmjs.com/cli/ci)** behaves it is supposed to delete the `node_modules` folder even if it exists for a deterministic build. — ambianBeing, Aug 29 '19 at 09:14

Kiliandeca · Accepted Answer · 2020-02-22T18:21:13.293

NPM cache is located in ~/.npm but in most CIs you can only cache things inside your working directory.

What you can do to circumvent this is changing the cache directory to your current directory with npm set cache .npm. The NPM cache will now be located in ./.npm an you can cache this folder between CI jobs.

Example with GitLab CI:

my-super-job:
  image: node:13-alpine
  script:
    - npm set cache .npm
    - npm ci 
  cache:
    paths:
      - .npm

EDIT: Just discovered that you can set the config as a command line flag so npm ci --cache .npm should do the same

score 10 · Answer 2 · answered Oct 01 '19 at 14:38

tl;dr No.

npm ci should be preferred in CI because it respects the package-lock.json file. Unlike npm install, which rewrites the file and always installs new versions.

By design this command always purges all local packages, by removing the node_modules directory in the beginning. This is the main reason for long builds. And there is no option to avoid this irritating behaviour.

On a local machine you may speed up npm ci by adding the option --prefer-offline, which tells NPM to ignore the cache minimum time and use locally cached packages right away instead of verifying them against the registry.

However, the NPM cache is located in ~/.npm on Unix, or %AppData%/npm-cache on Windows. These folders are not cacheable by default in most CIs. For example GitLab CI caches only have the repository as available workspace. Other directories like the home directory or system directories (eg. apt) are not cached. Therefore this setting probably wont affect your CI build time.

In older version of NPM the option --progress=false had a significant effect on build time, by removing progress bars. This issue seems to be gone however, I can not measure a noteworthy difference anymore.

A best practice and definitely a gain in speed is to separate packages into production and development. By passing the option --only=production NPM will then ignore development dependencies. Due to the reasons above this wont affect caching.

In gitlab CI, you can cache `npm ci` result with the following techniques: https://docs.gitlab.com/ee/ci/caching/#caching-nodejs-dependencies — Saqib Ahmed, Jun 24 '20 at 12:38

score 3 · Answer 3 · edited Apr 21 '21 at 22:29

3

You could tell your CI to cache npm's cache dir, and then use the options --prefer-offline and --no-audit, example:
npm ci --prefer-offline --no-audit

edited Apr 21 '21 at 22:29

riQQ

4,188
4
19
33

answered Aug 26 '19 at 20:58

Philippe GRANET

159
1
5

score 0 · Answer 4 · answered Mar 12 '21 at 20:51

0

This was helpful for me, though it's probably only useful locally https://stackoverflow.com/a/61364681/9727824

Basically using npm ci --production which will skip installing dev-dependencies

answered Mar 12 '21 at 20:51

Sam Lahm

67
7

Is there a way to speedup npm ci using cache?

4 Answers4