Document configuring code coverage for component tests (webpack, vite)

[jhiester]

Description

Not clear on how to do code coverage for component testing.

Why is this needed?

It has been really helpful to me and my organization to be able to use the code coverage reports from e2e/integration/unit tests to guide our testing and development. Having component level coverage tests would help in a similar way.

More Info

• Comment: Document configuring code coverage for component tests (webpack, vite) #5519
• Official Docs: https://docs.cypress.io/guides/tooling/code-coverage

Acceptance Criteria

• Document how to set up Code Coverage for Webpack and Vite with simple example repositories
• See this issue for lots of info - ideally we should document doing code coverage w/ Webpack and Vite workflows, and provide 2 example repos.
• If we can add something to docs, that might be good - we've got some info on E2E, but not much for CT (although process is mostly the same): https://docs.cypress.io/guides/tooling/code-coverage

[yann-combarnous]

Example setup for code coverage with Create-React-App, in cypress/plugins/index.js:

// Component / storybook testing
import injectDevServer from '@cypress/react/plugins/react-scripts';
import findReactScriptsWebpackConfig from '@cypress/react/plugins/react-scripts/findReactScriptsWebpackConfig';
import { startDevServer } from '@cypress/webpack-dev-server';
// Code coverage
import codeCoverageTask from '@cypress/code-coverage/task';

// This function is called when a project is opened or re-opened (e.g. due to
// the project's config changing)

/**
 * @type {Cypress.PluginConfig}
 */
// eslint-disable-next-line no-unused-vars
module.exports = (on, config) => {
  // `on` is used to hook into various events Cypress emits
  // `config` is the resolved Cypress config

  // Tell Cypress to report on code coverage
  codeCoverageTask(on, config);

  // Unit/Component testing web dev server setup for instrumenting code coverage
  // For integration testing, this is done in CRA start command
  if (config.testingType === 'component') {
    injectDevServer(on, config);
    const webpackConfig = findReactScriptsWebpackConfig(config, {
      webpackConfigPath: 'react-scripts/config/webpack.config',
    });
    const rules = webpackConfig.module.rules.find((rule) => !!rule.oneOf).oneOf;
    const babelRule = rules.find((rule) => /babel-loader/.test(rule.loader));
    babelRule.options.plugins.push(require.resolve('babel-plugin-istanbul'));

    on('dev-server:start', (options) => {
      return startDevServer({ options, webpackConfig });
    });
  }
  return config;
};

[elevatebart]

Hello @jhiester

I believe there could be two subjects to cover here:

How to get component testing code coverage

• If you are using create react app see in this repo https://github.com/elevatebart/cy-ct-cra
• If you are using Vue see this example https://github.com/cypress-io/cypress/tree/npm/vue/v2/npm/vue/examples/code-coverage

How to merge it with e2e testing coverage

On that front there is good news:
If you run the e2e job and the component job successively, their test coverage going in the same folder, istanbul is smart and will merge the two reports automagically.

If this does not answer your question, feel free to respond below.
If it does, you can close this issue.

Thank you in advance.

[Dorious]

Hey @yann-combarnous, I have similar conf and it doesn't collect coverage for CT, for E2E it does it but after running cypress run-ct it doesn't generate anything.

[elevatebart]

Hello @Dorious

What @yann-combarnous is doing above should be extracting the coverage in Component testing.

But this line

injectDevServer(on, config);

Will do behind the scenes the same as these lines.

on('dev-server:start', (options) => {
  return startDevServer({ options, webpackConfig });
});

They will be redundant.

The first one will inject the server without setting up any coverage.
The second one will inject and have coverage.

We probably should check that in the processing and send a warning.

Did you see that?

[Dorious]

@elevatebart I don't have injectDevServer because I don't use react-scripts. My cypress plugins/index.ts

/// <reference types="cypress" />

import browserify from '@cypress/browserify-preprocessor';
import task from '@cypress/code-coverage/task';
import { initPlugin } from 'cypress-plugin-snapshots/plugin';
import { startDevServer } from '@cypress/webpack-dev-server';

import webpackConfig from '../../webpack.cypress';

/**
 * @type {Cypress.PluginConfig}
 */
const configurePlugins: Cypress.PluginConfig = (on, config) => {
  task(on, config);
  initPlugin(on, config);

  on(
    'file:preprocessor',
    browserify({
      typescript: require.resolve('typescript'),
    })
  );

  if (config.testingType === 'component') {
    // console.log(webpackConfig.module.rules[4].use[0].options);
    on('dev-server:start', (options) => startDevServer({ options, webpackConfig }));
  }

  return config;
};

export default configurePlugins;

The console log is there to check is istanbul there and it is.

[Dorious]

Also I see that the coverage is there each test:

But it doesn't push it into coverage dir. In integration tests (cypress run) it works.

[Dorious]

@elevatebart ok I figured out what was wrong in my case, in support/ I have separate component.ts and we need to put import '@cypress/code-coverage' :) now coverage is collected properly. @jhiester maybe that the trick for you too.

[elevatebart]

Lovely !!!

Code coverage implementation might be a good "recipe" to add to our docs.

To get coverage, you need:

• In plugins/index.js, instrumentation of your code (using babel or another istanbul plugin) in the config you pass to the dev server (that is always the hard part).
• In plugins/index.js implement the code coverage task with the proper task.

const installCoverageTask = require("@cypress/code-coverage/task");
module.exports = (on, config) => {
  installCoverageTask(on, config)
}

• In support/index.js import the code-coverage implementation to run this task when needed.

import '@cypress/code-coverage/support'

[vrknetha]

@elevatebart I have followed all the steps mentioned in the repo. But the instrumentation is not happening at all in my case. we use the same react scripts.

[elevatebart]

@vrknetha What is your stack? Webpack? vite? vue? react?

[elevatebart]

@vrknetha just re-read your answer...
Sorry about that, you already said you are indeed using react-scripts.

Can you share your setup?

• package.json
• cypress.json
• cypress/plugins & cypress/support

A repository would be even better, but I understand not everyone can take this time.

[jovancacvetkovic]

Same thing here, CT code coverage is not working. CRA + Typescript. What finally made it work I added

{
        test: /\.(tsx|ts)?$/,
        include: [SRC_PATH],
        use: [{
          loader: 'istanbul-instrumenter-loader',
          options: { esModules: true }
        }]
      }

just before my typescript loader, in webpack config for

on('dev-server:start', options => startDevServer({
    options,
    webpackConfig
  }));

This is what I have:

const webpackConfig = {
    module: {
      rules: [{
        test: /\.(tsx|ts)?$/,
        include: [SRC_PATH],
        use: [{
          loader: 'istanbul-instrumenter-loader',
          options: { esModules: true }
        }]
      }, {
        test: /\.(tsx|ts)?$/,
        include: [SRC_PATH, CYPRESS_PATH, CYPRESS_NODE_PATH],
        use: [{
          loader: 'awesome-typescript-loader',
          options: {
            configFileName: CONFIG_PATH
          }
        }]
      }]
    },
    resolve: {
      extensions: ['.tsx', '.ts', '.js', '.json']
    }
  };

  on('dev-server:start', options => startDevServer({
    options,
    webpackConfig
  }));

And it works for me like this...
Hope it helps someone.

[elevatebart]

@YeYaLab that really helps. You rock.
@cowboy can you check and add this to the examples when you get a chance?

[cowboy]

Yeah, adding code coverage is absolutely on my list!