Plugins

Plugin Basics

Cucumber Performance supports a number of output and processing plugins. This functions similar to Cucumber plugins with a few notable differences.

• Multiple Options: Each plugin can take up to two arguments. The first can be any number of class types. While the second is a array of string options. This is declared by adding a second colon following the plugins argument.

  • Example: myplugin:argument1:option1,option2,.. or myplugin:argument1:{option1,option2,..}

• Post Fixes: Certain formatter plugins support adding dynamic post fix's to the name of a output file. See the formatter plugins section for details.

• Statistics plugins: Certain plugins can be added to your execution that provide new statistics at the end of your test.

You can add a plugin to your test with the plugin flag "plugin=" or "pg="

Example: pg=default_summary

Also like Cucumber you can add any number of user created plugins. Just make sure that the plugin is in your class path and use the full class name as the id.

Example: pg=test.my.class

Statistics

1. Statistics

   This internal plugin processes the all the non chart related statistics.

   It is enabled by default. Adding a second statistics formatter is generally ignored.

   -----Version 3.0.0 and up------

   You can now specify extra statistics to add to your output. These can be manually specified or passed to the statistics formatter by other plugins.

   To manually specify statistics you can add a statistics creator via the plugin options.

   Example: statistics:stdev,prctl:70

2. Standard Deviation Calculates the standard deviation for each group, scenario and step in your test run.

   Example: statistics:stdev

3. Percentile Calculates the the percentile for each group, scenario and step in your test run. This plugin supports one option which is the percentage to use.

   Example: statistics:prctl:70,prctl:95.5

Display Plugins

1. Null Display

   This is the default display plugin. No output will be made to console.

   Example: null_display

2. Detail Display

   This plugin will write out a single line summary of execution.

   Example: detail_display

Summary Plugins

1. Null Summary

   No output will be made to console.

   Example: null_display

2. Default Summary

   Prints out a complete report for each simulation executed. This is the default.

   Example: default_summary

Formatter Plugins

These support postfix's for file names. The | char separates the path from the postfix.

A auto incremented number with padding is supported:

format: #[PADDING_ZEROS][START_NUM] example: #0001

Time stamps are also supported:

format: @[JAVA_DATE_FORMAT] example: @HHmmss

1. SummaryText Formatter

   Creates a file with the text from the default summary.

   Example: summary_text:file://C:/test/summary

2. Junit Formatter

   Creates a junit report for each scenario

   Example: junit:target/junit-report.xml

3. Chartpoints Formatter

   Creates a csv file with points for creating charts.

   Example: chart_points:file://C:/test/chartpoints

   This can also take a number as an option. Which is the number of points to calculate (with a default of 20).

   Example: chart_points:file://C:/test/chartpoints:50

   -----Version 3.0.0 and up------

   You can add any number of statistics plugins as options. Also multiple charts can be created for each run.

   Example: chart_points:file://C:/test/chartpoints:50,prctl:90.5,prctl:99

4. Summary Text Formatter

   Creates a file with the results summary.

   Example: summary_text:file://C:/test/chartpoints

5. Logger Formatter

   Creates and logs to a file.

   Example: logger:file://C:/test/log.json

   Can also process an existing log file. This was also run any other formatter's after processing the file.

   Example: logger:file://C:/test/log.json:file://C:/test/oldlog.json

6. Taurus Formatter

   -----Version 3.0.0 and up------

   Creates a file in the Taurus Reporting final stats csv format. Automatically adds statistics formatters to fill out columns values.

   Example: taurus:file://C:/test/final_stats.csv

Cucumber

There are a few options in cucumber that will cause issues with Cucumber Perf.

1. Progress Reporter

   This plugin writes out a character for the completion of each scenario. Because of this behavior Display Plugins in Cucumber Perf are disabled on use. This plugin can be useful if you want to run tests from eclipse and verify progress.

   To disable pass in cucumber plugin "cucumber.formatter.NullFormatter" (--plugin cucumber.formatter.NullFormatter)

2. Summary Printer

   This plugin write out a summary of results for each feature completion. Since Cucumber-perf is running many features this can become quite ugly.

   To disable pass in cucumber plugin "null_summary" (-p null_summary)