README.md

SimpleCov

Code coverage for Ruby

• Source Code
• API documentation
• Changelog
• Rubygem
• Continuous Integration

SimpleCov is a code coverage analysis tool for Ruby. It uses Ruby's built-in Coverage library to gather code coverage data, but makes processing its results much easier by providing a clean API to filter, group, merge, format, and display those results, giving you a complete code coverage suite that can be set up with just a couple lines of code. SimpleCov/Coverage track covered ruby code, gathering coverage for common templating solutions like erb, slim and haml is not supported.

In most cases, you'll want overall coverage results for your projects, including all types of tests, Cucumber features, etc. SimpleCov automatically takes care of this by caching and merging results when generating reports, so your report actually includes coverage across your test suites and thereby gives you a better picture of blank spots.

The official formatter of SimpleCov is packaged as a separate gem called simplecov-html, but will be installed and configured automatically when you launch SimpleCov. If you're curious, you can find it on GitHub, too.

Contact

Code and Bug Reports

• Issue Tracker
• See CONTRIBUTING for how to contribute along with some common problems to check out before creating an issue.

Questions, Problems, Suggestions, etc.

• Mailing List "Open mailing list for discussion and announcements on Google Groups"

Getting started

1. Add SimpleCov to your Gemfile and bundle install:

   gem 'simplecov', require: false, group: :test

2. Load and launch SimpleCov at the very top of your test/test_helper.rb (or spec_helper.rb, rails_helper, cucumber env.rb, or whatever your preferred test framework uses):

   require 'simplecov'
   SimpleCov.start
   
   # Previous content of test helper now starts here

   Note: If SimpleCov starts after your application code is already loaded (via require), it won't be able to track your files and their coverage! The SimpleCov.start must be issued before any of your application code is required!

   This is especially true if you use anything that keeps your tests application loaded like spring, check out the spring section.

   SimpleCov must be running in the process that you want the code coverage analysis to happen on. When testing a server process (e.g. a JSON API endpoint) via a separate test process (e.g. when using Selenium) where you want to see all code executed by the rails server, and not just code executed in your actual test files, you need to require SimpleCov in the server process. For rails for instance, you'll want to add something like this to the top of bin/rails, but below the "shebang" line (#! /usr/bin/env ruby) and after config/boot is required:

   if ENV['RAILS_ENV'] == 'test'
     require 'simplecov'
     SimpleCov.start 'rails'
     puts "required simplecov"
   end

3. Run your full test suite to see the percent coverage that your application has.

4. After running your tests, open coverage/index.html in the browser of your choice. For example, in a Mac Terminal, run the following command from your application's root directory:

   open coverage/index.html

   in a debian/ubuntu Terminal,

   xdg-open coverage/index.html

   Note: This guide can help if you're unsure which command your particular operating system requires.

5. Add the following to your .gitignore file to ensure that coverage results are not tracked by Git (optional):

   echo coverage >> .gitignore

   If you're making a Rails application, SimpleCov comes with built-in configurations (see below for information on profiles) that will get you started with groups for your Controllers, Models and Helpers. To use it, the first two lines of your test_helper should be like this:

   require 'simplecov'
   SimpleCov.start 'rails'

Example output

Coverage results report, fully browsable locally with sorting and much more:

Source file coverage details view:

Use it with any framework!

Similarly to the usage with Test::Unit described above, the only thing you have to do is to add the SimpleCov config to the very top of your Cucumber/RSpec/whatever setup file.

Add the setup code to the top of features/support/env.rb (for Cucumber) or spec/spec_helper.rb (for RSpec). Other test frameworks should work accordingly, whatever their setup file may be:

require 'simplecov'
SimpleCov.start 'rails'

You could even track what kind of code your UI testers are touching if you want to go overboard with things. SimpleCov does not care what kind of framework it is running in; it just looks at what code is being executed and generates a report about it.

Notes on specific frameworks and test utilities

For some frameworks and testing tools there are quirks and problems you might want to know about if you want to use SimpleCov with them. Here's an overview of the known ones:

Framework	Notes	Issue
parallel_tests	As of 0.8.0, SimpleCov should correctly recognize parallel_tests and supplement your test suite names with their corresponding test env numbers. SimpleCov locks the resultset cache while merging, ensuring no race conditions occur when results are merged.	#64 & #185
knapsack_pro	To make SimpleCov work with Knapsack Pro Queue Mode to split tests in parallel on CI jobs you need to provide CI node index number to the SimpleCov.command_name in KnapsackPro::Hooks::Queue.before_queue hook.	Tip
RubyMine	The RubyMine IDE has built-in support for SimpleCov's coverage reports, though you might need to explicitly set the output root using `SimpleCov.root('foo/bar/baz')`	#95
Spork	Because of how Spork works internally (using preforking), there used to be trouble when using SimpleCov with it, but that has apparently been resolved with a specific configuration strategy. See this comment.	#42
Spring	See section below.	#381
Test/Unit	Test Unit 2 used to mess with ARGV, leading to a failure to detect the test process name in SimpleCov. test-unit releases 2.4.3+ (Dec 11th, 2011) should have this problem resolved.	#45 & test-unit/test-unit#12

Configuring SimpleCov

Configuration settings can be applied in three formats, which are completely equivalent:

• The most common way is to configure it directly in your start block:

  SimpleCov.start do
    some_config_option 'foo'
  end

• You can also set all configuration options directly:

  SimpleCov.some_config_option 'foo'

• If you do not want to start coverage immediately after launch or want to add additional configuration later on in a concise way, use:

  SimpleCov.configure do
    some_config_option 'foo'
  end

Please check out the Configuration API documentation to find out what you can customize.

Using .simplecov for centralized config

If you use SimpleCov to merge multiple test suite results (e.g. Test/Unit and Cucumber) into a single report, you'd normally have to set up all your config options twice, once in test_helper.rb and once in env.rb.

To avoid this, you can place a file called .simplecov in your project root. You can then just leave the require 'simplecov' in each test setup helper (at the top) and move the SimpleCov.start code with all your custom config options into .simplecov:

# test/test_helper.rb
require 'simplecov'

# features/support/env.rb
require 'simplecov'

# .simplecov
SimpleCov.start 'rails' do
  # any custom configs like groups and filters can be here at a central place
end

Using .simplecov rather than separately requiring SimpleCov multiple times is recommended if you are merging multiple test frameworks like Cucumber and RSpec that rely on each other, as invoking SimpleCov multiple times can cause coverage information to be lost.

Branch coverage (ruby "~> 2.5")

Add branch coverage measurement statistics to your results. Supported in CRuby versions 2.5+.

SimpleCov.start do
  enable_coverage :branch
end

Branch coverage is a feature introduced in Ruby 2.5 concerning itself with whether a particular branch of a condition had been executed. Line coverage on the other hand is only interested in whether a line of code has been executed.

This comes in handy for instance for one line conditionals:

number.odd? ? "odd" : "even"

In line coverage this line would always be marked as executed but you'd never know if both conditions were met. Guard clauses have a similar story:

return if number.odd?

# more code

If all the code in that method was covered you'd never know if the guard clause was ever triggered! With line coverage as just evaluating the condition marks it as covered.

In the HTML report the lines of code will be annotated like branch_type: hit_count:

• then: 2 - the then branch (of an if) was executed twice
• else: 0 - the else branch (of an if or case) was never executed

Not that even if you don't declare an else branch it will still show up in the coverage reports meaning that the condition of the if was not hit or that no when of case was hit during the test runs.

Is branch coverage strictly better? No. Branch coverage really only concerns itself with conditionals - meaning coverage of sequential code is of no interest to it. A file without conditional logic will have no branch coverage data and SimpleCov will report 0 of 0 branches covered as 100% (as everything that can be covered was covered).

Hence, we recommend looking at both metrics together. Branch coverage might also be a good overall metric to look at - while you might be missing only 10% of your lines that might account for 50% of your branches for instance.

Primary Coverage

By default, the primary coverage type is line. To set the primary coverage to something else, use the following:

# or in configure SimpleCov.primary_coverage :branch
SimpleCov.start do
  enable_coverage :branch
  primary_coverage :branch
end

Primary coverage determines what will come in first all output, and the type of coverage to check if you don't specify the type of coverage when customizing exit behavior (SimpleCov.minimum_coverage 90).

Note that coverage must first be enabled for non-default coverage types.

Coverage for eval

You can measure coverage for code that is evaluated by Kernel#eval. Supported in CRuby versions 3.2+.

SimpleCov.start do
  enable_coverage_for_eval
end

This is typically useful for ERB. Set ERB#filename= to make it possible for SimpleCov to trace the original .erb source file.

Filters

Filters can be used to remove selected files from your coverage data. By default, a filter is applied that removes all files OUTSIDE of your project's root directory - otherwise you'd end up with billions of coverage reports for source files in the gems you are using.

You can define your own to remove things like configuration files, tests or whatever you don't need in your coverage report.

Defining custom filters

You can currently define a filter using either a String or Regexp (that will then be Regexp-matched against each source file's path), a block or by passing in your own Filter class.

String filter

SimpleCov.start do
  add_filter "/test/"
end

This simple string filter will remove all files that match "/test/" in their path.

Regex filter

SimpleCov.start do
  add_filter %r{^/test/}
end

This simple regex filter will remove all files that start with /test/ in their path.

Block filter