Commit 7610ef24 authored by Sean Whitton's avatar Sean Whitton

Merge tag 'v1.9'

parents 546c5ed2 18a88795
current_version = 1.7
current_version = 1.9
parse = (?P<major>\d+)\.(?P<minor>.*)
serialize = {major}.{minor}
files = buttercup.el buttercup-pkg.el
......@@ -4,6 +4,8 @@ env:
- EVM_EMACS=emacs-24.3-travis
- EVM_EMACS=emacs-24.4-travis
- EVM_EMACS=emacs-24.5-travis
- EVM_EMACS=emacs-25.1-travis
- EVM_EMACS=emacs-25.2-travis
- curl -fsSkL > && source ./
- evm install "$EVM_EMACS" --use --skip
......@@ -8,8 +8,8 @@ DISTFILES := $(ELISP_FILES) buttercup-pkg.el
all: test
test: compile
$(EMACS) -batch -L . -l buttercup.el -f buttercup-run-markdown docs/
./bin/buttercup -L .
$(EMACS) -batch -L . -l buttercup.el -f buttercup-run-markdown docs/
compile: $(patsubst %.el,%.elc,$(ELISP_FILES))
......@@ -7,12 +7,57 @@ else
usage () {
cat <<EOF
Buttercup will search recursively in each of DIRS for test files (any
elisp file starting with "test-" or ending with "-test.el" or
"-tests.el"). It will load all of those files and then run the tests
defined in those files. If no directories are specified, buttercup
will search in the current directory.
Options can include the options described below, as well as the
standard Emacs options "--directory", "--funcall", "--load", "--eval",
and "--execute". These Emacs options should be used to ensure that any
Elisp files required for the tests can be found in Emacs' load path.
For package testing, "-L ." is commonly used. See "emacs --help" for
more information on Emacs options.
Buttercup options:
--pattern, -p PATTERN Only run tests with names matching PATTERN.
This option can be used multiple times, in
which case tests will be run if they match
any of the given patterns.
--no-color, -c Do not colorize test output.
--traceback STYLE When printing backtraces for errors that occur
during tests, print them in the chosen
STYLE. Available styles are "full", which
shows the full function call for each stack
frame on a single line, "crop", which
truncates each stack frame to 80 characters
(the default), and "pretty", which uses
Emacs' pretty-printing facilities to print
each stack frame, and also annotates each
frame with a lambda or M to indicate whether
it is a normal function call or a
macro/special form.
while [[ "$#" -gt 0 ]]
case "$1" in
......@@ -29,6 +74,12 @@ do
(define-package "buttercup" "1.7"
(define-package "buttercup" "1.9"
"Behavior-Driven Emacs Lisp Testing")
This diff is collapsed.
......@@ -19,7 +19,7 @@ A test suite begins with a call to the Buttercup macro `describe` with
the first parameter describing the suite and the rest being the body
of code that implements the suite.
(describe "A suite"
(it "contains a spec with an expectation"
(expect t :to-be t)))
......@@ -42,7 +42,7 @@ functions internally, so they can contain any executable code
necessary to implement the rules. Emacs Lisp scoping rules apply, so
make sure to define your spec file to be lexically scoped.
(describe "A suite is just a function"
:var (a)
(it "and so is a spec"
......@@ -69,7 +69,7 @@ the spec.
Any matcher can evaluate to a negative assertion by prepending it with
the `:not` matcher.
(describe "The :to-be matcher compares with `eq'"
(it "and has a positive case"
(expect t :to-be t))
......@@ -85,7 +85,7 @@ matchers (see the `buttercup-define-matcher` macro for further
information) for when a project’s domain calls for specific assertions
that are not included below.
(describe "Included matchers:"
(it "The :to-be matcher compares with `eq'"
(let* ((a 12)
......@@ -151,26 +151,30 @@ that are not included below.
(expect pi :to-be-close-to e 0)))
(describe "The :to-throw matcher"
(it "is for testing if a function throws an exception"
(let ((foo (lambda () (+ 1 2)))
(bar (lambda () (+ a 1))))
(expect foo :not :to-throw)
(expect bar :to-throw)))
(it "is for testing if an expression throws an exception"
(expect (+ 1 2) :not :to-throw)
(expect (+ a 1) :to-throw))
(it "accepts a symbol to check for the signal thrown"
(let ((foo (lambda () (/ 1 0)))
(bar (lambda () (+ a 1))))
(expect foo :not :to-throw 'void-variable)
(expect bar :to-throw 'void-variable)))
(expect (/ 1 0) :not :to-throw 'void-variable)
(expect (+ a 1) :to-throw 'void-variable))
(it "optionally matches arguments to signals"
(let ((foo (lambda () (+ a 1)))
(bar (lambda () (+ a 1))))
(expect foo :not :to-throw 'void-variable '(b))
(expect bar :to-throw 'void-variable '(a))))
(it "only works on functions"
(expect (lambda () (expect nil :to-throw 'error))
:to-throw 'void-function)
(expect (lambda () (expect "hello" :not :to-throw 'error))
:to-throw 'invalid-function))))
(expect (+ a 1) :not :to-throw 'void-variable '(b))
(expect (+ a 1) :to-throw 'void-variable '(a)))))
If you are migrating from ERT, you can also use `should` and similar
macros inside a buttercup test just like you would inside an
`ert-deftest` form.
(require 'ert)
(describe "ERT support"
(it "allows you to use ERT macros in tests"
(let* ((a 12)
(b a))
(should (= a b))
(should-not (eq a nil))
(should-error (error "Throws an error")))))
## Grouping Related Specs with `describe`
......@@ -182,7 +186,7 @@ finding specs in a large suite. If you name them well, your specs read
as full sentences in traditional
[BDD]( style.
(describe "A spec"
(it "is just a function, so it can contain any code"
(let ((foo 0))
......@@ -213,7 +217,7 @@ variable under test is defined at the top-level scope — the `describe`
block — and initialization code is moved into a `before-each` block.
The `after-each` block resets the variable before continuing.
(describe "A spec using `before-each' and `after-each'"
:var (foo)
......@@ -241,7 +245,7 @@ However, be careful using `before-all` and `after-all`! Since they are
not reset between specs, it is easy to accidentally leak state between
your specs so that they erroneously pass or fail.
(describe "A spec using `before-all' and `after-all'"
:var (foo)
......@@ -266,7 +270,7 @@ spec is executed, Buttercup walks down the tree executing each
`before-each` function in order. After the spec is executed, Buttercup
walks through the `after-each` functions similarly.
(describe "A spec"
:var (foo)
......@@ -299,7 +303,7 @@ macros, respectively. These suites and any specs inside them are
skipped when run and thus their results will not appear in the
(xdescribe "A spec"
:var (foo)
......@@ -319,7 +323,7 @@ Any spec declared with `xit` is marked as pending.
Any spec declared without a function body will also be marked as
pending in results.
(describe "Pending specs"
(xit "can be declared using `xit'"
(expect t :to-be nil))
......@@ -340,7 +344,7 @@ special matchers for interacting with spies. The
at all. The `:to-have-been-called-with` matcher will return true if
the argument list matches any of the recorded calls to the spy.
(describe "A spy"
:var (foo bar)
......@@ -367,7 +371,7 @@ the argument list matches any of the recorded calls to the spy.
The `:to-have-been-called-times` matcher will return true if the spy
was called a certain number of times.
(describe "A spy"
:var (foo bar)
......@@ -389,7 +393,7 @@ was called a certain number of times.
The keyword argument `:and-call-through` to `spy-on` will make the spy
call the original function instead of returning `nil`.
(describe "A spy, when configured to call through"
:var (bar set-bar get-bar fetched-bar)
......@@ -418,7 +422,7 @@ call the original function instead of returning `nil`.
The keyword argument `:and-return-value` specifies the value the
spied-on function should return.
(describe "A spy, when configured to fake a return value"
:var (bar set-bar get-bar fetched-bar)
......@@ -447,7 +451,7 @@ spied-on function should return.
The keyword argument `:and-call-fake` delegates calls to a supplied
(describe "A spy, when configured with an alternate implementation"
:var (bar set-bar get-bar fetched-bar)
......@@ -476,7 +480,7 @@ function.
With the keyword argument `:and-throw-error`, all calls to the spy
will `signal` the specified value as an error.
(describe "A spy, when configured to throw an error"
:var (bar set-bar get-bar fetched-bar)
......@@ -488,7 +492,7 @@ will `signal` the specified value as an error.
(spy-on 'get-bar :and-throw-error 'error))
(it "throws the error"
(expect (lambda () (get-bar))
(expect (get-bar)
:to-throw 'error)))
......@@ -509,7 +513,7 @@ arguments for the first call.
Finally, `spy-calls-reset` clears all tracking for a spy.
(describe "A spy"
:var (set-foo foo)
......@@ -604,3 +608,12 @@ Finally, `spy-calls-reset` clears all tracking for a spy.
## Warnings in tests
(describe "A test"
(it "can issue warnings while running"
(display-warning 'buttercup "This warning should be visible after the test report.")
(expect (+ 2 2) :to-equal 4)))
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment