zsh-syntax-highlighting/tests
Daniel Shahaf 9b64ad750f tests: Add a 'print failures only' mode to 'make test', called 'make quiet-test'.
Fixes zsh-users/zsh-syntax-highlighting#262.

Currently, 'make quiet-test' uses Perl.  However, since it is considered a development
tool rather than a user-facing tool, users and downstream packages needn't install Perl.
Furthermore, even this dev-only dependency may be dropped in the future.

The only difference between tests/tap-filter here and the one in the issue is using
a `cat` subshell v. using 'undef $/; <STDIN>'.
2016-01-02 21:22:01 +00:00
..
README.md tests: Add a 'print failures only' mode to 'make test', called 'make quiet-test'. 2016-01-02 21:22:01 +00:00
tap-colorizer.zsh 'make test': Fix breakage introduced by 4d3da30f8b: 2015-10-27 11:59:20 +02:00
tap-filter tests: Add a 'print failures only' mode to 'make test', called 'make quiet-test'. 2016-01-02 21:22:01 +00:00
test-highlighting.zsh tests: Add a 'print failures only' mode to 'make test', called 'make quiet-test'. 2016-01-02 21:22:01 +00:00
test-perfs.zsh 'make perf': Unbreak by porting over 'make test' harness changes. 2015-11-18 17:52:46 +00:00

zsh-syntax-highlighting / tests

Utility scripts for testing zsh-syntax-highlighting highlighters.

The tests harness expects the highlighter directory to contain a test-data directory with test data files. See the main highlighter for examples.

Each test should define the array parameter $expected_region_highlight. The value of that parameter is a list of strings of the form "$i $j $style". or "$i $j $style $todo". Each string specifies the highlighting that $BUFFER[$i,$j] should have; that is, $i and $j specify a range, 1-indexed, inclusive of both endpoints. If $todo exists, the test point is marked as TODO (the failure of that test point will not fail the test), and $todo is used as the explanation.

Note: $region_highlight uses the same "$i $j $style" syntax but interprets the indexes differently.

Isolation: Each test is run in a separate subshell, so any variables, aliases, functions, etc., it defines will be visible to the tested code (that computes $region_highlight), but will not affect subsequent tests. The current working directory of tests is set to a newly-created empty directory, which is automatically cleaned up after the test exits.

Highlighting test

test-highlighting.zsh tests the correctness of the highlighting. Usage:

zsh test-highlighting.zsh <HIGHLIGHTER NAME>

All tests may be run with

make test

which will run all highlighting tests and report results in TAP format. By default, the results of all tests will be printed; to show only "interesting" results (tests that failed but were expected to succeed, or vice-versa), run make quiet-test (or make test QUIET=y).

Performance test

test-perfs.zsh measures the time spent doing the highlighting. Usage:

zsh test-perfs.zsh <HIGHLIGHTER NAME>

All tests may be run with

make perf