* danielsh/tests-skip-cardinality-v1:
tests: Minor documentation readability tweak
Add a test for issue #641.5, using the infrastructure added in the previous commits.
tests: Skip cardinality tests whenever any test point is expected to fail.
tests: Make $expected_mismatch skip the cardinality check, rather than consider it an expected failure.
tests: Include the name of the 'cardinality check' test point in the output
- Print the test name and data after the plan line
- Split on the plan line rather than on comments
+ That makes tap-filter more suitable to filter TAP output generated by other
TAP producers.
+ However, the filtered output deletes the plan line and adds a blank line in
its stead. This suits our use-case of interactive test runs.
When writing an expected-to-fail test case, the cardinality of $region_highlight
at the time the test is written may differ from the cardinality it will have
once the bug is fixed. For example, with issue #641.5, the current highlighting
is ['nice', 'x=y', 'y', 'ls'] — four elements — but the correct highlighting
would have three elements: ['nice', 'x=y', 'ls']. There is no point in reporting
a separate test failure for the cardinality check in this case, nor for 'ls' being
highlighted as 'command' rather than 'default'.
At the same time, in other cases the current and correct highlighting may have the
same number of elements (for example, this would be the case for a hypothetical
"the command word is highlighted as an alias rather than a function" bug). Thus,
the previous commit, q.v..
than consider it an expected failure.
With this change, if $expected_region_highlight and $region_highlight
coincidentally have the same number of elements, the test won't be considered
to fail.
This is useful in conjunction with the next commit, q.v..
At this time, no tests set $expected_mismatch explicitly. However, the
commit after next (this commit's grandchild) will add a test that will
set $expected_mismatch implicitly, using the functionality in the next
commit (this commit's child).
The new failure mode is:
.
Running test main
Bail out! On './highlighters/main/test-data/glob.zsh': Either 'PREBUFFER' or 'BUFFER' must be declared and non-blank
Bail out! On './highlighters/main/test-data/glob.zsh': output on stderr
Running test pattern
Fixes#478.
