% git co HEAD^ && repeat 3 { zsh -f tests/test-zprof.zsh main | tee … | grep -w _zsh_highlight | head -n1 }
HEAD is now at f1948df tests: Add a performance testing script, for measuring the performance of the 'main' highlighter on a large file.
19) 1 34378.97 34378.97 100.00% 5.43 5.43 0.02% _zsh_highlight
19) 1 34058.34 34058.34 100.00% 5.50 5.50 0.02% _zsh_highlight
19) 1 34364.80 34364.80 100.00% 5.36 5.36 0.02% _zsh_highlight
(q-) passes through newlines and NUL bytes verbatim. Using (qqqq) ensures the
escaped string will be on a single line (as required by the TAP format) and be
readable even if it contains control characters.
Fixes an issue whereby the '# TODO "issue #687"' directive in the output of
opt-shwordsplit1.zsh was truncated, because the test itself had set the
SH_WORD_SPLIT option and that affected the evaluation of
«${(z)expected_region_highlight[i]}» in the test harness.
Furthermore, this patch also independently fixes the error under
zsh-5.0.8 and earlier that was fixed by the previous commit.
The output of test-data/opt-shwordsplit1.zsh on zsh 5.7 is:
.
1..2
## opt-shwordsplit1
# BUFFER=vim
not ok 1 - [1,7] «$EDITOR» - expected (1 7 "function"), observed (1 7 "unknown-token"). # TODO "issue
ok 2 - cardinality check # SKIP cardinality check disabled whilst regular test points are expected to fail
On zsh 5.0.8, tap_escape() choked when called on the arguments argv=('[1,7]'
'«vim»'). This patch fixes that.
As you may have noticed, under zsh 5.7 the diagnostic message of test point 1
is truncated. That'll be fixed in the next commit.
The cardinality check shall —
- if the test sets \$expected_mismatch, be XFail;
- elif any test points is XFail, be skipped;
- else, be expected to pass.
To test this, change «6 * 9» to «6 + 9» in test-data/arith1.zsh that
will be added in the after-next (grandchild) commit.
* danielsh/tests-skip-cardinality-v1:
tests: Minor documentation readability tweak
Add a test for issue #641.5, using the infrastructure added in the previous commits.
tests: Skip cardinality tests whenever any test point is expected to fail.
tests: Make $expected_mismatch skip the cardinality check, rather than consider it an expected failure.
tests: Include the name of the 'cardinality check' test point in the output
- Print the test name and data after the plan line
- Split on the plan line rather than on comments
+ That makes tap-filter more suitable to filter TAP output generated by other
TAP producers.
+ However, the filtered output deletes the plan line and adds a blank line in
its stead. This suits our use-case of interactive test runs.
When writing an expected-to-fail test case, the cardinality of $region_highlight
at the time the test is written may differ from the cardinality it will have
once the bug is fixed. For example, with issue #641.5, the current highlighting
is ['nice', 'x=y', 'y', 'ls'] — four elements — but the correct highlighting
would have three elements: ['nice', 'x=y', 'ls']. There is no point in reporting
a separate test failure for the cardinality check in this case, nor for 'ls' being
highlighted as 'command' rather than 'default'.
At the same time, in other cases the current and correct highlighting may have the
same number of elements (for example, this would be the case for a hypothetical
"the command word is highlighted as an alias rather than a function" bug). Thus,
the previous commit, q.v..
than consider it an expected failure.
With this change, if $expected_region_highlight and $region_highlight
coincidentally have the same number of elements, the test won't be considered
to fail.
This is useful in conjunction with the next commit, q.v..
At this time, no tests set $expected_mismatch explicitly. However, the
commit after next (this commit's grandchild) will add a test that will
set $expected_mismatch implicitly, using the functionality in the next
commit (this commit's child).
The new failure mode is:
.
Running test main
Bail out! On './highlighters/main/test-data/glob.zsh': Either 'PREBUFFER' or 'BUFFER' must be declared and non-blank
Bail out! On './highlighters/main/test-data/glob.zsh': output on stderr
Running test pattern
Fixes#478.
