Commit Graph

663 Commits

Author SHA1 Message Date
Daniel Shahaf
d4df626c03 'main': Optimize the unquoted arguments characterwise walker
Fixes #730.

% repeat 3 { zsh -f tests/test-zprof.zsh main | tee … | grep -w _zsh_highlight | head -n1 }
19)    1       26787.11 26787.11  100.00%      5.33     5.33    0.02%  _zsh_highlight
19)    1       26900.64 26900.64  100.00%      5.45     5.45    0.02%  _zsh_highlight
19)    1       26915.76 26915.76  100.00%      5.47     5.47    0.02%  _zsh_highlight

Merge remote-tracking branch 'danielsh/perf-argument-1-v2'

* danielsh/perf-argument-1-v2:
  'main': Further optimize argument parsing.
  'main': Optimize a hot path.
  tests: Add a performance testing script, for measuring the performance of the 'main' highlighter on a large file.
2020-05-22 04:43:04 +00:00
Daniel Shahaf
8f5d74d219 'main': Further optimize argument parsing.
% repeat 3 { zsh -f tests/test-zprof.zsh main | tee … | grep -w _zsh_highlight | head -n1 }
18)    1       26895.86 26895.86  100.00%      6.35     6.35    0.02%  _zsh_highlight
19)    1       27399.11 27399.11  100.00%      5.52     5.52    0.02%  _zsh_highlight
19)    1       27027.58 27027.58  100.00%      5.66     5.66    0.02%  _zsh_highlight

----

This commit has been rebased.  The above statistics were measured after
the rebase.  The below statistics had been measured before the rebase.

num  calls                time                       self            name
-----------------------------------------------------------------------------------
 1)    3       25689.17  8563.06   98.15%  18422.01  6140.67   70.38%  _zsh_highlight_main_highlighter_highlight_list
 2) 32390        5706.13     0.18   21.80%   2315.68     0.07    8.85%  _zsh_highlight_main_highlighter_highlight_argument
19)    1       26173.33 26173.33  100.00%      5.27     5.27    0.02%  _zsh_highlight

Interestingly, if I make the change in this diff to
_zsh_highlight_main_highlighter_highlight_double_quote —

>     diff --git a/highlighters/main/main-highlighter.zsh b/highlighters/main/main-highlighter.zsh
>     index da6ab2b..bb17618 100644
>     --- a/highlighters/main/main-highlighter.zsh
>     +++ b/highlighters/main/main-highlighter.zsh
>     @@ -1462,10 +1462,13 @@ _zsh_highlight_main_highlighter_highlight_double_quote()
>        local i j k ret style
>        reply=()
>
>     -  for (( i = $1 + 1 ; i <= $#arg ; i += 1 )) ; do
>     +  (( i = $1 ))
>     +  while (( ++i <= $#arg )); do
>     +    i=${arg[(ib.i.)[\"\`\$\\\\${histchars[1]}]]}
>          (( j = i + start_pos - 1 ))
>          (( k = j + 1 ))
>          case "$arg[$i]" in
>     +      ("")  break;;
>            ('"') break;;
>            ('`') saved_reply=($reply)
>                  _zsh_highlight_main_highlighter_highlight_backtick $i

— it actually makes things measurably slower (!), even on input that has
a large number of pasted double-quoted strings: on «BUFFER=": ${(r.8*1500..foo"bar".):-}"»
the slowdown is (1123.24ms / 1091.06ms = 1.0295).  Therefore, I won't be
committing that change.
2020-05-22 04:36:59 +00:00
Daniel Shahaf
700c0e18fe 'main': Optimize a hot path.
% git co HEAD^ && repeat 3 { zsh -f tests/test-zprof.zsh main | tee … | grep -w _zsh_highlight | head -n1 }
HEAD is now at 64e3651 'main': Optimize a hot path.
19)    1       28765.13 28765.13  100.00%      5.57     5.57    0.02%  _zsh_highlight
19)    1       28566.46 28566.46  100.00%      5.91     5.91    0.02%  _zsh_highlight
19)    1       28248.12 28248.12  100.00%      5.57     5.57    0.02%  _zsh_highlight

----

This commit has been rebased.  The above statistics were measured after
the rebase.  The below statistics had been measured before the rebase.

Before this patch:

    num  calls                time                       self            name
    -----------------------------------------------------------------------------------
     1)    3       33410.81 11136.94   98.51%  19277.07  6425.69   56.84%  _zsh_highlight_main_highlighter_highlight_list
    19)    1       33916.21 33916.21  100.00%      5.27     5.27    0.02%  _zsh_highlight

With this patch:

    num  calls                time                       self            name
    -----------------------------------------------------------------------------------
     1)    3       27167.49  9055.83   98.17%  18754.77  6251.59   67.77%  _zsh_highlight_main_highlighter_highlight_list
    19)    1       27674.40 27674.40  100.00%      5.39     5.39    0.02%  _zsh_highlight

And if test-zprof.zsh is changed to not set interactivecomments:

    num  calls                time                       self            name
    -----------------------------------------------------------------------------------
     1) 13360       36029.12     2.70   83.56%  30304.23     2.27   70.28%  _zsh_highlight_main_highlighter_highlight_argument
    21)    1       43117.76 43117.76  100.00%      4.52     4.52    0.01%  _zsh_highlight

    num  calls                time                       self            name
    -----------------------------------------------------------------------------------
     1) 13360       14782.89     1.11   68.12%   9163.42     0.69   42.23%  _zsh_highlight_main_highlighter_highlight_argument
    21)    1       21699.93 21699.93  100.00%      4.17     4.17    0.02%  _zsh_highlight
2020-05-22 04:32:39 +00:00
Daniel Shahaf
ea3ae74164 'main': Fix the last commit's bug concerning parameter elision not happening in redirects in command position. 2020-05-13 12:51:52 +00:00
Daniel Shahaf
41b8a74692 'main': Add a test for parameter elision not happening in redirects in command position.
Will be fixed in the next commit.
2020-05-13 12:51:40 +00:00
Daniel Shahaf
3a4b212c7d 'main': Fix regression in zsh 5.3.1 and older: all precmd hooks later than z-sy-h would be aborted.
In those versions of zsh, «[[ -o nosuchoption ]]» is regarded as
a syntax error.  In newer zsh versions, it merely returns non-zero
(specifically, it returns 3, unlike «[[ -o unsetoption ]]» which
returns 1).

Fixes #732.
Fixes #733.
2020-05-06 20:27:55 +00:00
Daniel Shahaf
e65ebf0466 'main': Fix a regression caused by the great-grandparent commit's WARN_NESTED_VAR fix.
An error message was emitted on versions of zsh that don't have the
WARN_NESTED_VAR option.

Fixes #731.
2020-05-05 14:31:42 +00:00
Daniel Shahaf
06710f3780 'main': Don't run _zsh_highlight_main__type on every non-command word.
Fixes #728, the performance regression from 0.7.1.
2020-05-04 18:35:15 +00:00
Daniel Shahaf
16d818a21f 'main': Don't trip WARN_NESTED_VAR.
Fixes #727.
2020-05-04 13:27:33 +00:00
Daniel Shahaf
f1b9fbbaf0 'main': Follow-up to previous: Document the version number, and deduplicate some option letters. 2020-04-20 11:11:29 +00:00
Daniel Shahaf
8d32609a7b 'main': precommands += strace 2020-04-20 11:09:24 +00:00
Dimitris Apostolou
415e762ab2
Fix typo 2020-04-08 21:04:48 +03:00
Daniel Shahaf
291634ecfe tests: Add a test for the infinite loop fixed by each of the last two commits.
Accidentally lost during a rebase.
2020-04-03 01:04:31 +00:00
Daniel Shahaf
96e6cbe22f 'main': Fix expansion of positional parameters in _zsh_highlight_main_highlighter__try_expand_parameter.
As described in the last commit's log message, ${parameter_name_pattern]
explicitly matches positional parameters but ${parameters[$MATCH]}
expands to nothing in that case (when, e.g., [[ $MATCH == '1' ]]; note
this is equality of strings, not integers).

As a side effect, this removes the dependency on the zsh/parameter
module for expanding parameters.
2020-04-02 23:52:05 +00:00
Daniel Shahaf
2a30d4fb5a 'main': Fix an infinite loop.
On the test case, the behaviour was as follows:

+highlighters/main/main-highlighter.zsh:733> _zsh_highlight_main_highlighter__try_expand_parameter '$1'
+highlighters/main/main-highlighter.zsh:432> local arg='$1'
+highlighters/main/main-highlighter.zsh:433> unset reply
+highlighters/main/main-highlighter.zsh:439> local -a match mbegin mend
+highlighters/main/main-highlighter.zsh:440> local MATCH
+highlighters/main/main-highlighter.zsh:440> integer MBEGIN MEND
+highlighters/main/main-highlighter.zsh:441> local parameter_name
+highlighters/main/main-highlighter.zsh:442> local -a words
+highlighters/main/main-highlighter.zsh:443> [[ '$' != \$ ]]
+highlighters/main/main-highlighter.zsh:446> [[ 1 == { ]]
+highlighters/main/main-highlighter.zsh:449> parameter_name=1
+highlighters/main/main-highlighter.zsh:451> [[ none == none ]]
+highlighters/main/main-highlighter.zsh:451> zmodload -e zsh/parameter
+highlighters/main/main-highlighter.zsh:452> [[ ${parameter_name} -regex-match ^${~parameter_name_pattern}$ ]]
+highlighters/main/main-highlighter.zsh:453> [[ '' != *special* ]]
+highlighters/main/main-highlighter.zsh:456> case array-special (*array*|*assoc*)
+highlighters/main/main-highlighter.zsh:458> words=( '$1' )
+highlighters/main/main-highlighter.zsh:469> reply=( '$1' )

There are two problems here:

- In terms of _zsh_highlight_main_highlighter__try_expand_parameter's
  pre- and postconditions, the expansion of the word «$1» (line 733)
  included that same word (line 469).

  That happened because word-to-be-expanded is passed to
  _zsh_highlight_main_highlighter__try_expand_parameter as its first
  positional parameter, and in this case the word happened to be «$1».

- Furthermore, the exclusion of special parameters (line 453) false
  negatived.  That happened because $parameter_name_pattern explicitly
  allows positional parameters, but ${parameters[(e)1]} expands to
  nothing.  This will be fixed in the next commit.

Not a regression from 0.7.1.
2020-04-02 23:44:11 +00:00
Daniel Shahaf
90fec4d658 'main': precommands += ionice(1) (from util-linux) 2020-04-01 06:38:41 +00:00
Daniel Shahaf
2aca4e2c02 'main': Make logic more robust. No functional change.
Before this commit, if the value didn't begin with a dollar sign,
_zsh_highlight_main_highlighter__try_expand_parameter() would return 1
by accident.¹  Tweak the input validation to make this behaviour
explicit.  No functional change.

¹ Specifically, it would return 1 because ${parameter_name}'s value
would be the empty string and ${parameter_name_pattern} wouldn't match
that.
2020-03-28 03:57:13 +00:00
Daniel Shahaf
7678a8a227 'main': Break out an anonymous function into a named function.
This is in order to allow it to be reused.

No functional change.
2020-03-27 01:29:11 +00:00
Daniel Shahaf
5d139fcd94 Fix typos in comments. 2020-03-27 01:28:11 +00:00
Matthew Martin
d1802e388e main: Add test for issue #713 2020-03-22 15:01:53 -05:00
Daniel Shahaf
aecfd61bbd 'main': Support the 'env' precommand. 2020-03-21 01:45:34 +00:00
Daniel Shahaf
e4f24f8901
Merge pull request #669 from danielshahaf/cdpath-and-nonexecutable-in-command-position-v1
Command position: non-executable files, non-cdable directories, CDPATH false positives
2020-03-19 21:22:52 +00:00
Daniel Shahaf
1000da306a 'main': Correctly highlight '&&' and '||' inside '[[ … ]]' conditions. 2020-03-19 20:16:43 +00:00
Daniel Shahaf
b44964c545 'main': Highlight reserved words following assignments as errors.
Fixes #461.
2020-03-19 19:48:44 +00:00
Daniel Shahaf
f54d829f03 tests: Add tests for issue #461. 2020-03-19 19:43:12 +00:00
Daniel Shahaf
b0af27f25f tests: Add a unit test for a path specified with mixed quoting.
Fixes #475.
2020-03-19 06:49:40 +00:00
Daniel Shahaf
bd9094cc61 tests: Add a test for issue #498, which has already been fixed. 2020-03-19 06:44:12 +00:00
Daniel Shahaf
c7229a000a tests: Test that global qualifiers and command substitutions aren't evaluated.
Fixes #504.
2020-03-19 06:41:38 +00:00
Daniel Shahaf
f2726d0464 'main': Don't consider path_prefix in alias expansions. 2020-03-19 06:06:30 +00:00
Daniel Shahaf
8184266338 'main': Add a test for aliases to AUTO_CD directories. 2020-03-19 06:06:30 +00:00
Daniel Shahaf
83ac855ceb 'main': Let AUTO_CD directories be highlighted with their own style. 2020-03-19 06:06:30 +00:00
Daniel Shahaf
3f930fb0c1 'main': Add an auxiliary variable for readability. 2020-03-19 06:06:30 +00:00
Daniel Shahaf
5545fb9ab2 'main': In command position, do not highlight directories (unless AUTO_CD is set) and non-executable files.
Fixes #202.

Test expectations are updated.  For example, BUFFER='/bin' is now
highlighted as path_prefix because it's a prefix of '/bin/sh' which
would be valid.  However, BUFFER='/bin;' is now properly highlighted
as an error (unless AUTO_CD is set).
2020-03-19 06:06:30 +00:00
Daniel Shahaf
a6eb966d96 'main': Extend tests to capture the current behaviour.
The next commits will change this behaviour.
2020-03-19 06:05:23 +00:00
Daniel Shahaf
c67372e96c 'main': Add an XFail test for issue #202. 2020-03-19 06:05:23 +00:00
Daniel Shahaf
29ca0bc6c8 'main': Highlight errors from the EQUALS option.
Fixes #430.
2020-03-19 05:59:04 +00:00
Daniel Shahaf
08839bbd87 'main': Let the type determination ignore global aliases when it ignores regular ones. 2020-03-19 05:28:17 +00:00
Daniel Shahaf
e2dddb91c6 'main': Add a regression test for parameters that expand to global aliases.
Will be fixed in the next commit.
2020-03-19 05:27:31 +00:00
Daniel Shahaf
cfef4f3ae0 'main': Enable the zsh/parameter codepath of global aliases highlighting. 2020-03-19 05:13:46 +00:00
Daniel Shahaf
3c5f63d959 'main': Highlight global aliases 2020-03-19 05:05:37 +00:00
Daniel Shahaf
5a44d9f32a tests: Record current behaviour on global aliases. 2020-03-19 05:04:15 +00:00
Daniel Shahaf
fdf23e06c7 'main': Support the "close file descriptor" and "coproc" redirection syntaxes
Part of issue #645.
2020-03-19 03:15:37 +00:00
Daniel Shahaf
10171731f3 tests: Add a test for the "close file descriptor" and "coproc" redirection syntaxes
Part of issue #645.
2020-03-19 03:14:15 +00:00
Daniel Shahaf
dfc41123d7 tests: Fix the test added in the last commit. 2020-03-19 02:30:58 +00:00
Daniel Shahaf
d6defe715a tests: Add a test for issue #705, concerning continuation lines. 2020-03-19 01:39:33 +00:00
Daniel Shahaf
0f627fdf60 Merge remote-tracking branch 'danielsh/whitespace-historical'
* danielsh/whitespace-historical:
  Fix historical instances of one-space indentation.
2020-03-17 17:08:06 +00:00
Daniel Shahaf
e165f18c75 'main': Fix a bug manifesting under zsh 5.2 and older.
The escaped caret was taken for a negated character class.  This caused test-data/arith1.zsh
to XPass: the arithmetic expansion was consumed by the 'while' loop.
2020-03-17 16:13:32 +00:00
Daniel Shahaf
d237a60c9b 'main': Don't highlight arithmetic expansions as command substitutions.
This is not perfect: we don't try to detect cases such as «$((ls); (ls))»,
which look like arithmetic expansions but are in fact command substitutions.

Fixes part of #607.

Introduces #704.
2020-03-17 15:05:32 +00:00
Daniel Shahaf
2e65bb6d7d tests: Add a test documenting the current state, prior to introducing #704. 2020-03-17 15:05:32 +00:00
Daniel Shahaf
e79ce6afd0 'main': Document additional meanings of the 'S' $braces_stack flag. 2020-03-17 14:12:00 +00:00