Merge remote-tracking branch 'upstream/master' into upstream/pr/356

* upstream/master:
  'main': Enable fallback to the 'arg0' style.
  'main': Set fallback style for the 'arg0' style.
  dev tools: Automagically handle newlines (\n) in $BUFFER.
  tests: Also test arguments to an anonymous function.
  'main': Directly count spaces to skip, don't leverage proc_buf length
  'main': Highlight several more special (non-alphabetic) parameters.
  'main': Highlight shell's PID ($$) inside double quotes.
  'main': Highlight command substitutions inside double quotes.
  'main': Permit subshells to end at command position.
  'main': Highlight mismatched foreach/end.
  dev tools: Add a TODO to introduce code reuse.
  'main': Highlight mismatched do/done.
  'main': Yet another test for mismatched braces.
  'main' / *_check_path: Precompute $#BUFFER and use it for speed gain
  dev tools: Remove a superfluous empty line.
  'main': Highlight 'always' blocks.
  'main': Don't find command positions within multiline array literals.
  dev tools: Stage the copyright block to make diffs smaller.
  dev tools: Tolerate invalid input.
  'main': Restore compatibility with zsh-4.3.14 and older (after e3182c18de8f).
  dev tools: Automate a recurring step: Set year correctly for new tests.
  'main': Highlight first command word in named functions defined in the sh syntax when MULTI_FUNC_DEF is set.
  'main': More tests for mismstached parentheses and braces.
  'main': Highlight mismatched parentheses and braces.
  driver: Warn just once when a highlighter is missing.
  'main': Support the IGNORE_BRACES option.
  'main': Add tests for the IGNORE_BRACES option.
  'main': Followup to 51614ca2c994: Run cheaper conditions first.
  'brackets': Don't highlight corresponding bracket on accept-line
  driver: Declare global variables
  'main': Avoid triggering a zsh bug related to hashed commands.
  *: error messages: Fix quoting.
  highlighters: Rename entry points.
  driver: Rename highlighter entry points
  driver: Followup to 80148f6c840299f0980f4359ec5307ca63837dff: don't squat on the highlighters' namespace.
  driver: Followup to d711563fe1bf8fa6810bc34ac92a2fd3150290ed: actually make the driver reentrant.
  noop: Rewrap.
  README: Add a reference to upstream's documentation.
  README: Rephrase a question non-negatively^W neutrally.
  noop: Add comments.
  'main': The word after 'sudo' is only a non-command word if it is an option.
  'main': Test for redirection earlier.
  noop: Restructure code for clarity.
  noop: Fix indentation.
  driver: Followup to last: make the value more unique.
  driver: Make it reentrant.
  minor: Fix typo in development usage message.
  driver: Enable highlighting during isearch under zsh≥5.3.
  driver: Don't highlight in isearch
  driver: Always bind zle-line-finish and use it instead of accept-*
  driver: Widget binding: Support binding incomplete/nonexistent widgets
  dev tools: Extend tests/generate.zsh.
  dev tools: Add a script that generates a test-data file.
  'main': Enable test for issue #238.
  'main': Add test for issue #343, concerning the 'command' precommand.
  'main': New test, related to issue #328.
  *: s/echo/print/
  'main': Highlight a broken symlink as a file.
  tests: Add an XFail test for issue #342.
  docs: Minor tweak.
  docs: State highlighters' designated namespace.
  docs: s/myhighlighter/acme/g
  driver: Change a variable name to avoid squatting the highlighters' namespace.
  tests: Add a regression test for issue #267, concerning highlighting a vi linewise region.
  driver: Widget binding: Use ${(k)widgets} instead of $(zle -la)
  Support linewise region.

Conflicts:
	README.md
	zsh-syntax-highlighting.zsh

README.md

@@ -44,6 +44,23 @@ a `zle-line-pre-redraw` hook.  Hooks are run in order of registration,
 therefore, z-sy-h must be sourced (and register its hook) after anything else
 that adds hooks that modify the command-line buffer.
 
+### Does syntax highlighting work during incremental history search?
+
+Highlighting the command line during an incremental history search
+(with the `history-incremental-search-backward` widget, which is
+bound by default to <kbd>Ctrl+R</kbd> in zsh's emacs keymap) requires zsh 5.3
+or newer.
+
+Under zsh 5.2 and older, the zsh-default [underlining][zshzle-Character-Highlighting]
+of the matched portion of the buffer remains available, but zsh-syntax-highlighting's
+additional highlighting is unavailable.  (Those versions of zsh do not provide
+enough information to allow computing the highlighting correctly.)
+
+See [issue #288][i288] for details.
+
+[zshzle-Character-Highlighting]: http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html#Character-Highlighting
+[i288]: https://github.com/zsh-users/zsh-syntax-highlighting/pull/288
+
 ### How are new releases announced?
 
 There is currently no "push" announcements channel.  However, the following

docs/highlighters.md

@@ -49,21 +49,21 @@ highlighter's documentation for details and examples.
 How to implement a new highlighter
 ----------------------------------
 
-To create your own `myhighlighter` highlighter:
+To create your own `acme` highlighter:
 
 * Create your script at
-    `highlighters/${myhighlighter}/${myhighlighter}-highlighter.zsh`.
+    `highlighters/acme/acme-highlighter.zsh`.
 
-* Implement the `_zsh_highlight_myhighlighter_highlighter_predicate` function.
+* Implement the `_zsh_highlight_highlighter_acme_predicate` function.
   This function must return 0 when the highlighter needs to be called and
   non-zero otherwise, for example:
 
-        _zsh_highlight_myhighlighter_highlighter_predicate() {
+        _zsh_highlight_highlighter_acme_predicate() {
           # Call this highlighter in SVN working copies
           [[ -d .svn ]]
         }
 
-* Implement the `_zsh_highlight_myhighlighter_highlighter` function.
+* Implement the `_zsh_highlight_highlighter_acme_paint` function.
   This function does the actual syntax highlighting, by calling
   `_zsh_highlight_add_highlight` with the start and end of the region to
   be highlighted and the `ZSH_HIGHLIGHT_STYLES` key to use. Define the default
@@ -71,15 +71,27 @@ To create your own `myhighlighter` highlighter:
   `: ${ZSH_HIGHLIGHT_STYLES[key]:=value}`, being sure to prefix
   the key with your highlighter name and a colon. For example:
 
-        : ${ZSH_HIGHLIGHT_STYLES[myhighlighter:aurora]:=fg=green}
+        : ${ZSH_HIGHLIGHT_STYLES[acme:aurora]:=fg=green}
 
-        _zsh_highlight_myhighlighter_highlighter() {
+        _zsh_highlight_highlighter_acme_paint() {
           # Colorize the whole buffer with the 'aurora' style
-          _zsh_highlight_add_highlight 0 $#BUFFER myhighlighter:aurora
+          _zsh_highlight_add_highlight 0 $#BUFFER acme:aurora
         }
 
+* Name your own functions and global variables `_zsh_highlight_acme_*`.
+
+    - In zsh-syntax-highlighting 0.4.0 and earlier, the entrypoints 
+        `_zsh_highlight_highlighter_acme_predicate` and
+        `_zsh_highlight_highlighter_acme_paint`
+        were named
+        `_zsh_highlight_acme_highlighter_predicate` and
+        `_zsh_highlight_highlighter_acme_paint` respectively.
+
+        These names are still supported for backwards compatibility;
+        however, support for them will be removed in a a future major or minor release (v0.x.0 or v1.0.0).
+
 * Activate your highlighter in `~/.zshrc`:
 
-        ZSH_HIGHLIGHT_HIGHLIGHTERS+=(myhighlighter)
+        ZSH_HIGHLIGHT_HIGHLIGHTERS+=(acme)
 
 * [Write tests](../tests/README.md).

docs/highlighters/main.md

@@ -44,6 +44,7 @@ This highlighter defines the following styles:
 * `assign` - parameter assignments
 * `redirection` - redirection operators (`<`, `>`, etc)
 * `comment` - comments, when `setopt INTERACTIVE_COMMENTS` is in effect (`echo # foo`)
+* `arg0` - a command word other than one of those enumrated above (other than a command, precommand, alias, function, or shell builtin command).
 * `default` - everything else
 
 To override one of those styles, change its entry in `ZSH_HIGHLIGHT_STYLES`,
@@ -65,4 +66,26 @@ The syntax for values is the same as the syntax of "types of highlighting" of
 the zsh builtin `$zle_highlight` array, which is documented in [the `zshzle(1)`
 manual page][zshzle-Character-Highlighting].
 
+### Useless trivia
+
+#### Forward compatibility.
+
+zsh-syntax-highlighting attempts to be forward-compatible with zsh.
+Specifically, we attempt to facilitate highlighting _command word_ types that
+had not yet been invented when this version of zsh-syntax-highlighting was
+released.
+
+A _command word_ is something like a function name, external command name, et
+cetera.  (See
+[Simple Commands & Pipelines in `zshmisc(1)`][zshmisc-Simple-Commands-And-Pipelines]
+for a formal definition.)
+
+If a new _kind_ of command word is ever added to zsh — something conceptually
+different than "function" and "alias" and "external command" — then command words
+of that (new) kind will be highlighted by the style `arg0_$kind`,
+where `$kind` is the output of `type -w` on the new kind of command word.  If that
+style is not defined, then the style `arg0` will be used instead.
+
+[zshmisc-Simple-Commands-And-Pipelines]: http://zsh.sourceforge.net/Doc/Release/Shell-Grammar.html#Simple-Commands-_0026-Pipelines
+
 [zshzle-Character-Highlighting]: http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html#Character-Highlighting

highlighters/brackets/brackets-highlighter.zsh

@@ -38,13 +38,13 @@
 : ${ZSH_HIGHLIGHT_STYLES[cursor-matchingbracket]:=standout}
 
 # Whether the brackets highlighter should be called or not.
-_zsh_highlight_brackets_highlighter_predicate()
+_zsh_highlight_highlighter_brackets_predicate()
 {
-  _zsh_highlight_cursor_moved || _zsh_highlight_buffer_modified
+  [[ $WIDGET == zle-line-finish ]] || _zsh_highlight_cursor_moved || _zsh_highlight_buffer_modified
 }
 
 # Brackets highlighting function.
-_zsh_highlight_brackets_highlighter()
+_zsh_highlight_highlighter_brackets_paint()
 {
   local char style
   local -i bracket_color_size=${#ZSH_HIGHLIGHT_STYLES[(I)bracket-level-*]} buflen=${#BUFFER} level=0 matchingpos pos
@@ -85,11 +85,13 @@ _zsh_highlight_brackets_highlighter()
     _zsh_highlight_add_highlight $((pos - 1)) $pos $style
   done
 
-  # If cursor is on a bracket, then highlight corresponding bracket, if any
+  # If cursor is on a bracket, then highlight corresponding bracket, if any.
-  pos=$((CURSOR + 1))
+  if [[ $WIDGET != zle-line-finish ]]; then
-  if [[ -n $levelpos[$pos] ]] && [[ -n $matching[$pos] ]]; then
+    pos=$((CURSOR + 1))
-    local -i otherpos=$matching[$pos]
+    if [[ -n $levelpos[$pos] ]] && [[ -n $matching[$pos] ]]; then
-    _zsh_highlight_add_highlight $((otherpos - 1)) $otherpos cursor-matchingbracket
+      local -i otherpos=$matching[$pos]
+      _zsh_highlight_add_highlight $((otherpos - 1)) $otherpos cursor-matchingbracket
+    fi
   fi
 }
 

highlighters/brackets/test-data/cursor-matchingbracket-line-finish.zsh

@@ -0,0 +1,37 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+WIDGET=zle-line-finish
+
+BUFFER=': $foo[bar]'
+CURSOR=6 # cursor is zero-based
+
+expected_region_highlight=(
+  "11 11 NONE"
+)

highlighters/cursor/cursor-highlighter.zsh

@@ -32,17 +32,16 @@
 : ${ZSH_HIGHLIGHT_STYLES[cursor]:=standout}
 
 # Whether the cursor highlighter should be called or not.
-_zsh_highlight_cursor_highlighter_predicate()
+_zsh_highlight_highlighter_cursor_predicate()
 {
-  # accept-* may trigger removal of cursor highlighting
+  # remove cursor highlighting when the line is finished
-  [[ $WIDGET == accept-* ]] ||
+  [[ $WIDGET == zle-line-finish ]] || _zsh_highlight_cursor_moved
-    _zsh_highlight_cursor_moved
 }
 
 # Cursor highlighting function.
-_zsh_highlight_cursor_highlighter()
+_zsh_highlight_highlighter_cursor_paint()
 {
-  [[ $WIDGET == accept-* ]] && return
+  [[ $WIDGET == zle-line-finish ]] && return
   
   _zsh_highlight_add_highlight $CURSOR $(( $CURSOR + 1 )) cursor
 }

highlighters/line/line-highlighter.zsh

@@ -32,13 +32,13 @@
 : ${ZSH_HIGHLIGHT_STYLES[line]:=}
 
 # Whether the root highlighter should be called or not.
-_zsh_highlight_line_highlighter_predicate()
+_zsh_highlight_highlighter_line_predicate()
 {
   _zsh_highlight_buffer_modified
 }
 
 # root highlighting function.
-_zsh_highlight_line_highlighter()
+_zsh_highlight_highlighter_line_paint()
 {
   _zsh_highlight_add_highlight 0 $#BUFFER line
 }

highlighters/main/main-highlighter.zsh

@@ -32,17 +32,11 @@
 : ${ZSH_HIGHLIGHT_STYLES[default]:=none}
 : ${ZSH_HIGHLIGHT_STYLES[unknown-token]:=fg=red,bold}
 : ${ZSH_HIGHLIGHT_STYLES[reserved-word]:=fg=yellow}
-: ${ZSH_HIGHLIGHT_STYLES[alias]:=fg=green}
 : ${ZSH_HIGHLIGHT_STYLES[suffix-alias]:=fg=green,underline}
-: ${ZSH_HIGHLIGHT_STYLES[builtin]:=fg=green}
-: ${ZSH_HIGHLIGHT_STYLES[function]:=fg=green}
-: ${ZSH_HIGHLIGHT_STYLES[command]:=fg=green}
 : ${ZSH_HIGHLIGHT_STYLES[precommand]:=fg=green,underline}
 : ${ZSH_HIGHLIGHT_STYLES[commandseparator]:=none}
-: ${ZSH_HIGHLIGHT_STYLES[hashed-command]:=fg=green}
 : ${ZSH_HIGHLIGHT_STYLES[path]:=underline}
 : ${ZSH_HIGHLIGHT_STYLES[path_pathseparator]:=}
-: ${ZSH_HIGHLIGHT_STYLES[path_prefix]:=underline}
 : ${ZSH_HIGHLIGHT_STYLES[path_prefix_pathseparator]:=}
 : ${ZSH_HIGHLIGHT_STYLES[globbing]:=fg=blue}
 : ${ZSH_HIGHLIGHT_STYLES[history-expansion]:=fg=blue}
@@ -58,13 +52,13 @@
 : ${ZSH_HIGHLIGHT_STYLES[assign]:=none}
 : ${ZSH_HIGHLIGHT_STYLES[redirection]:=none}
 : ${ZSH_HIGHLIGHT_STYLES[comment]:=fg=black,bold}
+: ${ZSH_HIGHLIGHT_STYLES[arg0]:=fg=green}
 
 # Whether the highlighter should be called or not.
-_zsh_highlight_main_highlighter_predicate()
+_zsh_highlight_highlighter_main_predicate()
 {
-  # accept-* may trigger removal of path_prefix highlighting
+  # may need to remove path_prefix highlighting when the line ends
-  [[ $WIDGET == accept-* ]] ||
+  [[ $WIDGET == zle-line-finish ]] || _zsh_highlight_buffer_modified
-    _zsh_highlight_buffer_modified
 }
 
 # Helper to deal with tokens crossing line boundaries.
@@ -72,6 +66,33 @@ _zsh_highlight_main_add_region_highlight() {
   integer start=$1 end=$2
   shift 2
 
+  if (( $+argv[2] )); then
+    # Caller specified inheritance explicitly.
+  else
+    # Automate inheritance.
+    typeset -A fallback_of; fallback_of=(
+        alias arg0
+        suffix-alias arg0
+        builtin arg0
+        function arg0
+        command arg0
+        precommand arg0
+        hashed-command arg0
+        
+        path_prefix path
+        # The path separator fallback won't ever be used, due to the optimisation
+        # in _zsh_highlight_main_highlighter_highlight_path_separators().
+        path_pathseparator path
+        path_prefix_pathseparator path_prefix
+    )
+    local needle=$1 value
+    while [[ -n ${value::=$fallback_of[$needle]} ]]; do
+      unset "fallback_of[$needle]" # paranoia against infinite loops
+      argv+=($value)
+      needle=$value
+    done
+  fi
+
   # The calculation was relative to $PREBUFFER$BUFFER, but region_highlight is
   # relative to $BUFFER.
   (( start -= $#PREBUFFER ))
@@ -114,7 +135,14 @@ _zsh_highlight_main__type() {
       REPLY=builtin
     elif (( $+commands[(e)$1] )); then
       REPLY=command
-    elif ! builtin type -w -- $1 >/dev/null 2>&1; then
+    # zsh 5.2 and older have a bug whereby running 'type -w ./sudo' implicitly
+    # runs 'hash ./sudo=/usr/local/bin/./sudo' (assuming /usr/local/bin/sudo
+    # exists and is in $PATH).  Avoid triggering the bug, at the expense of
+    # falling through to the $() below, incurring a fork.  (Issue #354.)
+    #
+    # The second disjunct mimics the isrelative() C call from the zsh bug.
+    elif {  [[ $1 != */* ]] || is-at-least 5.3 } &&
+         ! builtin type -w -- $1 >/dev/null 2>&1; then
       REPLY=none
     fi
   fi
@@ -149,24 +177,48 @@ _zsh_highlight_main__resolve_alias() {
   fi
 }
 
+# Check that the top of $braces_stack has the expected value.  If it does, set
+# the style according to $2; otherwise, set style=unknown-token.
+#
+# $1: character expected to be at the top of $braces_stack
+# $2: assignment to execute it if matches
+_zsh_highlight_main__stack_pop() {
+  if [[ $braces_stack[1] == $1 ]]; then
+    braces_stack=${braces_stack:1}
+    eval "$2"
+  else
+    style=unknown-token
+  fi
+}
+
 # Main syntax highlighting function.
-_zsh_highlight_main_highlighter()
+_zsh_highlight_highlighter_main_paint()
 {
   ## Before we even 'emulate -L', we must test a few options that would reset.
   if [[ -o interactive_comments ]]; then
     local interactive_comments= # set to empty
   fi
+  if [[ -o ignore_braces ]] || eval '[[ -o ignore_close_braces ]] 2>/dev/null'; then
+    local right_brace_is_recognised_everywhere=false
+  else
+    local right_brace_is_recognised_everywhere=true
+  fi
   if [[ -o path_dirs ]]; then
     integer path_dirs_was_set=1
   else
     integer path_dirs_was_set=0
   fi
+  if [[ -o multi_func_def ]]; then
+    integer multi_func_def=1
+  else
+    integer multi_func_def=0
+  fi
   emulate -L zsh
   setopt localoptions extendedglob bareglobqual
 
   # At the PS3 prompt and in vared, highlight nothing.
   #
-  # (We can't check this in _zsh_highlight_main_highlighter_predicate because
+  # (We can't check this in _zsh_highlight_highlighter_main_predicate because
   # if the predicate returns false, the previous value of region_highlight
   # would be reused.)
   if [[ $CONTEXT == (select|vared) ]]; then
@@ -182,6 +234,14 @@ _zsh_highlight_main_highlighter()
   local -a options_to_set # used in callees
   local buf="$PREBUFFER$BUFFER"
   integer len="${#buf}"
+  integer pure_buf_len=$(( len - ${#PREBUFFER} ))   # == $#BUFFER, used e.g. in *_check_path
+
+  # "R" for round
+  # "Q" for square
+  # "Y" for curly
+  # "D" for do/done
+  # "$" for 'end' (matches 'foreach' always; also used with cshjunkiequotes in repeat/while)
+  local braces_stack
 
   if (( path_dirs_was_set )); then
     options_to_set+=( PATH_DIRS )
@@ -218,6 +278,8 @@ _zsh_highlight_main_highlighter()
     '!' # reserved word; unrelated to $histchars[1]
   )
 
+  local -a match mbegin mend
+
   # State machine
   #
   # The states are:
@@ -228,6 +290,7 @@ _zsh_highlight_main_highlighter()
   #                words) but not in "-ufoo" (one word).
   # - :regular:    "Not a command word", and command delimiters are permitted.
   #                Mainly used to detect premature termination of commands.
+  # - :always:     The word 'always' in the «{ foo } always { bar }» syntax.
   #
   # When the kind of a word is not yet known, $this_word / $next_word may contain
   # multiple states.  For example, after "sudo -i", the next word may be either
@@ -259,6 +322,7 @@ _zsh_highlight_main_highlighter()
   local proc_buf="$buf"
   for arg in ${interactive_comments-${(z)buf}} \
              ${interactive_comments+${(zZ+c+)buf}}; do
+    # Initialize $next_word.
     if (( in_redirection )); then
       (( --in_redirection ))
     fi
@@ -268,6 +332,14 @@ _zsh_highlight_main_highlighter()
     else
       # Stall $next_word.
     fi
+
+    # Initialize per-"simple command" [zshmisc(1)] variables:
+    #
+    #   $already_added       (see next paragraph)
+    #   $style               how to highlight $arg
+    #   $in_array_assignment boolean flag for "between '(' and ')' of array assignment"
+    #   $highlight_glob      boolean flag for "'noglob' is in effect"
+    #
     # $already_added is set to 1 to disable adding an entry to region_highlight
     # for this iteration.  Currently, that is done for "" and $'' strings,
     # which add the entry early so escape sequences within the string override
@@ -281,7 +353,7 @@ _zsh_highlight_main_highlighter()
       fi
     fi
 
-    # advance $start_pos, skipping over whitespace in $buf.
+    # Compute the new $start_pos and $end_pos, skipping over whitespace in $buf.
     if [[ $arg == ';' ]] ; then
       # We're looking for either a semicolon or a newline, whichever comes
       # first.  Both of these are rendered as a ";" (SEPER) by the ${(z)..}
@@ -296,13 +368,27 @@ _zsh_highlight_main_highlighter()
       (( start_pos += offset ))
       (( end_pos = start_pos + $#arg ))
     else
-      integer offset=$(((len-start_pos)-${#${proc_buf##([[:space:]]|\\[[:space:]])#}}))
+      # The line was:
+      #
+      # integer offset=$(((len-start_pos)-${#${proc_buf##([[:space:]]|\\[[:space:]])#}}))
+      #
+      # - len-start_pos is length of current proc_buf; basically: initial length minus where
+      #   we are, and proc_buf is chopped to the "where we are" (compare the "previous value
+      #   of start_pos" below, and the len-(start_pos-offset) = len-start_pos+offset)
+      # - what's after main minus sign is: length of proc_buf without spaces at the beginning
+      # - so what the line actually did, was computing length of the spaces!
+      # - this can be done via (#b) flag, like below
+      if [[ "$proc_buf" = (#b)(#s)(([[:space:]]|\\[[:space:]])##)* ]]; then
+          # The first, outer parenthesis
+          integer offset="${#match[1]}"
+      else
+          integer offset=0
+      fi
       ((start_pos+=offset))
       ((end_pos=$start_pos+${#arg}))
     fi
 
-    # Above `if` computes new start_pos and end_pos.
+    # Compute the new $proc_buf. We advance it
-    # Here we compute new proc_buf. We advance it
     # (chop off characters from the beginning)
     # beyond what end_pos points to, by skipping
     # as many characters as end_pos was advanced.
@@ -321,6 +407,9 @@ _zsh_highlight_main_highlighter()
     # Why [,-1] is slower than [,length] isn't clear.
     proc_buf="${proc_buf[offset + $#arg + 1,len]}"
 
+    # Handle the INTERACTIVE_COMMENTS option.
+    #
+    # We use the (Z+c+) flag so the entire comment is presented as one token in $arg.
     if [[ -n ${interactive_comments+'set'} && $arg[1] == $histchars[3] ]]; then
       if [[ $this_word == *(':regular:'|':start:')* ]]; then
         style=comment
@@ -332,6 +421,19 @@ _zsh_highlight_main_highlighter()
       continue
     fi
 
+    # Analyse the current word.
+    if _zsh_highlight_main__is_redirection $arg ; then
+      # A '<' or '>', possibly followed by a digit
+      in_redirection=2
+    fi
+
+    # Special-case the first word after 'sudo'.
+    if (( ! in_redirection )); then
+      if [[ $this_word == *':sudo_opt:'* ]] && [[ $arg != -* ]]; then
+        this_word=${this_word//:sudo_opt:/}
+      fi
+    fi
+
     # Parse the sudo command line
     if (( ! in_redirection )); then
       if [[ $this_word == *':sudo_opt:'* ]]; then
@@ -349,9 +451,14 @@ _zsh_highlight_main_highlighter()
         next_word+=':sudo_opt:'
         next_word+=':start:'
       fi
-    fi
+   fi
 
-    if [[ $this_word == *':start:'* ]] && (( in_redirection == 0 )); then # $arg is the command word
+   # The Great Fork: is this a command word?  Is this a non-command word?
+   if [[ $this_word == *':always:'* && $arg == 'always' ]]; then
+     # try-always construct
+     style=reserved-word # de facto a reserved word, although not de jure
+     next_word=':start:'
+   elif [[ $this_word == *':start:'* ]] && (( in_redirection == 0 )); then # $arg is the command word
      if [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS:#"$arg"} ]]; then
       style=precommand
      elif [[ "$arg" = "sudo" ]]; then
@@ -382,7 +489,33 @@ _zsh_highlight_main_highlighter()
         fi
       }
       case $res in
-        reserved)       style=reserved-word;;
+        reserved)       # reserved word
+                        style=reserved-word
+                        case $arg in
+                          ($'\x7b')
+                            braces_stack='Y'"$braces_stack"
+                            ;;
+                          ($'\x7d')
+                            # We're at command word, so no need to check $right_brace_is_recognised_everywhere
+                            _zsh_highlight_main__stack_pop 'Y' style=reserved-word
+                            if [[ $style == reserved-word ]]; then
+                              next_word+=':always:'
+                            fi
+                            ;;
+                          ('do')
+                            braces_stack='D'"$braces_stack"
+                            ;;
+                          ('done')
+                            _zsh_highlight_main__stack_pop 'D' style=reserved-word
+                            ;;
+                          ('foreach')
+                            braces_stack='$'"$braces_stack"
+                            ;;
+                          ('end')
+                            _zsh_highlight_main__stack_pop '$' style=reserved-word
+                            ;;
+                        esac
+                        ;;
         'suffix alias') style=suffix-alias;;
         alias)          () {
                           integer insane_alias
@@ -433,10 +566,8 @@ _zsh_highlight_main_highlighter()
                           else
                             style=unknown-token
                           fi
-                        elif _zsh_highlight_main__is_redirection $arg; then
+                        elif (( in_redirection == 2 )); then
-                          # A '<' or '>', possibly followed by a digit
                           style=redirection
-                          (( in_redirection=2 ))
                         elif [[ $arg[1,2] == '((' ]]; then
                           # Arithmetic evaluation.
                           #
@@ -454,10 +585,16 @@ _zsh_highlight_main_highlighter()
                             _zsh_highlight_main_add_region_highlight $((end_pos - 2)) $end_pos $style
                             already_added=1
                           fi
-                        elif [[ $arg == '()' || $arg == $'\x28' ]]; then
+                        elif [[ $arg == '()' ]]; then
                           # anonymous function
+                          style=reserved-word
+                        elif [[ $arg == $'\x28' ]]; then
                           # subshell
                           style=reserved-word
+                          braces_stack='R'"$braces_stack"
+                        elif [[ $arg == $'\x29' ]]; then
+                          # end of subshell
+                          _zsh_highlight_main__stack_pop 'R' style=reserved-word
                         else
                           if _zsh_highlight_main_highlighter_check_path; then
                             style=$REPLY
@@ -466,7 +603,7 @@ _zsh_highlight_main_highlighter()
                           fi
                         fi
                         ;;
-        *)              _zsh_highlight_main_add_region_highlight $start_pos $end_pos commandtypefromthefuture-$res
+        *)              _zsh_highlight_main_add_region_highlight $start_pos $end_pos arg0_$res arg0
                         already_added=1
                         ;;
       esac
@@ -482,9 +619,30 @@ _zsh_highlight_main_highlighter()
                    in_array_assignment=false
                    next_word+=':start:'
                  else
-                   style=reserved-word
+                   _zsh_highlight_main__stack_pop 'R' style=reserved-word
                  fi;;
-        $'\x7d') style=reserved-word;; # block
+        $'\x28\x29') # possibly a function definition
+                 if (( multi_func_def )) || false # TODO: or if the previous word was a command word
+                 then
+                   next_word+=':start:'
+                 fi
+                 style=reserved-word
+                 ;;
+        $'\x7d') # right brace
+                 #
+                 # Parsing rule: # {
+                 #
+                 #     Additionally, `tt(})' is recognized in any position if neither the
+                 #     tt(IGNORE_BRACES) option nor the tt(IGNORE_CLOSE_BRACES) option is set."""
+                 if $right_brace_is_recognised_everywhere; then
+                   _zsh_highlight_main__stack_pop 'Y' style=reserved-word
+                   if [[ $style == reserved-word ]]; then
+                     next_word+=':always:'
+                   fi
+                 else
+                   # Fall through to the catchall case at the end.
+                 fi
+                 ;|
         '--'*)   style=double-hyphen-option;;
         '-'*)    style=single-hyphen-option;;
         "'"*)    style=single-quoted-argument;;
@@ -502,6 +660,8 @@ _zsh_highlight_main_highlighter()
         [*?]*|*[^\\][*?]*)
                  $highlight_glob && style=globbing || style=default;;
         *)       if false; then
+                 elif [[ $arg = $'\x7d' ]] && $right_brace_is_recognised_everywhere; then
+                   # was handled by the $'\x7d' case above
                  elif [[ $arg[0,1] = $histchars[0,1] ]] && (( $#arg[0,2] == 2 )); then
                    style=history-expansion
                  elif [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_COMMANDSEPARATOR:#"$arg"} ]]; then
@@ -510,9 +670,8 @@ _zsh_highlight_main_highlighter()
                    else
                      style=unknown-token
                    fi
-                 elif _zsh_highlight_main__is_redirection $arg; then
+                 elif (( in_redirection == 2 )); then
                    style=redirection
-                   (( in_redirection=2 ))
                  else
                    if _zsh_highlight_main_highlighter_check_path; then
                      style=$REPLY
@@ -528,8 +687,13 @@ _zsh_highlight_main_highlighter()
       [[ $style == path || $style == path_prefix ]] && _zsh_highlight_main_highlighter_highlight_path_separators
     fi
     if [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_COMMANDSEPARATOR:#"$arg"} ]]; then
-      next_word=':start:'
+      if [[ $arg == ';' ]] && $in_array_assignment; then
-      highlight_glob=true
+        # literal newline inside an array assignment
+        next_word=':regular:'
+      else
+        next_word=':start:'
+        highlight_glob=true
+      fi
     elif
        [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_CONTROL_FLOW:#"$arg"} && $this_word == *':start:'* ]] ||
        [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS:#"$arg"} && $this_word == *':start:'* ]]; then
@@ -547,7 +711,12 @@ _zsh_highlight_main_highlighter()
       this_word=':start::regular:'
     fi
     start_pos=$end_pos
-    (( in_redirection == 0 )) && this_word=$next_word
+    if (( in_redirection == 0 )); then
+      # This is the default/common codepath.
+      this_word=$next_word
+    else
+      # Stall $this_word.
+    fi
   done
 }
 
@@ -582,6 +751,7 @@ _zsh_highlight_main_highlighter_check_path()
   REPLY=path
 
   [[ -z $expanded_path ]] && return 1
+  [[ -L $expanded_path ]] && return 0
   [[ -e $expanded_path ]] && return 0
 
   # Search the path in CDPATH
@@ -594,8 +764,8 @@ _zsh_highlight_main_highlighter_check_path()
   [[ ! -d ${expanded_path:h} ]] && return 1
 
   # If this word ends the buffer, check if it's the prefix of a valid path.
-  if [[ ${BUFFER[1]} != "-" && ${#BUFFER} == $end_pos ]] &&
+  if [[ ${BUFFER[1]} != "-" && $pure_buf_len == $end_pos ]] &&
-     [[ $WIDGET != accept-* ]]; then
+     [[ $WIDGET != zle-line-finish ]]; then
     local -a tmp
     tmp=( ${expanded_path}*(N) )
     (( $#tmp > 0 )) && REPLY=path_prefix && return 0
@@ -625,6 +795,16 @@ _zsh_highlight_main_highlighter_highlight_string()
             elif [[ ${arg:$i} =~ ^[{]([A-Za-z_][A-Za-z0-9_]*|[0-9]+)[}] ]] ; then
               (( k += $#MATCH )) # highlight the parameter name and braces
               (( i += $#MATCH )) # skip past it
+            elif [[ $arg[i+1] == '$' ]]; then
+              # $$ - pid
+              (( k += 1 )) # highlight both dollar signs
+              (( i += 1 )) # don't consider the second one as introducing another parameter expansion
+            elif [[ $arg[i+1] == [-#*@?] ]]; then
+              # $#, $*, $@, $?, $- - like $$ above
+              (( k += 1 )) # highlight both dollar signs
+              (( i += 1 )) # don't consider the second one as introducing another parameter expansion
+            elif [[ $arg[i+1] == $'\x28' ]]; then
+              # Highlight just the '$'.
             else
               continue
             fi
@@ -693,7 +873,7 @@ _zsh_highlight_main_highlighter_highlight_dollar_string()
 # Does not perform filename generation (globbing).
 _zsh_highlight_main_highlighter_expand_path()
 {
-  (( $# == 1 )) || echo "zsh-syntax-highlighting: BUG: _zsh_highlight_main_highlighter_expand_path: called without argument" >&2
+  (( $# == 1 )) || print -r -- >&2 "zsh-syntax-highlighting: BUG: _zsh_highlight_main_highlighter_expand_path: called without argument"
 
   # The $~1 syntax normally performs filename generation, but not when it's on the right-hand side of ${x:=y}.
   setopt localoptions nonomatch
@@ -714,7 +894,7 @@ if add-zsh-hook precmd _zsh_highlight_main__precmd_hook 2>/dev/null; then
   # Initialize command type cache
   typeset -gA _zsh_highlight_main__command_type_cache
 else
-  echo 'zsh-syntax-highlighting: Failed to load add-zsh-hook. Some speed optimizations will not be used.' >&2
+  print -r -- >&2 'zsh-syntax-highlighting: Failed to load add-zsh-hook. Some speed optimizations will not be used.'
   # Make sure the cache is unset
   unset _zsh_highlight_main__command_type_cache
 fi

highlighters/main/test-data/always1.zsh

@@ -0,0 +1,41 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='{ ls } always { pwd }'
+
+expected_region_highlight=(
+  '1 1 reserved-word' # {
+  '3 4 command' # ls
+  '6 6 reserved-word' # }
+  '8 13 reserved-word' # always
+  '15 15 reserved-word' # {
+  '17 19 builtin' # pwd
+  '21 21 reserved-word' # }
+)

highlighters/main/test-data/always2.zsh

@@ -0,0 +1,43 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=$'{\nls\n} always { pwd }'
+
+expected_region_highlight=(
+  '1 1 reserved-word' # {
+  '2 2 unknown-token' # \n
+  '3 4 command' # ls
+  '5 5 commandseparator' # \n
+  '6 6 reserved-word' # }
+  '8 13 reserved-word' # always
+  '15 15 reserved-word' # {
+  '17 19 builtin' # pwd
+  '21 21 reserved-word' # }
+)

highlighters/main/test-data/always3.zsh

@@ -0,0 +1,43 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+setopt ignorebraces
+BUFFER='echo { foo } always { bar }'
+
+expected_region_highlight=(
+  '1 4 builtin' # echo
+  '6 6 default' # {
+  '8 10 default' # foo
+  '12 12 default' # }
+  '14 19 default' # always
+  '21 21 default' # {
+  '23 25 default' # bar
+  '27 27 default' # }
+)

highlighters/main/test-data/anonymous-function.zsh

@@ -27,7 +27,7 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
-BUFFER='() echo hello; () { echo world }'
+BUFFER='() echo hello; () { echo world } "argument"'
 
 expected_region_highlight=(
   "1 2 reserved-word" # ()
@@ -37,4 +37,5 @@ expected_region_highlight=(
   "16 17 reserved-word" # ()
   "19 19 reserved-word" # {
   "21 24 builtin" # echo
+  "34 43 double-quoted-argument" # "argument"
 )

highlighters/main/test-data/braces1.zsh

@@ -0,0 +1,41 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=$'() { echo }\n}'
+# no special setopts
+
+expected_region_highlight=(
+  '1 2 reserved-word' # ()
+  '4 4 reserved-word' # {
+  '6 9 builtin' # echo
+  '11 11 reserved-word' # }
+  '12 12 commandseparator' # \n
+  '13 13 unknown-token' # }
+)

highlighters/main/test-data/braces2.zsh

@@ -0,0 +1,41 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=$'() { echo }\n}'
+setopt ignorebraces
+
+expected_region_highlight=(
+  '1 2 reserved-word' # ()
+  '4 4 reserved-word' # {
+  '6 9 builtin' # echo
+  '11 11 default' # }
+  '12 12 commandseparator' # \n
+  '13 13 reserved-word' # }
+)

highlighters/main/test-data/brackets-mismatch1.zsh

@@ -0,0 +1,39 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='() { echo foo )'
+
+expected_region_highlight=(
+  '1 2 reserved-word' # ()
+  '4 4 reserved-word' # {
+  '6 9 builtin' # echo
+  '11 13 default' # foo
+  '15 15 unknown-token' # )
+)

highlighters/main/test-data/brackets-mismatch2.zsh

@@ -0,0 +1,39 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='() ( echo foo }'
+
+expected_region_highlight=(
+  '1 2 reserved-word' # ()
+  '4 4 reserved-word' # (
+  '6 9 builtin' # echo
+  '11 13 default' # foo
+  '15 15 unknown-token' # }
+)

highlighters/main/test-data/brackets-mismatch3.zsh

@@ -0,0 +1,36 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='echo )'
+
+expected_region_highlight=(
+  '1 4 builtin' # echo
+  '6 6 unknown-token' # )
+)

highlighters/main/test-data/brackets-mismatch4.zsh

@@ -0,0 +1,36 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='echo }'
+
+expected_region_highlight=(
+  '1 4 builtin' # echo
+  '6 6 unknown-token' # }
+)

highlighters/main/test-data/brackets-mismatch5.zsh

@@ -0,0 +1,37 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='echo { }'
+
+expected_region_highlight=(
+  '1 4 builtin' # echo
+  '6 6 default' # {
+  '8 8 unknown-token' # }
+)

highlighters/main/test-data/brackets-mismatch6.zsh

@@ -0,0 +1,40 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='(repeat 1; do)'
+
+expected_region_highlight=(
+  '1 1 reserved-word' # (
+  '2 7 reserved-word' # repeat
+  '9 9 default' # 1
+  '10 10 commandseparator' # ;
+  '12 13 reserved-word' # do
+  '14 14 unknown-token' # )
+)

highlighters/main/test-data/brackets-mismatch7.zsh

@@ -0,0 +1,44 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2012 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='for n in *; do echo $n; end'
+
+expected_region_highlight=(
+  '1 3 reserved-word' # for
+  '5 5 default' # n
+  '7 8 default' # in
+  '10 10 globbing' # *
+  '11 11 commandseparator' # ;
+  '13 14 reserved-word' # do
+  '16 19 builtin' # echo
+  '21 22 default' # $n
+  '23 23 commandseparator' # ;
+  '25 27 unknown-token' # end
+)

highlighters/main/test-data/brackets-mismatch8.zsh

@@ -0,0 +1,38 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='(ls&)'
+
+expected_region_highlight=(
+  '1 1 reserved-word' # (
+  '2 3 command' # ls
+  '4 4 commandseparator' # &
+  '5 5 reserved-word' # )
+)

highlighters/main/test-data/dollar-dollar.zsh

@@ -0,0 +1,39 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=': "$$ $$foo"'
+
+expected_region_highlight=(
+  '1 1 builtin' # :
+  '3 3 double-quoted-argument' # "
+  '4 5 dollar-double-quoted-argument' # $$
+  '7 8 dollar-double-quoted-argument' # $$
+  '9 12 double-quoted-argument' # foo"
+)

highlighters/main/test-data/dollar-noise.zsh

@@ -0,0 +1,46 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=': "$- $# $* $@ $?"'
+
+expected_region_highlight=(
+  '1 1 builtin' # :
+  '3 3 double-quoted-argument' # "
+  '4 5 dollar-double-quoted-argument' # $-
+  '6 6 double-quoted-argument' # <space>
+  '7 8 dollar-double-quoted-argument' # $#
+  '9 9 double-quoted-argument' # <space>
+  '10 11 dollar-double-quoted-argument' # $*
+  '12 12 double-quoted-argument' # <space>
+  '13 14 dollar-double-quoted-argument' # $@
+  '15 15 double-quoted-argument' # <space>
+  '16 17 dollar-double-quoted-argument' # $?
+  '18 18 double-quoted-argument' # "
+)

highlighters/main/test-data/dollar-paren.zsh

@@ -0,0 +1,38 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=': "$(:)"'
+
+expected_region_highlight=(
+  '1 1 builtin' # :
+  '3 3 double-quoted-argument' # "
+  '4 4 dollar-double-quoted-argument' # $
+  '5 8 double-quoted-argument' # (:)"
+)

highlighters/main/test-data/exec-redirection1.zsh

@@ -31,7 +31,7 @@ BUFFER='exec {foo}>&/tmp ls'
 
 expected_region_highlight=(
   "1 4 precommand" # exec
-  # TODO: "6 10 redirection 'issue #238'" # {foo}
+  "6 10 redirection 'issue #238'" # {foo}
   "11 12 redirection" # >&
   "13 16 path" # /tmp
   "18 19 command 'issue #238'" # ls

highlighters/main/test-data/function-named1.zsh

@@ -0,0 +1,44 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='f() pwd; f() { balanced braces }'
+
+expected_region_highlight=(
+  '1 1 TBD "issue #223"' # f
+  '2 3 reserved-word' # ()
+  '5 7 builtin' # pwd
+  '8 8 commandseparator' # ;
+  '10 10 TBD "issue #223"' # f
+  '11 12 reserved-word' # ()
+  '14 14 reserved-word' # {
+  '16 23 unknown-token' # balanced
+  '25 30 default' # braces
+  '32 32 reserved-word' # }
+)

highlighters/main/test-data/function-named2.zsh

@@ -0,0 +1,39 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='f g h () pwd'
+
+expected_region_highlight=(
+  '1 1 TBD "issue #223"' # f
+  '3 3 TBD "issue #223"' # g
+  '5 5 TBD "issue #223"' # h
+  '7 8 reserved-word' # ()
+  '10 12 builtin' # pwd
+)

highlighters/main/test-data/inheritance.zsh

@@ -0,0 +1,40 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+_zsh_highlight_add_highlight()
+{
+  region_highlight+=("$1 $2 ${(j.,.)argv[3,-1]}")
+}
+
+BUFFER='type'
+
+expected_region_highlight=(
+  '1 4 builtin,arg0' # type
+)

highlighters/main/test-data/multiline-array-assignment1.zsh

@@ -0,0 +1,39 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=$'foo=(\nbar) env'
+
+expected_region_highlight=(
+  '1 5 assign' # foo=(
+  '6 6 commandseparator' # \n
+  '7 9 default' # bar
+  '10 10 assign' # )
+  '12 14 command' # env
+)

highlighters/main/test-data/path-broken-symlink.zsh

@@ -0,0 +1,36 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+ln -s /nonexistent broken-symlink
+BUFFER=': broken-symlink'
+CURSOR=5 # to make path_prefix ineligible
+
+expected_region_highlight=(
+  "3 16 path" # broken-symlink
+)

highlighters/main/test-data/path-dollared-word2.zsh

@@ -0,0 +1,36 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+lambda="''"
+touch \$lambda
+BUFFER=': \$lambda'
+
+expected_region_highlight=(
+  "3 8 path" # \$lambda
+)

highlighters/main/test-data/path_prefix2.zsh

@@ -31,7 +31,7 @@
 # Related to path_prefix.zsh
 
 BUFFER='ls /bin/s'
-WIDGET=accept-line
+WIDGET=zle-line-finish
 
 expected_region_highlight=(
   "4 9 default"    # /bin/s

highlighters/main/test-data/precommand2.zsh

@@ -0,0 +1,36 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='command -v ls'
+
+expected_region_highlight=(
+  "1 7 precommand" # command
+  "9 10 single-hyphen-option 'issue #343'" # -v
+  "12 13 command 'issue #343'" # ls
+)

highlighters/main/test-data/sudo-command.zsh

@@ -31,7 +31,7 @@
 # * -i  (no argument)
 # * -C3 (pasted argument)
 # * -u otheruser (non-pasted argument)
-BUFFER='sudo -C3 -u otheruser -i ls /; sudo ; sudo -u ;'
+BUFFER='sudo -C3 -u otheruser -i ls /; sudo ; sudo -u ; sudo notacommand'
 
 expected_region_highlight=(
   "1 4 precommand" # sudo
@@ -43,4 +43,5 @@ expected_region_highlight=(
   "29 29 path" # /
   "37 37 unknown-token" # ;, error because empty command
   "47 47 unknown-token" # ;, error because incomplete command
+  "54 64 unknown-token" # notacommand - doesn't falls back to "not a command word" codepath
 )

highlighters/main/test-data/vi-linewise-mode.zsh

@@ -0,0 +1,38 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+# See issue #267 for the magic numbers
+BUFFER=$'foo foo\nbar bar'
+REGION_ACTIVE=2
+CURSOR=4
+MARK=12
+
+expected_region_highlight=(
+  "1 3 standout" # foo
+)

highlighters/pattern/pattern-highlighter.zsh

@@ -32,13 +32,13 @@
 typeset -gA ZSH_HIGHLIGHT_PATTERNS
 
 # Whether the pattern highlighter should be called or not.
-_zsh_highlight_pattern_highlighter_predicate()
+_zsh_highlight_highlighter_pattern_predicate()
 {
   _zsh_highlight_buffer_modified
 }
 
 # Pattern syntax highlighting function.
-_zsh_highlight_pattern_highlighter()
+_zsh_highlight_highlighter_pattern_paint()
 {
   setopt localoptions extendedglob
   local pattern

highlighters/root/root-highlighter.zsh

@@ -32,13 +32,13 @@
 : ${ZSH_HIGHLIGHT_STYLES[root]:=standout}
 
 # Whether the root highlighter should be called or not.
-_zsh_highlight_root_highlighter_predicate()
+_zsh_highlight_highlighter_root_predicate()
 {
   _zsh_highlight_buffer_modified
 }
 
 # root highlighting function.
-_zsh_highlight_root_highlighter()
+_zsh_highlight_highlighter_root_paint()
 {
   if (( EUID == 0 )) { _zsh_highlight_add_highlight 0 $#BUFFER root }
 }

tests/README.md

@@ -39,6 +39,21 @@ which is automatically cleaned up after the test exits. For example:
       "1 21 command" # bar/testing-issue-228
     )
 
+
+Writing new tests
+-----------------
+
+An experimental tool is available to generate test files:
+
+    zsh -f tests/generate.zsh 'ls -x' acme newfile
+
+This generates a `highlighters/acme/test-data/newfile.zsh` test file based on
+the current highlighting of the given `$BUFFER` (in this case, `ls -x`).
+
+_This tool is experimental._  Its interface may change.  In particular it may
+grow ways to set `$PREBUFFER` to inject free-form code into the generated file.
+
+
 Highlighting test
 -----------------
 

tests/generate.zsh

@@ -0,0 +1,99 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+emulate -LR zsh
+setopt localoptions extendedglob
+
+# Argument parsing.
+if (( $# != 3 )) || [[ $1 == -* ]]; then
+  print -r -- >&2 "$0: usage: $0 BUFFER HIGHLIGHTER BASENAME"
+  print -r -- >&2 ""
+  print -r -- >&2 "Generate highlighters/HIGHLIGHTER/test-data/BASENAME.zsh based on the"
+  print -r -- >&2 "current highlighting of BUFFER."
+  exit 1
+fi
+buffer=$1
+ZSH_HIGHLIGHT_HIGHLIGHTERS=( $2 )
+fname=${0:A:h:h}/highlighters/$2/test-data/${3%.zsh}.zsh
+
+# Load the main script.
+. ${0:A:h:h}/zsh-syntax-highlighting.zsh
+
+# Overwrite _zsh_highlight_add_highlight so we get the key itself instead of the style
+_zsh_highlight_add_highlight()
+{
+  region_highlight+=("$1 $2 $3")
+}
+
+
+# Copyright block
+year="`LC_ALL=C date +%Y`"
+if ! read -q "?Set copyright year to $year? "; then
+  year="YYYY"
+fi
+exec >$fname
+<$0 sed -n -e '1,/^$/p' | sed -e "s/2[0-9][0-9][0-9]/${year}/"
+# Assumes stdout is line-buffered
+git add -- $fname
+
+# Buffer
+print -n 'BUFFER='
+if [[ $buffer != (#s)[$'\t -~']#(#e) ]]; then
+  print -r -- ${(qqqq)buffer}
+else
+  print -r -- ${(qq)buffer}
+fi
+echo ""
+
+# Expectations
+print 'expected_region_highlight=('
+() {
+  local i
+  local PREBUFFER
+  local BUFFER
+  
+  PREBUFFER=""
+  BUFFER="$buffer"
+  region_highlight=()
+  # TODO: use run_test() from tests/test-highlighting.zsh (to get a tempdir)
+  _zsh_highlight
+
+  for ((i=1; i<=${#region_highlight}; i++)); do
+    local -a highlight_zone; highlight_zone=( ${(z)region_highlight[$i]} )
+    integer start=$highlight_zone[1] end=$highlight_zone[2]
+    if (( start < end )) # region_highlight ranges are half-open
+    then
+      (( --end )) # convert to closed range, like expected_region_highlight
+      (( ++start, ++end )) # region_highlight is 0-indexed; expected_region_highlight is 1-indexed
+    fi
+    printf "  %s # %s\n" ${(qq):-"$start $end $highlight_zone[3]"} ${${(qqqq)BUFFER[start,end]}[3,-2]}
+  done
+}
+print ')'

zsh-syntax-highlighting.zsh

@@ -32,8 +32,8 @@
 0=${(%):-%N}
 if true; then
   # $0 is reliable
-  ZSH_HIGHLIGHT_VERSION=$(<"${0:A:h}"/.version)
+  typeset -g ZSH_HIGHLIGHT_VERSION=$(<"${0:A:h}"/.version)
-  ZSH_HIGHLIGHT_REVISION=$(<"${0:A:h}"/.revision-hash)
+  typeset -g ZSH_HIGHLIGHT_REVISION=$(<"${0:A:h}"/.revision-hash)
   if [[ $ZSH_HIGHLIGHT_REVISION == \$Format:* ]]; then
     # When running from a source tree without 'make install', $ZSH_HIGHLIGHT_REVISION
     # would be set to '$Format:%H$' literally.  That's an invalid value, and obtaining
@@ -102,6 +102,13 @@ _zsh_highlight()
   # Store the previous command return code to restore it whatever happens.
   local ret=$?
 
+  # Remove all highlighting in isearch, so that only the underlining done by zsh itself remains.
+  # For details see FAQ entry 'Why does syntax highlighting not work while searching history?'.
+  if [[ $WIDGET == zle-isearch-update ]] && ! (( $+ISEARCHMATCH_ACTIVE )); then
+    region_highlight=()
+    return $ret
+  fi
+
   setopt localoptions warncreateglobal
   setopt localoptions noksharrays
   local REPLY # don't leak $REPLY into global scope
@@ -125,11 +132,15 @@ _zsh_highlight()
     local highlighter; for highlighter in $ZSH_HIGHLIGHT_HIGHLIGHTERS; do
 
       # eval cache place for current highlighter and prepare it
-      cache_place="_zsh_highlight_${highlighter}_highlighter_cache"
+      cache_place="_zsh_highlight__highlighter_${highlighter}_cache"
       typeset -ga ${cache_place}
 
       # If highlighter needs to be invoked
-      if "_zsh_highlight_${highlighter}_highlighter_predicate"; then
+      if ! type "_zsh_highlight_highlighter_${highlighter}_predicate" >&/dev/null; then
+        echo "zsh-syntax-highlighting: warning: disabling the ${(qq)highlighter} highlighter as it has not been loaded" >&2
+        # TODO: use ${(b)} rather than ${(q)} if supported
+        ZSH_HIGHLIGHT_HIGHLIGHTERS=( ${ZSH_HIGHLIGHT_HIGHLIGHTERS:#${highlighter}} )
+      elif "_zsh_highlight_highlighter_${highlighter}_predicate"; then
 
         # save a copy, and cleanup region_highlight
         region_highlight_copy=("${region_highlight[@]}")
@@ -137,7 +148,7 @@ _zsh_highlight()
 
         # Execute highlighter and save result
         {
-          "_zsh_highlight_${highlighter}_highlighter"
+          "_zsh_highlight_highlighter_${highlighter}_paint"
         } always {
           eval "${cache_place}=(\"\${region_highlight[@]}\")"
         }
@@ -155,7 +166,22 @@ _zsh_highlight()
     # Re-apply zle_highlight settings
 
     # region
-    (( REGION_ACTIVE )) && _zsh_highlight_apply_zle_highlight region standout "$MARK" "$CURSOR"
+    if (( REGION_ACTIVE == 1 )); then
+      _zsh_highlight_apply_zle_highlight region standout "$MARK" "$CURSOR"
+    elif (( REGION_ACTIVE == 2 )); then
+      () {
+        local needle=$'\n'
+        integer min max
+        if (( MARK > CURSOR )) ; then
+          min=$CURSOR max=$MARK
+        else
+          min=$MARK max=$CURSOR
+        fi
+        (( min = ${${BUFFER[1,$min]}[(I)$needle]} ))
+        (( max += ${${BUFFER:($max-1)}[(i)$needle]} - 1 ))
+        _zsh_highlight_apply_zle_highlight region standout "$min" "$max"
+      }
+    fi
 
     # yank / paste (zsh-5.1.1 and newer)
     (( $+YANK_ACTIVE )) && (( YANK_ACTIVE )) && _zsh_highlight_apply_zle_highlight paste standout "$YANK_START" "$YANK_END"
@@ -288,43 +314,64 @@ else
   _zsh_highlight_bind_widgets()
   {
     setopt localoptions noksharrays
-
+    typeset -F SECONDS
+    local prefix=orig-s$SECONDS-r$RANDOM # unique each time, in case we're sourced more than once
+  
     # Load ZSH module zsh/zleparameter, needed to override user defined widgets.
     zmodload zsh/zleparameter 2>/dev/null || {
-      echo 'zsh-syntax-highlighting: failed loading zsh/zleparameter.' >&2
+      print -r -- >&2 'zsh-syntax-highlighting: failed loading zsh/zleparameter.'
       return 1
     }
 
     # Override ZLE widgets to make them invoke _zsh_highlight.
+    local -U widgets_to_bind
+    widgets_to_bind=(${${(k)widgets}:#(.*|run-help|which-command|beep|set-local-history|yank)})
+  
+    # Always wrap special zle-line-finish widget. This is needed to decide if the
+    # current line ends and special highlighting logic needs to be applied.
+    # E.g. remove cursor imprint, don't highlight partial paths, ...
+    widgets_to_bind+=(zle-line-finish)
+  
+    # Always wrap special zle-isearch-update widget to be notified of updates in isearch.
+    # This is needed because we need to disable highlighting in that case.
+    widgets_to_bind+=(zle-isearch-update)
+  
     local cur_widget
-    for cur_widget in ${${(f)"$(builtin zle -la)"}:#(.*|orig-*|run-help|which-command|beep|set-local-history|yank)}; do
+    for cur_widget in $widgets_to_bind; do
       case $widgets[$cur_widget] in
-
+  
         # Already rebound event: do nothing.
         user:_zsh_highlight_widget_*);;
-
+  
         # The "eval"'s are required to make $cur_widget a closure: the value of the parameter at function
         # definition time is used.
         #
         # We can't use ${0/_zsh_highlight_widget_} because these widgets are always invoked with
         # NO_function_argzero, regardless of the option's setting here.
-
+  
         # User defined widget: override and rebind old one with prefix "orig-".
-        user:*) zle -N orig-$cur_widget ${widgets[$cur_widget]#*:}
+        user:*) zle -N $prefix-$cur_widget ${widgets[$cur_widget]#*:}
-                eval "_zsh_highlight_widget_${(q)cur_widget}() { _zsh_highlight_call_widget orig-${(q)cur_widget} -- \"\$@\" }"
+                eval "_zsh_highlight_widget_${(q)prefix}-${(q)cur_widget}() { _zsh_highlight_call_widget ${(q)prefix}-${(q)cur_widget} -- \"\$@\" }"
-                zle -N $cur_widget _zsh_highlight_widget_$cur_widget;;
+                zle -N $cur_widget _zsh_highlight_widget_$prefix-$cur_widget;;
-
+  
         # Completion widget: override and rebind old one with prefix "orig-".
-        completion:*) zle -C orig-$cur_widget ${${(s.:.)widgets[$cur_widget]}[2,3]} 
+        completion:*) zle -C $prefix-$cur_widget ${${(s.:.)widgets[$cur_widget]}[2,3]} 
-                      eval "_zsh_highlight_widget_${(q)cur_widget}() { _zsh_highlight_call_widget orig-${(q)cur_widget} -- \"\$@\" }"
+                      eval "_zsh_highlight_widget_${(q)prefix}-${(q)cur_widget}() { _zsh_highlight_call_widget ${(q)prefix}-${(q)cur_widget} -- \"\$@\" }"
-                      zle -N $cur_widget _zsh_highlight_widget_$cur_widget;;
+                      zle -N $cur_widget _zsh_highlight_widget_$prefix-$cur_widget;;
-
+  
         # Builtin widget: override and make it call the builtin ".widget".
-        builtin) eval "_zsh_highlight_widget_${(q)cur_widget}() { _zsh_highlight_call_widget .${(q)cur_widget} -- \"\$@\" }"
+        builtin) eval "_zsh_highlight_widget_${(q)prefix}-${(q)cur_widget}() { _zsh_highlight_call_widget .${(q)cur_widget} -- \"\$@\" }"
-                 zle -N $cur_widget _zsh_highlight_widget_$cur_widget;;
+                 zle -N $cur_widget _zsh_highlight_widget_$prefix-$cur_widget;;
-
+  
+        # Incomplete or nonexistent widget: Bind to z-sy-h directly.
+        *) 
+           if [[ $cur_widget == zle-* ]] && [[ -z $widgets[$cur_widget] ]]; then
+             _zsh_highlight_widget_${cur_widget}() { :; _zsh_highlight }
+             zle -N $cur_widget _zsh_highlight_widget_$cur_widget
+           else
         # Default: unhandled case.
-        *) echo "zsh-syntax-highlighting: unhandled ZLE widget '$cur_widget'" >&2 ;;
+             print -r -- >&2 "zsh-syntax-highlighting: unhandled ZLE widget ${(qq)cur_widget}"
+           fi
       esac
     done
   }
@@ -340,7 +387,7 @@ _zsh_highlight_load_highlighters()
 
   # Check the directory exists.
   [[ -d "$1" ]] || {
-    echo "zsh-syntax-highlighting: highlighters directory '$1' not found." >&2
+    print -r -- >&2 "zsh-syntax-highlighting: highlighters directory ${(qq)1} not found."
     return 1
   }
 
@@ -348,13 +395,26 @@ _zsh_highlight_load_highlighters()
   local highlighter highlighter_dir
   for highlighter_dir ($1/*/); do
     highlighter="${highlighter_dir:t}"
-    [[ -f "$highlighter_dir/${highlighter}-highlighter.zsh" ]] && {
+    [[ -f "$highlighter_dir/${highlighter}-highlighter.zsh" ]] &&
       . "$highlighter_dir/${highlighter}-highlighter.zsh"
-      type "_zsh_highlight_${highlighter}_highlighter" &> /dev/null &&
+    if type "_zsh_highlight_highlighter_${highlighter}_paint" &> /dev/null &&
-      type "_zsh_highlight_${highlighter}_highlighter_predicate" &> /dev/null || {
+       type "_zsh_highlight_highlighter_${highlighter}_predicate" &> /dev/null;
-        echo "zsh-syntax-highlighting: '${highlighter}' highlighter should define both required functions '_zsh_highlight_${highlighter}_highlighter' and '_zsh_highlight_${highlighter}_highlighter_predicate' in '${highlighter_dir}/${highlighter}-highlighter.zsh'." >&2
+    then
-      }
+        # New (0.5.0) function names
-    }
+    elif type "_zsh_highlight_${highlighter}_highlighter" &> /dev/null &&
+         type "_zsh_highlight_${highlighter}_highlighter_predicate" &> /dev/null;
+    then
+        # Old (0.4.x) function names
+        if false; then
+            # TODO: only show this warning for plugin authors/maintainers, not for end users
+            print -r -- >&2 "zsh-syntax-highlighting: warning: ${(qq)highlighter} highlighter uses deprecated entry point names; please ask its maintainer to update it: https://github.com/zsh-users/zsh-syntax-highlighting/issues/329"
+        fi
+        # Make it work.
+        eval "_zsh_highlight_highlighter_${(q)highlighter}_paint() { _zsh_highlight_${(q)highlighter}_highlighter \"\$@\" }"
+        eval "_zsh_highlight_highlighter_${(q)highlighter}_predicate() { _zsh_highlight_${(q)highlighter}_highlighter_predicate \"\$@\" }"
+    else
+        print -r -- >&2 "zsh-syntax-highlighting: ${(qq)highlighter} highlighter should define both required functions '_zsh_highlight_highlighter_${highlighter}_paint' and '_zsh_highlight_highlighter_${highlighter}_predicate' in ${(qq):-"$highlighter_dir/${highlighter}-highlighter.zsh"}."
+    fi
   done
 }
 
@@ -365,13 +425,13 @@ _zsh_highlight_load_highlighters()
 
 # Try binding widgets.
 _zsh_highlight_bind_widgets || {
-  echo 'zsh-syntax-highlighting: failed binding ZLE widgets, exiting.' >&2
+  print -r -- >&2 'zsh-syntax-highlighting: failed binding ZLE widgets, exiting.'
   return 1
 }
 
 # Resolve highlighters directory location.
 _zsh_highlight_load_highlighters "${ZSH_HIGHLIGHT_HIGHLIGHTERS_DIR:-${${0:A}:h}/highlighters}" || {
-  echo 'zsh-syntax-highlighting: failed loading highlighters, exiting.' >&2
+  print -r -- >&@ 'zsh-syntax-highlighting: failed loading highlighters, exiting.'
   return 1
 }
 
@@ -383,11 +443,13 @@ _zsh_highlight_preexec_hook()
 }
 autoload -U add-zsh-hook
 add-zsh-hook preexec _zsh_highlight_preexec_hook 2>/dev/null || {
-    echo 'zsh-syntax-highlighting: failed loading add-zsh-hook.' >&2
+    print -r -- >&2 'zsh-syntax-highlighting: failed loading add-zsh-hook.'
   }
 
 # Load zsh/parameter module if available
 zmodload zsh/parameter 2>/dev/null || true
 
+autoload -U is-at-least
+
 # Initialize the array of active highlighters if needed.
 [[ $#ZSH_HIGHLIGHT_HIGHLIGHTERS -eq 0 ]] && ZSH_HIGHLIGHT_HIGHLIGHTERS=(main) || true