A "clearsnippets" feature

=========================

It's difficult for the user to control which of the default
bundled snippets are active in his environment.  The
'runtimepath' variable must be set to the root of the ultisnips
installation, which brings in all of the bundled snippets.
Though the user may individually override the definition of the
bundled snippets using the "!" flag, the method has a couple of
problems:

- There's no way to remove a snippet, only to override it (and
  each snippet must be overridden individually).
- The "!" flag currently doesn't remove the overridden snippets
  from the "list snippets" command.

It might be considered a feature that "!" doesn't actually
remove the snippets from the "list snippets" command, though
perhaps that's an unintended effect.  In any case, it would be
more convenient to allow the user to selectively remove the
bundled snippets from his environment.

A patch is provided in the following branch to address these problems:
http://code.launchpad.net/~drmikehenry/ultisnips/clearsnippets

The branch's primary purpose is the addition of a
"clearsnippets" command that may be placed in a user's
~/.vim/UltiSnips/ft.snippets file.  The user may clear all
lower-priority snippet for that file type with the line:

  clearsnippets

Alternatively, he may clear individual snippets by listing their
triggers:

  clearsnippets trigger1 trigger2

A few changes were made to the testing system as part of the
incorporation of this new feature.  These changes include:

- The "extends" directive is now supported on multiple lines
  throughout file.

- A completely empty .snippets file is now possible.

- The test.py scripts now handles most of the vim setup,
  simplifying the running of the tests.  The invocation of Vim
  now reduces to:

    vim -u NONE

  Instructions for running the tests are included at top of
  test.py, where they should be more visible to interested
  users; UltiSnips.vim now just points to test.py's
  instructions.

- A new function vim_quote() encodes an arbitrary string into a
  singly-quoted Vim string, with embedded quotes escaped.
- SnippetsFileParser() now allows file_data to be passed
  directly for unit testing, avoiding the need to create files
  in the filesystem for test purposes.
- A new _error() function reports errors to the user.  At
  runtime, this function uses :echo_err in general, but also can
  append error text to current buffer to check for expected
  errors during unit tests.
- Added error checks to snippets file parsing, along with unit
  tests for the parsing.
- Increased retries from 2 to 4 (on my system, occasionally the
  timing still causes tests to fail).

doc/UltiSnips.txt

@@ -20,6 +20,7 @@ UltiSnips                                      *snippet* *snippets* *UltiSnips*
    4.7 Transformations                          |UltiSnips-transformations|
       4.7.1 Replacement String                  |UltiSnips-replacement-string|
       4.7.2 Demos                               |UltiSnips-demos|
+   4.8 Clearing snippets                        |UltiSnips-clearing-snippets|
 5. Roadmap                                      |UltiSnips-roadmap|      
 6. Helping out                                  |UltiSnips-helping|      
 7. Contact                                      |UltiSnips-contact|      
@@ -150,12 +151,13 @@ CUDA files, i keep the file type set to ":set ft=cpp.c" or ":set
 ft=cuda.cpp.c". This activates all snippets for each file type in the order
 specified.
 
-As an alternative, the first line of each snippet file can look like this: >
+As an alternative, a snippet file may contain a line that looks like this: >
    extends ft1, ft2, ft3
 For example, the first line in cpp.snippets looks like this: >
    extends c
 This means, first check all triggers for c, then add the triggers from this
 file. This is a convenient way to add more special cases to more general ones.
+Multiple "extends" lines are permitted throughout the snippets file.
 
 The snippets file format is simple. A line starting with # is a comment, each
 snippet starts with a line in the form of: >
@@ -467,6 +469,27 @@ printf("A is: %s\n", A); // End of line
 There are many more examples of what can be done with transformations in the
 bundled snippets.
 
+ 4.8 Clearing snippets                          *UltiSnips-clearing-snippets*
+
+To remove snippets for the current file type, use the "clearsnippets"
+directive:
+------------------- SNIP -------------------
+clearsnippets
+------------------- SNAP -------------------
+
+Without arguments, "clearsnippets" removes all snippets defined so far for the
+current file type.  UltiSnips travels in reverse along the 'runtimepath', so
+"clearsnippets" removes snippet definitions appearing later in the
+'runtimepath' than the ".snippets" file in which it's encountered.
+
+Instead of clearing all snippets for the current file type, one or more
+individual snippets may be cleared by specifying a space-separated list of
+their triggers, e.g.:
+
+------------------- SNIP -------------------
+clearsnippets trigger1 trigger2
+------------------- SNAP -------------------
+
 =============================================================================
 5. ROADMAP                                                *UltiSnips-roadmap*
 

plugin/UltiSnips.vim

@@ -4,10 +4,8 @@
 " Last Modified: July 21, 2009
 "
 " Testing Info: {{{
-"   Running vim + ultisnips with the absolute bar minimum settings inside a   screen session:
+"   See directions at the top of the test.py script located one 
-"     $ screen -S vim
+"   directory above this file.
-"     $ vim -u NONE -U NONE -c ':set nocompatible' -c ':set runtimepath+=.'
-"     $ ./test.py  # launch the testsuite
 " }}}
 
 if exists('did_UltiSnips_vim') || &cp || version < 700 || !has("python")

plugin/UltiSnips/__init__.py

@@ -11,6 +11,10 @@ from UltiSnips.Geometry import Position
 from UltiSnips.TextObjects import *
 from UltiSnips.Buffer import VimBuffer
 
+def vim_quote(s):
+    """Quote string s as Vim literal string."""
+    return "'" + s.replace("'", "''") + "'"
+
 class _SnippetDictionary(object):
     def __init__(self, *args, **kwargs):
         self._snippets = []
@@ -26,6 +30,17 @@ class _SnippetDictionary(object):
         else:
             return [ s for s in self._snippets if s.could_match(trigger) ]
 
+    def clear_snippets(self, triggers=[]):
+        """Remove all snippets that match each trigger in triggers.
+            When triggers is empty, removes all snippets.
+        """
+        if triggers:
+            for t in triggers:
+                for s in self.get_matching_snippets(t, potentially=False):
+                    self._snippets.remove(s)
+        else:
+            self._snippets = []
+
     def extends():
         def fget(self):
             return self._extends
@@ -35,15 +50,45 @@ class _SnippetDictionary(object):
     extends = property(**extends())
 
 class _SnippetsFileParser(object):
-    def __init__(self, ft, fn, snip_manager):
+    def __init__(self, ft, fn, snip_manager, file_data=None):
         self._sm = snip_manager
         self._ft = ft
+        self._fn = fn
+        if file_data is None:
             self._lines = open(fn).readlines()
+        else:
+            self._lines = file_data.splitlines(True)
 
         self._idx = 0
 
-    def _parse_snippet(self):
+    def _error(self, msg):
+        fn = vim.eval("""fnamemodify(%s, ":~:.")""" % vim_quote(self._fn))
+        self._sm._error("%s in %s(%d)" % (msg, fn, self._idx + 1))
+
+    def _line(self):
+        if self._idx < len(self._lines):
             line = self._lines[self._idx]
+        else:
+            line = ""
+        return line
+
+    def _line_head_tail(self):
+        parts = re.split(r"\s+", self._line().rstrip(), maxsplit=1)
+        parts.append('')
+        return parts[:2]
+
+    def _line_head(self):
+        return self._line_head_tail()[0]
+
+    def _line_tail(self):
+        return self._line_head_tail()[1]
+
+    def _goto_next_line(self):
+        self._idx += 1
+        return self._line()
+
+    def _parse_snippet(self):
+        line = self._line()
 
         cdescr = ""
         coptions = ""
@@ -55,31 +100,35 @@ class _SnippetsFileParser(object):
             cdescr = line[left+1:right]
             coptions = line[right:].strip()
 
-        self._idx += 1
         cv = ""
-        while 1:
+        while self._goto_next_line():
-            line = self._lines[self._idx]
+            line = self._line()
-            if line.startswith("endsnippet"):
+            if line.rstrip() == "endsnippet":
                 cv = cv[:-1] # Chop the last newline
                 self._sm.add_snippet(cs, cv, cdescr, coptions, self._ft)
                 break
-
             cv += line
-            self._idx += 1
+        else:
+            self._error("Missing 'endsnippet' for %r" % cs)
 
     def parse(self):
-        if self._lines[0].startswith("extends"):
+        while self._line():
+            head, tail = self._line_head_tail()
+            print "head, tail=%r, %r" % (head, tail)
+            if head == "extends":
+                if tail:
                     self._sm.add_extending_info(self._ft,
-                [ p.strip() for p in self._lines[0][7:].split(',') ])
+                        [ p.strip() for p in tail.split(',') ])
-
+                else:
-        while self._idx < len(self._lines):
+                    self._error("'extends' without file types")
-            line = self._lines[self._idx]
+            elif head == "snippet":
-
-            if not line.startswith('#'):
-                if line.startswith("snippet"):
                 self._parse_snippet()
-
+            elif head == "clearsnippets":
-            self._idx += 1
+                self._sm.clear_snippets(tail.split(), self._ft)
+            elif head and not head.startswith('#'):
+                self._error("Invalid line %r" % self._line().rstrip())
+                break
+            self._goto_next_line()
 
 
 
@@ -287,7 +336,8 @@ class SnippetManager(object):
 
         self.reset()
 
-    def reset(self):
+    def reset(self, test_error=False):
+        self._test_error = test_error
         self._snippets = {}
         self._csnippets = []
         self._reinit()
@@ -399,6 +449,10 @@ class SnippetManager(object):
             Snippet(trigger, value, descr, options)
         )
 
+    def clear_snippets(self, triggers = [], ft = "all"):
+        if ft in self._snippets:
+            self._snippets[ft].clear_snippets(triggers)
+
     def add_extending_info(self, ft, parents):
         if ft not in self._snippets:
             self._snippets[ft] = _SnippetDictionary()
@@ -489,6 +543,21 @@ class SnippetManager(object):
     ###################################
     # Private/Protect Functions Below #
     ###################################
+    def _error(self, msg):
+        msg = vim_quote("UltiSnips: " + msg)
+        if self._test_error:
+            msg = msg.replace('"', r'\"')
+            msg = msg.replace('|', r'\|')
+            vim.command("let saved_pos=getpos('.')")
+            vim.command("$:put =%s" % msg)
+            vim.command("call setpos('.', saved_pos)")
+        elif False:
+            vim.command("echohl WarningMsg")
+            vim.command("echomsg %s" % msg)
+            vim.command("echohl None")
+        else:
+            vim.command("echoerr %s" % msg)
+
     def _reinit(self):
         self._ctab = None
         self._span_selected = None
@@ -705,6 +774,9 @@ class SnippetManager(object):
         return self._csnippets[-1]
     _cs = property(_cs)
 
+    def _parse_snippets(self, ft, fn, file_data=None):
+        _SnippetsFileParser(ft, fn, self, file_data).parse()
+
     # Loading
     def _load_snippets_for(self, ft):
         self._snippets[ft] = _SnippetDictionary()
@@ -713,7 +785,7 @@ class SnippetManager(object):
                     "*%s.snippets" % ft
 
             for fn in glob.glob(pattern):
-                _SnippetsFileParser(ft, fn, self).parse()
+                self._parse_snippets(ft, fn)
 
         # Now load for the parents
         for p in self._snippets[ft].extends:

test.py

@@ -1,11 +1,34 @@
 #!/usr/bin/env python
 # encoding: utf-8
 #
+# To execute this test requires two terminals, one for running Vim and one
+# for executing the test script.  Both terminals should have their current
+# working directories set to this directory (the one containing this test.py
+# script).
+#
+# In one terminal, launch a GNU ``screen`` session named ``vim``:
+#   $ screen -S vim
+#
+# Within this new session, launch Vim with the absolute bare minimum settings
+# to ensure a consistent test environment:
+#  $ vim -u NONE
+#
+# The '-u NONE' disables normal .vimrc and .gvimrc processing (note
+# that '-u NONE' implies '-U NONE').
+#
+# All other settings are configured by the test script.
+#
+# Now, from another terminal, launch the testsuite:
+#    $ ./test.py
+# 
+# The testsuite will use ``screen`` to inject commands into the Vim under test,
+# and will compare the resulting output to expected results.
 
 import os
 import tempfile
 import unittest
 import time
+from textwrap import dedent
 
 # Some constants for better reading
 BS = '\x7f'
@@ -26,6 +49,7 @@ COMPL_KW = chr(24)+chr(14)
 COMPL_ACCEPT = chr(25)
 
 def send(s,session):
+    s = s.replace("'", r"'\''")
     os.system("screen -x %s -X stuff '%s'" % (session, s))
 
 def type(str, session, sleeptime):
@@ -39,8 +63,10 @@ def type(str, session, sleeptime):
 
 class _VimTest(unittest.TestCase):
     snippets = ("dummy", "donotdefine")
+    snippets_test_file = ("", "", "")  # file type, file name, file content
     text_before = " --- some text before --- "
     text_after =  " --- some text after --- "
+    expected_error = ""
     wanted = ""
     keys = ""
     sleeptime = 0.00
@@ -48,13 +74,18 @@ class _VimTest(unittest.TestCase):
     def send(self,s):
         send(s, self.session)
 
+    def send_py(self,s):
+        self.send(":py << EOF\n%s\nEOF\n" % s)
+
     def type(self,s):
         type(s, self.session, self.sleeptime)
 
     def check_output(self):
         wanted = self.text_before + '\n\n' + self.wanted + \
                 '\n\n' + self.text_after
-        for i in range(2):
+        if self.expected_error:
+            wanted = wanted + "\n" + self.expected_error
+        for i in range(4):
             if self.output != wanted:
                 self.setUp()
         self.assertEqual(self.output, wanted)
@@ -70,7 +101,10 @@ class _VimTest(unittest.TestCase):
     def setUp(self):
         self.send(ESC)
 
-        self.send(":py UltiSnips_Manager.reset()\n")
+        self.send(":py UltiSnips_Manager.reset(test_error=True)\n")
+
+        # Clear the buffer
+        self.send("bggVGd")
 
         if not isinstance(self.snippets[0],tuple):
             self.snippets = ( self.snippets, )
@@ -84,16 +118,13 @@ class _VimTest(unittest.TestCase):
             if len(s) > 3:
                 options = s[3]
 
-            self.send(''':py << EOF
+            self.send_py("UltiSnips_Manager.add_snippet(%r, %r, %r, %r)" %
-UltiSnips_Manager.add_snippet("%s","""%s""", "%s", "%s")
+                (sv, content, descr, options))
-EOF
-''' % (sv,content.encode("string-escape"), descr.encode("string-escape"),
-      options
-      )
-      )
 
-        # Clear the buffer
+        ft, fn, file_data = self.snippets_test_file
-        self.send("bggVGd")
+        if ft:
+            self.send_py("UltiSnips_Manager._parse_snippets(%r, %r, %r)" %
+                (ft, fn, dedent(file_data + '\n')))
 
         if not self.interrupt:
             # Enter insert mode
@@ -1208,6 +1239,92 @@ class ListAllAvailable_testtypedSecondOpt_ExceptCorrectResult(_ListAllSnippets):
     keys = "hallo test" + LS + "2\n"
     wanted = "hallo TEST ONE"
 
+#########################
+# SNIPPETS FILE PARSING #
+#########################
+
+class ParseSnippets_SimpleSnippet(_VimTest):
+    snippets_test_file = ("all", "test_file", r"""
+        snippet testsnip "Test Snippet" b!
+        This is a test snippet!
+        endsnippet
+        """)
+    keys = "testsnip" + EX
+    wanted = "This is a test snippet!"
+
+class ParseSnippets_MissingEndSnippet(_VimTest):
+    snippets_test_file = ("all", "test_file", r"""
+        snippet testsnip "Test Snippet" b!
+        This is a test snippet!
+        """)
+    keys = "testsnip" + EX
+    wanted = "testsnip" + EX
+    expected_error = dedent("""
+        UltiSnips: Missing 'endsnippet' for 'testsnip' in test_file(5)
+        """).strip()
+
+class ParseSnippets_UnknownDirective(_VimTest):
+    snippets_test_file = ("all", "test_file", r"""
+        unknown directive
+        """)
+    keys = "testsnip" + EX
+    wanted = "testsnip" + EX
+    expected_error = dedent("""
+        UltiSnips: Invalid line 'unknown directive' in test_file(2)
+        """).strip()
+
+class ParseSnippets_ExtendsWithoutFiletype(_VimTest):
+    snippets_test_file = ("all", "test_file", r"""
+        extends
+        """)
+    keys = "testsnip" + EX
+    wanted = "testsnip" + EX
+    expected_error = dedent("""
+        UltiSnips: 'extends' without file types in test_file(2)
+        """).strip()
+
+class ParseSnippets_ClearAll(_VimTest):
+    snippets_test_file = ("all", "test_file", r"""
+        snippet testsnip "Test snippet"
+        This is a test.
+        endsnippet
+
+        clearsnippets
+        """)
+    keys = "testsnip" + EX
+    wanted = "testsnip" + EX
+
+class ParseSnippets_ClearOne(_VimTest):
+    snippets_test_file = ("all", "test_file", r"""
+        snippet testsnip "Test snippet"
+        This is a test.
+        endsnippet
+
+        snippet toclear "Snippet to clear"
+        Do not expand.
+        endsnippet
+
+        clearsnippets toclear
+        """)
+    keys = "toclear" + EX + "\n" + "testsnip" + EX
+    wanted = "toclear" + EX + "\n" + "This is a test."
+
+class ParseSnippets_ClearTwo(_VimTest):
+    snippets_test_file = ("all", "test_file", r"""
+        snippet testsnip "Test snippet"
+        This is a test.
+        endsnippet
+
+        snippet toclear "Snippet to clear"
+        Do not expand.
+        endsnippet
+
+        clearsnippets testsnip toclear
+        """)
+    keys = "toclear" + EX + "\n" + "testsnip" + EX
+    wanted = "toclear" + EX + "\n" + "testsnip" + EX
+
+
 ###########################################################################
 #                               END OF TEST                               #
 ###########################################################################
@@ -1240,6 +1357,13 @@ if __name__ == '__main__':
     test_loader = unittest.TestLoader()
     all_test_suites = test_loader.loadTestsFromModule(__import__("test"))
 
+    # Ensure we are not running in VI-compatible mode.
+    send(""":set nocompatible\n""", options.session)
+
+    # Ensure runtimepath includes only Vim's own runtime files
+    # and those of the UltiSnips directory under test ('.').
+    send(""":set runtimepath=$VIMRUNTIME,.\n""", options.session)
+
     # Set the options
     send(""":let g:UltiSnipsExpandTrigger="<tab>"\n""", options.session)
     send(""":let g:UltiSnipsJumpForwardTrigger="?"\n""", options.session)