Auto merge of #2377 - dhleong:dhleong/#2021-quiet_messages, r=micbou

Implement ycm_quiet_messages options (See #2021)

# PR Prelude

- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
  rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
  actually perform all of these steps.**

# Why this change is necessary and useful

See issue #2021. This is a partial implementation based on the syntastic option referenced, supporting the `!` flag and filters of type `regex` and `level`. Also supports filetype specific filters using `ycm_<ft>_quiet_messages`, but I couldn't think of a great way to fall back to syntastic configs for this one.

In terms of usefulness: I've been playing with C# recently, which has a bunch of style warnings that I don't want to follow, and prefer to only have the gutter showing if there are actually errors, or warnings that I *do* want to follow.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2377)
<!-- Reviewable:end -->

README.md

@@ -1803,6 +1803,39 @@ Default: `1`
 let g:ycm_echo_current_diagnostic = 1
 ```
 
+### The `g:ycm_filter_diagnostics` option
+
+This option controls which diagnostics will be rendered by YCM. This option
+holds a dictionary of key-values, where the keys are Vim's filetype strings
+delimited by commas and values are dictionaries describing the filter.
+
+A filter is a dictionary of key-values, where the keys are the type of filter,
+and the value is a list of arguments to that filter. In the case of just a
+single item in the list, you may omit the brackets and just provide the argument
+directly. If any filter matches a diagnostic, it will be dropped and YCM will 
+not render it.
+
+The following filter types are supported:
+
+- "regex": Accepts a string [regular expression][python-re]. This type matches
+when the regex (treated as case-insensitive) is found in the diagnostic text.
+- "level": Accepts a string level, either "warning" or "error." This type 
+matches when the diagnostic has the same level.
+
+NOTE: The regex syntax is **NOT** Vim's, it's [Python's][python-re].
+
+Default: `{}`
+
+```viml
+let g:ycm_filter_diagnostics = {
+  \ "java": {
+  \      "regex": [ ".*taco.*", ... ],
+  \      "level": "error",
+  \      ...
+  \    }
+  \ }
+```
+
 ### The `g:ycm_always_populate_location_list` option
 
 When this option is set, YCM will populate the location list automatically every

doc/youcompleteme.txt

@@ -89,40 +89,41 @@ Contents ~
   10. The |g:ycm_enable_diagnostic_signs| option
   11. The |g:ycm_enable_diagnostic_highlighting| option
   12. The |g:ycm_echo_current_diagnostic| option
-  13. The |g:ycm_always_populate_location_list| option
+  13. The |g:ycm_filter_diagnostics| option
-  14. The |g:ycm_open_loclist_on_ycm_diags| option
+  14. The |g:ycm_always_populate_location_list| option
-  15. The |g:ycm_allow_changing_updatetime| option
+  15. The |g:ycm_open_loclist_on_ycm_diags| option
-  16. The |g:ycm_complete_in_comments| option
+  16. The |g:ycm_allow_changing_updatetime| option
-  17. The |g:ycm_complete_in_strings| option
+  17. The |g:ycm_complete_in_comments| option
-  18. The |g:ycm_collect_identifiers_from_comments_and_strings| option
+  18. The |g:ycm_complete_in_strings| option
-  19. The |g:ycm_collect_identifiers_from_tags_files| option
+  19. The |g:ycm_collect_identifiers_from_comments_and_strings| option
-  20. The |g:ycm_seed_identifiers_with_syntax| option
+  20. The |g:ycm_collect_identifiers_from_tags_files| option
-  21. The |g:ycm_extra_conf_vim_data| option
+  21. The |g:ycm_seed_identifiers_with_syntax| option
-  22. The |g:ycm_server_python_interpreter| option
+  22. The |g:ycm_extra_conf_vim_data| option
-  23. The |g:ycm_server_keep_logfiles| option
+  23. The |g:ycm_server_python_interpreter| option
-  24. The |g:ycm_server_log_level| option
+  24. The |g:ycm_server_keep_logfiles| option
-  25. The |g:ycm_auto_start_csharp_server| option
+  25. The |g:ycm_server_log_level| option
-  26. The |g:ycm_auto_stop_csharp_server| option
+  26. The |g:ycm_auto_start_csharp_server| option
-  27. The |g:ycm_csharp_server_port| option
+  27. The |g:ycm_auto_stop_csharp_server| option
-  28. The |g:ycm_csharp_insert_namespace_expr| option
+  28. The |g:ycm_csharp_server_port| option
-  29. The |g:ycm_add_preview_to_completeopt| option
+  29. The |g:ycm_csharp_insert_namespace_expr| option
-  30. The |g:ycm_autoclose_preview_window_after_completion| option
+  30. The |g:ycm_add_preview_to_completeopt| option
-  31. The |g:ycm_autoclose_preview_window_after_insertion| option
+  31. The |g:ycm_autoclose_preview_window_after_completion| option
-  32. The |g:ycm_max_diagnostics_to_display| option
+  32. The |g:ycm_autoclose_preview_window_after_insertion| option
-  33. The |g:ycm_key_list_select_completion| option
+  33. The |g:ycm_max_diagnostics_to_display| option
-  34. The |g:ycm_key_list_previous_completion| option
+  34. The |g:ycm_key_list_select_completion| option
-  35. The |g:ycm_key_invoke_completion| option
+  35. The |g:ycm_key_list_previous_completion| option
-  36. The |g:ycm_key_detailed_diagnostics| option
+  36. The |g:ycm_key_invoke_completion| option
-  37. The |g:ycm_global_ycm_extra_conf| option
+  37. The |g:ycm_key_detailed_diagnostics| option
-  38. The |g:ycm_confirm_extra_conf| option
+  38. The |g:ycm_global_ycm_extra_conf| option
-  39. The |g:ycm_extra_conf_globlist| option
+  39. The |g:ycm_confirm_extra_conf| option
-  40. The |g:ycm_filepath_completion_use_working_dir| option
+  40. The |g:ycm_extra_conf_globlist| option
-  41. The |g:ycm_semantic_triggers| option
+  41. The |g:ycm_filepath_completion_use_working_dir| option
-  42. The |g:ycm_cache_omnifunc| option
+  42. The |g:ycm_semantic_triggers| option
-  43. The |g:ycm_use_ultisnips_completer| option
+  43. The |g:ycm_cache_omnifunc| option
-  44. The |g:ycm_goto_buffer_command| option
+  44. The |g:ycm_use_ultisnips_completer| option
-  45. The |g:ycm_disable_for_files_larger_than_kb| option
+  45. The |g:ycm_goto_buffer_command| option
-  46. The |g:ycm_python_binary_path| option
+  46. The |g:ycm_disable_for_files_larger_than_kb| option
+  47. The |g:ycm_python_binary_path| option
  11. FAQ                                                    |youcompleteme-faq|
   1. I used to be able to 'import vim' in '.ycm_extra_conf.py', but now can't |import-vim|
   2. On very rare occasions Vim crashes when I tab through the completion menu |youcompleteme-on-very-rare-occasions-vim-crashes-when-i-tab-through-completion-menu|
@@ -2080,6 +2081,39 @@ Default: '1'
   let g:ycm_echo_current_diagnostic = 1
 <
 -------------------------------------------------------------------------------
+The *g:ycm_filter_diagnostics* option
+
+This option controls which diagnostics will be rendered by YCM. This option
+holds a dictionary of key-values, where the keys are Vim's filetype strings
+delimited by commas and values are dictionaries describing the filter.
+
+A filter is a dictionary of key-values, where the keys are the type of filter,
+and the value is a list of arguments to that filter. In the case of just a
+single item in the list, you may omit the brackets and just provide the
+argument directly. If any filter matches a diagnostic, it will be dropped and
+YCM will not render it.
+
+The following filter types are supported:
+
+- "regex": Accepts a string regular expression [54]. This type matches when
+  the regex (treated as case-insensitive) is found in the diagnostic text.
+
+- "level": Accepts a string level, either "warning" or "error." This type
+  matches when the diagnostic has the same level.
+
+NOTE: The regex syntax is **NOT** Vim's, it's Python's [54].
+
+Default: '{}'
+>
+  let g:ycm_filter_diagnostics = {
+    \ "java": {
+    \      "regex": [ ".*taco.*", ... ],
+    \      "level": "error",
+    \      ...
+    \    }
+    \ }
+<
+-------------------------------------------------------------------------------
 The *g:ycm_always_populate_location_list* option
 
 When this option is set, YCM will populate the location list automatically
@@ -2174,7 +2208,7 @@ from the 'tagfiles()' Vim function which examines the 'tags' Vim option. See
 
 YCM will re-index your tags files if it detects that they have been modified.
 
-The only supported tag format is the Exuberant Ctags format [54]. The format
+The only supported tag format is the Exuberant Ctags format [55]. The format
 from "plain" ctags is NOT supported. Ctags needs to be called with the '--
 fields=+l' option (that's a lowercase 'L', not a one) because YCM needs the
 'language:<lang>' field in the tags output.
@@ -2541,7 +2575,7 @@ It's also possible to use a regular expression as a trigger. You have to prefix
 your trigger with 're!' to signify it's a regex trigger. For instance,
 're!\w+\.' would only trigger after the '\w+\.' regex matches.
 
-NOTE: The regex syntax is **NOT** Vim's, it's Python's [55].
+NOTE: The regex syntax is **NOT** Vim's, it's Python's [54].
 
 Default: '[see next line]'
 >
@@ -2869,7 +2903,7 @@ YCM does not read identifiers from my tags files ~
 First, put 'let g:ycm_collect_identifiers_from_tags_files = 1' in your vimrc.
 
 Make sure you are using Exuberant Ctags [57] to produce your tags files since
-the only supported tag format is the Exuberant Ctags format [54]. The format
+the only supported tag format is the Exuberant Ctags format [55]. The format
 from "plain" ctags is NOT supported. The output of 'ctags --version' should
 list "Exuberant Ctags".
 
@@ -3212,8 +3246,8 @@ References ~
 [51] https://github.com/Valloric/ycmd/blob/master/ycmd/completers/completer.py
 [52] https://github.com/Valloric/ListToggle
 [53] https://github.com/itchyny/lightline.vim
-[54] http://ctags.sourceforge.net/FORMAT
+[54] https://docs.python.org/2/library/re.html#regular-expression-syntax
-[55] https://docs.python.org/2/library/re.html#regular-expression-syntax
+[55] http://ctags.sourceforge.net/FORMAT
 [56] https://github.com/Valloric/YouCompleteMe/issues/18
 [57] http://ctags.sourceforge.net/
 [58] https://github.com/Raimondi/delimitMate

python/ycm/diagnostic_filter.py

@@ -0,0 +1,151 @@
+# Copyright (C) 2016  YouCompleteMe contributors
+#
+# This file is part of YouCompleteMe.
+#
+# YouCompleteMe is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# YouCompleteMe is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with YouCompleteMe.  If not, see <http://www.gnu.org/licenses/>.
+
+from __future__ import unicode_literals
+from __future__ import print_function
+from __future__ import division
+from __future__ import absolute_import
+from future import standard_library
+standard_library.install_aliases()
+from builtins import *  # noqa
+
+from future.utils import iterkeys, iteritems
+from ycm import vimsupport
+import re
+
+
+class DiagnosticFilter( object ):
+  def __init__( self, config_or_filters ):
+    if isinstance( config_or_filters, list ):
+      self._filters = config_or_filters
+
+    else:
+      self._filters = _CompileFilters( config_or_filters )
+
+
+  def IsAllowed( self, diagnostic ):
+    # NOTE: a diagnostic IsAllowed() ONLY if NO filters match it
+    for filterMatches in self._filters:
+      if filterMatches( diagnostic ):
+        return False
+
+    return True
+
+
+  def SubsetForTypes( self, filetypes ):
+    """Return a sub-filter limited to the given filetypes"""
+    # NOTE: actually, this class is already filtered
+    return self
+
+
+  @staticmethod
+  def CreateFromOptions( user_options ):
+    all_filters = dict( user_options.get( 'filter_diagnostics', {} ) )
+    compiled_by_type = {}
+    for type_spec, filter_value in iteritems( dict( all_filters ) ):
+      filetypes = [ type_spec ]
+      if type_spec.find( ',' ) != -1:
+        filetypes = type_spec.split( ',' )
+      for filetype in filetypes:
+        compiled_by_type[ filetype ] = _CompileFilters( filter_value )
+
+    return _MasterDiagnosticFilter( compiled_by_type )
+
+
+class _MasterDiagnosticFilter( object ):
+
+  def __init__( self, all_filters ):
+    self._all_filters = all_filters
+    self._cache = {}
+
+
+  def IsAllowed( self, diagnostic ):
+    # NOTE: in this class's implementation, we ask vimsupport for
+    #  the current filetypes and delegate automatically; it is probably,
+    #  more efficient, however, to call SubsetForTypes() and reuse
+    #  the returned DiagnosticFilter if it will be checked repeatedly.
+    filetypes = vimsupport.CurrentFiletypes()
+    return self.SubsetForTypes( filetypes ).IsAllowed( diagnostic )
+
+
+  def SubsetForTypes( self, filetypes ):
+    # check cache
+    cache_key = ','.join( filetypes )
+    cached = self._cache.get( cache_key )
+    if cached is not None:
+      return cached
+
+    # build a new DiagnosticFilter merging all filters
+    #  for the provided filetypes
+    spec = []
+    for filetype in filetypes:
+      type_specific = self._all_filters.get( filetype, [] )
+      spec.extend( type_specific )
+
+    new_filter = DiagnosticFilter( spec )
+    self._cache[ cache_key ] = new_filter
+    return new_filter
+
+
+def _ListOf( config_entry ):
+  if isinstance( config_entry, list ):
+    return config_entry
+
+  if config_entry is None:
+    return []
+
+  return [ config_entry ]
+
+
+def CompileRegex( raw_regex ):
+  pattern = re.compile( raw_regex, re.IGNORECASE )
+
+  def FilterRegex( diagnostic ):
+    return pattern.search( diagnostic[ 'text' ] ) is not None
+
+  return FilterRegex
+
+
+def CompileLevel( level ):
+  # valid kinds are WARNING and ERROR;
+  #  expected input levels are `warning` and `error`
+  # NOTE: we don't validate the input...
+  expected_kind = level.upper()
+
+  def FilterLevel( diagnostic ):
+    return diagnostic[ 'kind' ] == expected_kind
+
+  return FilterLevel
+
+
+FILTER_COMPILERS = { 'regex' : CompileRegex,
+                     'level' : CompileLevel }
+
+
+def _CompileFilters( config ):
+  """Given a filter config dictionary, return a list of compiled filters"""
+  filters = []
+
+  for filter_type in iterkeys( config ):
+    compiler = FILTER_COMPILERS.get( filter_type )
+
+    if compiler is not None:
+      for filter_config in _ListOf( config[ filter_type ] ):
+        compiledFilter = compiler( filter_config )
+        filters.append( compiledFilter )
+
+  return filters

python/ycm/diagnostic_interface.py

@@ -26,12 +26,14 @@ from builtins import *  # noqa
 from future.utils import itervalues, iteritems
 from collections import defaultdict, namedtuple
 from ycm import vimsupport
+from ycm.diagnostic_filter import DiagnosticFilter, CompileLevel
 import vim
 
 
 class DiagnosticInterface( object ):
   def __init__( self, user_options ):
     self._user_options = user_options
+    self._diag_filter = DiagnosticFilter.CreateFromOptions( user_options )
     # Line and column numbers are 1-based
     self._buffer_number_to_line_to_diags = defaultdict(
       lambda: defaultdict( list ) )
@@ -61,11 +63,13 @@ class DiagnosticInterface( object ):
 
   def PopulateLocationList( self, diags ):
     vimsupport.SetLocationList(
-      vimsupport.ConvertDiagnosticsToQfList( diags ) )
+      vimsupport.ConvertDiagnosticsToQfList(
+          self._ApplyDiagnosticFilter( diags ) ) )
 
 
   def UpdateWithNewDiagnostics( self, diags ):
-    normalized_diags = [ _NormalizeDiagnostic( x ) for x in diags ]
+    normalized_diags = [ _NormalizeDiagnostic( x ) for x in
+            self._ApplyDiagnosticFilter( diags ) ]
     self._buffer_number_to_line_to_diags = _ConvertDiagListToDict(
         normalized_diags )
 
@@ -81,6 +85,20 @@ class DiagnosticInterface( object ):
     if self._user_options[ 'always_populate_location_list' ]:
       self.PopulateLocationList( normalized_diags )
 
+
+  def _ApplyDiagnosticFilter( self, diags, extra_predicate = None ):
+    filetypes = vimsupport.CurrentFiletypes()
+    diag_filter = self._diag_filter.SubsetForTypes( filetypes )
+    predicate = diag_filter.IsAllowed
+    if extra_predicate is not None:
+      def Filter( diag ):
+        return extra_predicate( diag ) and diag_filter.IsAllowed( diag )
+
+      predicate = Filter
+
+    return filter( predicate, diags )
+
+
   def _EchoDiagnosticForLine( self, line_num ):
     buffer_num = vim.current.buffer.number
     diags = self._buffer_number_to_line_to_diags[ buffer_num ][ line_num ]
@@ -105,7 +123,8 @@ class DiagnosticInterface( object ):
       vim.current.buffer.number ]
 
     for diags in itervalues( line_to_diags ):
-      matched_diags.extend( list( filter( predicate, diags ) ) )
+      matched_diags.extend( list(
+        self._ApplyDiagnosticFilter( diags, predicate ) ) )
     return matched_diags
 
 
@@ -236,12 +255,8 @@ def _ConvertDiagListToDict( diag_list ):
   return buffer_to_line_to_diags
 
 
-def _DiagnosticIsError( diag ):
+_DiagnosticIsError = CompileLevel( 'error' )
-  return diag[ 'kind' ] == 'ERROR'
+_DiagnosticIsWarning = CompileLevel( 'warning' )
-
-
-def _DiagnosticIsWarning( diag ):
-  return diag[ 'kind' ] == 'WARNING'
 
 
 def _NormalizeDiagnostic( diag ):

python/ycm/tests/diagnostic_filter_test.py

@@ -0,0 +1,143 @@
+# Copyright (C) 2016  YouCompleteMe contributors
+#
+# This file is part of YouCompleteMe.
+#
+# YouCompleteMe is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# YouCompleteMe is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with YouCompleteMe.  If not, see <http://www.gnu.org/licenses/>.
+
+from __future__ import unicode_literals
+from __future__ import print_function
+from __future__ import division
+from __future__ import absolute_import
+from future import standard_library
+standard_library.install_aliases()
+from builtins import *  # noqa
+
+from ycm.tests.test_utils import MockVimModule
+MockVimModule()
+
+from hamcrest import assert_that, equal_to
+from ycm.diagnostic_filter import DiagnosticFilter
+
+
+def _assert_accept_equals( filter, text_or_obj, expected ):
+  if not isinstance( text_or_obj, dict ):
+    text_or_obj = { 'text': text_or_obj }
+
+  assert_that( filter.IsAllowed( text_or_obj ), equal_to( expected ) )
+
+
+def _assert_accepts( filter, text ):
+  _assert_accept_equals( filter, text, True )
+
+
+def _assert_rejects( filter, text ):
+  _assert_accept_equals( filter, text, False )
+
+
+def _JavaFilter( config ):
+  return { 'filter_diagnostics' : { 'java': config } }
+
+
+def _CreateFilterForTypes( opts, types ):
+  return DiagnosticFilter.CreateFromOptions( opts ).SubsetForTypes( types )
+
+
+def RegexFilter_test():
+  opts = _JavaFilter( { 'regex' : 'taco' } )
+  f = _CreateFilterForTypes( opts, [ 'java' ] )
+
+  _assert_rejects( f, 'This is a Taco' )
+  _assert_accepts( f, 'This is a Burrito' )
+
+
+def RegexSingleList_test():
+  opts = _JavaFilter( { 'regex' : [ 'taco' ] }  )
+  f = _CreateFilterForTypes( opts, [ 'java' ] )
+
+  _assert_rejects( f, 'This is a Taco' )
+  _assert_accepts( f, 'This is a Burrito' )
+
+
+def RegexMultiList_test():
+  opts = _JavaFilter( { 'regex' : [ 'taco', 'burrito' ] } )
+  f = _CreateFilterForTypes( opts, [ 'java' ] )
+
+  _assert_rejects( f, 'This is a Taco' )
+  _assert_rejects( f, 'This is a Burrito' )
+
+
+def RegexNotFiltered_test():
+  opts = _JavaFilter( { 'regex' : 'taco' } )
+  f = _CreateFilterForTypes( opts, [ 'cs' ] )
+
+  _assert_accepts( f, 'This is a Taco' )
+  _assert_accepts( f, 'This is a Burrito' )
+
+
+def LevelWarnings_test():
+  opts = _JavaFilter( { 'level' : 'warning' } )
+  f = _CreateFilterForTypes( opts, [ 'java' ] )
+
+  _assert_rejects( f, { 'text' : 'This is an unimportant taco',
+                        'kind' : 'WARNING' } )
+  _assert_accepts( f, { 'text' : 'This taco will be shown',
+                        'kind' : 'ERROR' } )
+
+
+def LevelErrors_test():
+  opts = _JavaFilter( { 'level' : 'error' } )
+  f = _CreateFilterForTypes( opts, [ 'java' ] )
+
+  _assert_accepts( f, { 'text' : 'This is an IMPORTANT taco',
+                        'kind' : 'WARNING' } )
+  _assert_rejects( f, { 'text' : 'This taco will NOT be shown',
+                        'kind' : 'ERROR' } )
+
+
+def MultipleFilterTypesTypeTest_test():
+
+  opts = _JavaFilter( { 'regex' : '.*taco.*',
+                        'level' : 'warning' } )
+  f = _CreateFilterForTypes( opts, [ 'java' ] )
+
+  _assert_rejects( f, { 'text' : 'This is an unimportant taco',
+                        'kind' : 'WARNING' } )
+  _assert_rejects( f, { 'text' : 'This taco will NOT be shown',
+                        'kind' : 'ERROR' } )
+  _assert_accepts( f, { 'text' : 'This burrito WILL be shown',
+                        'kind' : 'ERROR' } )
+
+
+def MergeMultipleFiletypes_test():
+
+  opts = { 'filter_diagnostics' : {
+    'java' : { 'regex' : '.*taco.*' },
+    'xml'  : { 'regex' : '.*burrito.*' } } }
+
+  f = _CreateFilterForTypes( opts, [ 'java', 'xml' ] )
+
+  _assert_rejects( f, 'This is a Taco' )
+  _assert_rejects( f, 'This is a Burrito' )
+  _assert_accepts( f, 'This is some Nachos' )
+
+
+def CommaSeparatedFiletypes_test():
+
+  opts = { 'filter_diagnostics' : {
+    'java,c,cs' : { 'regex' : '.*taco.*' } } }
+
+  f = _CreateFilterForTypes( opts, [ 'cs' ] )
+
+  _assert_rejects( f, 'This is a Taco' )
+  _assert_accepts( f, 'This is a Burrito' )