[READY] Rewrite completion system
There is a number of issues with the current completion system:
- UI is blocked until the completions are returned by the server, the request timed out, or a key is pressed by the user. This leads to a rough experience when completions take too much time: cursor disappearing and timeout errors (see https://github.com/Valloric/YouCompleteMe/issues/2192 and https://github.com/Valloric/YouCompleteMe/issues/2574). Users even [increase the timeout by manually editing the `completion_request.py` file](https://github.com/Valloric/YouCompleteMe/blob/master/python/ycm/client/completion_request.py#L30) to avoid these errors, which exacerbate the issue and, in some cases, make the plugin unusable (see https://github.com/Valloric/YouCompleteMe/issues/2574).
- no fuzzy matching from omnifunc when forcing semantic completion. See https://github.com/Valloric/YouCompleteMe/issues/961;
- no fuzzy matching when deleting characters. Vim filtering is used instead:
Neovim and MacVim are not affected.
- completion menu disappears after deleting a character and inserting one:
Neovim and MacVim are not affected.
- ignore the start column returned by the server. See PR https://github.com/Valloric/YouCompleteMe/pull/2489.
- subject to flickers. This one depends a lot on the version of Vim. Completion is almost flicker-free in Neovim and MacVim. Not too bad in console Vim (except in fast terminal like [alacritty](https://github.com/jwilm/alacritty)). Awful in gVim GTK2 (a bit better on GTK3) and gVim on Windows.
This PR is an attempt at fixing all of these issues while reducing flickers as best as possible (due to how completion works in Vim, a flicker-free experience is impossible to achieve). Here's how:
- poll for completions using a timer and call `completefunc` once the completions are ready. Use the start column returned by the server in `completefunc`;
- immediately display the last completions on the `TextChangedI` event to prevent the popup menu disappearing while waiting for the completions. This reduces flickering;
- use the `InsertCharPre` event to close the completion menu just before inserting a character. This way the `TextChangedI` event is triggered when the character is inserted (this event is not fired when the completion menu is visible). This replaces the `refresh` option set to `always` in `completefunc` and the `s:cursor_moved` hack;
- remap the backspace key to close the completion menu when deleting a character and thus triggering the `TextChangedI` event;
- send a request with `force_semantic` set to `True` when forcing semantic completion instead of calling the omnifunc. This fixes the issue where there is no fuzzy matching for custom omnifuncs.
Here's a demo where I added a spin animation on the command line while loading the completions to show that it doesn't block the Vim UI:
Fixes#961.
Fixes#1282.
Fixes#1881.
Fixes#2192.
Fixes#2500.
Fixes#2574.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2657)
<!-- Reviewable:end -->
Bring fully asynchronous completion by polling for completions with a timer
then calling completefunc once the completions are ready. Use the start column
returned by the server in completefunc. Immediately display the last completion
on the TextChangedI event to prevent the popup menu disappearing while waiting
for the completions. Handle the TextChangedI event not being triggered while
the completion menu is open by closing the menu when inserting a character
through the InsertCharPre event, and when deleting a character on the <BS> and
<C-h> keys.
[READY] Require UTF-8 encoding
YCM expects the character encoding used by Vim to be UTF-8. If another encoding is used, inserting a non-ASCII character raises a `UnicodeDecodeError` exception resulting in a Python traceback inside Vim. However, supporting other encodings would make the code more complex for no good reason. A file in a different encoding than UTF-8 can be edited in Vim without any issue when the `encoding` option is set to `utf-8`. For instance, editing a file encoded in `latin-1` is as simple as:
```viml
:e ++enc=latin1
```
There is no need to change the `encoding` option for that.
We don't force `encoding` to be `utf-8` because changing this option is not safe after Vim start. From [the `encoding` documentation](http://vimdoc.sourceforge.net/htmldoc/options.html#'encoding'):
```
NOTE: Changing this option will not change the encoding of the
existing text in Vim. It may cause non-ASCII text to become invalid.
It should normally be kept at its default value, or set when Vim
starts up.
```
Instead, we require this option to be set to `utf-8` when loading the plugin. If not, we tell users to put the line:
```viml
set encoding=utf-8
```
in their vimrc.
This change should only affect Vim users on Windows where `encoding` is set to `latin1` by default (`encoding` in Neovim is always `utf-8` and cannot be changed). We update the Windows installation guide accordingly.
Closes https://github.com/Valloric/YouCompleteMe/issues/2416.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2691)
<!-- Reviewable:end -->
[READY] Use codepoint offsets in identifier functions
`CurrentIdentifierFinished` and `LastEnteredCharIsIdentifierChar` incorrectly use byte offsets with unicode lines (`CurrentColumn` returns a byte offset and `CurrentLineContents` a unicode string). This leads to weird bugs when there is a non-ASCII character on the current line:
This is fixed by converting byte offsets to codepoint ones through the `ByteOffsetToCodepointOffset` function.
This changes the behavior of these two functions when the current column position is invalid. Both functions returned false in that case. They now return as if the current column were at the end of the line. In practice, this doesn't really matter since the position of the current column should always be valid.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2690)
<!-- Reviewable:end -->
CurrentIdentifierFinished and LastEnteredCharIsIdentifierChar incorrectly use
byte offsets with unicode lines. Convert those offsets to codepoint offsets.
[Ready] Refactored diag interface, removed dummy sign.
# PR Prelude
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
Removed dummy sign logic, now it places news signs, then removes obsolete ones. I did this in my fork long ago and never experienced any flicker.
Some source code refactoring, which I believe simplifies logic a little.
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2677)
<!-- Reviewable:end -->
[READY] Moved change tracking to python. Per buffer diagnostics
Thank you for working on YCM! :)
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
Adds an emulation of Vim buffer object and moves change tracking into python.
Now diagnostics are stored per buffer, which effectively solves #2165 .
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2631)
<!-- Reviewable:end -->
Include following changes:
- allow compilation for iOS projects;
- handle unicode pathname when loading source on Python 2;
- update JediHTTP;
- update Boost to 1.64.0;
- create a symlink instead of renaming libclang;
- ignore identifiers from comments and strings on certain events;
- ignore identifiers from comments and strings by filetype;
- improve CSS identifier regex;
- fix Godef build error;
- search Python library in lib64 folder in addition to lib;
- specify .NET Framework 4.5.
coverage.py 4.4 removed the path from the filename attribute in its reports.
This leads to incorrect coverage from codecov as it relies on this attribute to
find the source file. Require coverage.py older than version 4.4.
Filepath space escaping
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
[Please explain **in detail** why the changes in this PR are needed.]
This provides the most basic test for `EscapedFilepath()`.
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2662)
<!-- Reviewable:end -->
[READY] Clean C# inserting namespace code
Now that we require at least Vim 7.4.1578, we can remove the code that automatically insert namespaces in C# for Vim versions prior to 7.4.774.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2650)
<!-- Reviewable:end -->
[READY] Display diagnostics and parse initial buffer asynchronously using timers
Currently, we wait for an autocommand to trigger to display diagnostics. This is not ideal as users have to do some actions like moving the cursor or editing the buffer to see them. We can improve that by using [the timers feature introduced in Vim 7.4.1578](975b5271ee) to display diagnostics asynchronously. Here's a demo:
By displaying diagnostics asynchronously, we can drop the `CursorHold` and `CursorHoldI` autocommands as well as [the `g:ycm_allow_changing_updatetime` option](5a806dcb30/README.md (the-gycm_allow_changing_updatetime-option)). In addition, we can now parse the current buffer and display diagnostics on the `TextChanged` event instead of the `CursorMoved` one. Note that we still need to check `b:changed_tick` for the `InsertLeave` event.
Given that our policy is to support the latest version of Ubuntu LTS which is 16.04, and that version [bundles Vim 7.4.1689](http://packages.ubuntu.com/xenial/vim), we can increase our Vim version requirement to 7.4.1578.
Next steps will be to use timers for parsing the buffer when the server becomes ready and for completions.
Fixes https://github.com/Valloric/YouCompleteMe/issues/2165.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2636)
<!-- Reviewable:end -->
Use the timers feature to display diagnostics asynchronously instead of waiting
for an autocommand to trigger.
Increase Vim version requirement to 7.4.1578.
Drop the CursorHold and CursorHoldI autocommands.
Parse buffer on the TextChanged autocommand instead of CursorMoved.
[READY] Improve reparse requirement on BufEnter event
When opening a file, Vim triggers the `BufRead` event then the `BufEnter` one. By setting the `s:previous_allowed_buffer_number` variable in `s:AllowedToCompleteInBuffer`, we avoid a reparse on `BufEnter` after parsing the buffer on `BufRead`. This reduces the number of `BufferVisit` and `FileReadyToParse` requests sent to the server by one when opening a file.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2644)
<!-- Reviewable:end -->
[READY] Drop BufRead event
Since [we don't parse the buffer if no filetype is set](ad00f4c3f4/autoload/youcompleteme.vim (L334)), calling `s:OnBufferRead` on the `BufRead` event is only useful if the buffer filetype is set. But if that's the case, this means that the `FileType` event was fired before the `BufRead` one and thus the `s:OnBufferRead` function is already called. So, there is no point in calling `s:OnBufferRead` on the `BufRead` event. In fact, this costs us a `BufferVisit` and a `FileReadyToParse` request for nothing.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2645)
<!-- Reviewable:end -->
Considering that a buffer won't be parsed if no filetype is set and that it's
already parsed if the filetype is set when reading a buffer, there is no point
in parsing a buffer on the BufRead event.
[READY] Update autocommands examples in documentation
I am not aware of a Vim option to not switch to the location list (or quickfix) window when calling `lopen` (respectively `copen`) so I think it's a good idea to give an example in the docs on how to achieve this using the `YcmLocationOpened` and `YcmQuickFixOpened` autocommands.
Also, the `execute` calls are not needed.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2635)
<!-- Reviewable:end -->
[READY] Force parsing on buffer visit
When visiting a buffer that was previously parsed, its last parse may be outdated because of changes from other buffers. We need to force a new parse.
This partially fixes issue https://github.com/Valloric/YouCompleteMe/issues/2165 in the sense that you don't have to edit the buffer (or call `:YcmForceCompileAndDiagnostics`) anymore to refresh diagnostics but you still have to wait for the `CursorHold` event. This can be further improved by displaying diagnostics asynchronously using timers.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2629)
<!-- Reviewable:end -->
[READY] Refactor b:ycm_changedtick variable
Since we only use the `b:ycm_changedtick` variable to track the last time we parsed the buffer, we can change it to store a number instead of a dictionary.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2614)
<!-- Reviewable:end -->
