[READY] Insert keys at the start of the typeahead buffer
When sending keys to Vim through `feedkeys`, these keys are by default added to the end of the typeahead buffer. If there are already keys in the buffer, they will be processed first and may change the state that our keys combination was sent for. For instance, `<C-X><C-U><C-P>` may be sent in normal mode instead of insert mode (issue #2731) or `<C-e>` outside of completion mode (issue #2732). We avoid that by inserting the keys at the start of the typeahead buffer with the `i` option.
Fixes#2731.
Fixes#2732.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2733)
<!-- Reviewable:end -->
When sending keys to Vim, they are by default added to the end of the typeahead
buffer. If there are already keys in the buffer, they will be processed first
and may change the state that our keys combination was sent for (e.g.
<C-X><C-U><C-P> in normal mode instead of insert mode or <C-e> outside of
completion mode). We avoid that by inserting the keys at the start of the
typeahead buffer.
[RFC] Add a more obvious explanation for users of the default system Vim on macOS
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [X] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [X] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [X] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [X] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
No tests because we don't have Vimscript tests. But I tested:
* System Vim on macOS:
```
BeniMac:YouCompleteMe-Clean ben$ /usr/bin/vi
YouCompleteMe unavailable: requires Vim 7.4.1578+.
Info: You appear to be running the default system Vim on macOS. It reports as patch 8056, but it is really older than 1578. Please consider MacVim, homebrew Vim or a self-built Vim that satisfies the minimum requirement.
Press ENTER or type command to continue
```
* Manually built too-old Vim on macOS (regression test)
```
BeniMac:vim ben$ ./src/vim
YouCompleteMe unavailable: requires Vim 7.4.1578+.
Press ENTER or type command to continue
BeniMac:vim ben$
```
* MacVim (latest) works
* Homebrew Vim (latest) works
* Whatever random Vim I was using before works
# Why this change is necessary and useful
As the comment says, and referred in [this issue]( https://github.com/Valloric/YouCompleteMe/issues/2721#issuecomment-317272969) and [this comment](https://github.com/Valloric/YouCompleteMe/issues/549#issuecomment-317272203) users of macOS are being flummoxed by the reporting of its version on the launch screen as some bogus patch number. I guess Apple patched it for some reason (such as their OS protection or something), and just chose a random high number (or perhaps some v8 back patch, who knows).
Anyway, a specific error message for such users means they can get a working system without reporting to us, but it's our overhead to maintain the extra code. So RFC for that reason.
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2728)
<!-- Reviewable:end -->
[READY] Check completefunc when forcing semantic completion
When `completefunc` is not set (because YCM is not allowed to complete in the current buffer or the server failed to start) and semantic completion is forced with `<C-Space>`, Vim returns the following error:
```
E764: Option 'completefunc' is not set
```
This happens because we don't check if `completefunc` is set to `youcompleteme#CompleteFunc` in that case.
To completely avoid this issue, we always set the `completefunc` before invoking it. This may look inefficient but, in practice, the performance cost is almost nil.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2722)
<!-- Reviewable:end -->
Include following changes:
- update bottle to 0.12.13;
- update Clang to 4.0.1;
- fix compilation with Clang 3.4;
- only configure tests and benchmarks if required;
- fix finding Python library in virtualenv;
- switch back to Boost regex.
[READY] Fix unicode warning when unloading buffer on Python 2
When a buffer is unloaded, [we compare its name to the current buffer one in the `BuildRequestData` function](5b89d41832/python/ycm/client/base_request.py (L160)). This raises a unicode warning on Python 2 when the deleted buffer name contains non-ASCII characters because the deleted buffer name is a byte object while the current buffer name is a unicode one.
I updated the `BufferUnload` test to show the issue. I'll send the fix once the builds failed.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2710)
<!-- Reviewable:end -->
[READY] Do not cache omnifunc on FileReadyToParse event
We currently cache the omnifunc when the file is ready to be parsed i.e. on the `FileType`, `BufEnter` events and possibly on `InsertLeave` and `TextChanged`. That's problematic if the omnifunc is modified after these events and the user is requesting completions from it. This was the cause of issue https://github.com/Valloric/YouCompleteMe/issues/1027 but is not relevant anymore because we now always parse the file on the `BufEnter` event and thus the omnifunc is properly updated after switching buffers. Still, it's more correct to cache the omnifunc inside the `ShouldUseNow` method since `ComputeCandidatesInner` is called immediately after. Performance is not a concern: it takes less than 10 μs to get the omnifunc.
Fixes https://github.com/Valloric/YouCompleteMe/issues/1027.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2708)
<!-- Reviewable:end -->
[READY] Restore cursor position after omnifunc call
When compiled without C-family support, YCM will use the default omnifunc from Vim (`ccomplete#Complete`) to provide semantic completion. This omnifunc calls [`searchdecl`](http://vimdoc.sourceforge.net/htmldoc/eval.html#searchdecl()) to find a declaration, which is supposed to move the cursor to that declaration. However, the cursor is not moved when called through the omni completion mapping (`CTRL-X CTRL-O`). Since PR https://github.com/Valloric/YouCompleteMe/pull/2657, YCM calls the omnifunc outside completion mode and thus the cursor is moved to the found declaration after typing `.` or `->`.
Considering this `searchdecl` trick may be used by other omnifuncs, we fix the issue by always restoring the cursor position after calling the omnifunc.
Fixes#2698.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2707)
<!-- Reviewable:end -->
Calling directly the omnifunc may move the cursor position. This is the case
with the default Vim omnifunc for C-family languages (ccomplete#Complete) which
calls searchdecl to find a declaration. This function is supposed to move the
cursor to the found declaration but it doesn't when called through the omni
completion mapping (CTRL-X CTRL-O). So, we restore the cursor position after
calling the omnifunc.
[READY] Rewrite omnifunc tests
Change the `VimBuffer` object to accept a Python function as its omnifunc, e.g.
```python
def Omnifunc( findstart, base ):
if findstart:
return 5
return [ 'a', 'b', 'c' ]
```
and rewrite the omnifunc tests accordingly. This reduces the amount of mocking done directly in these tests.
Also, extend the `ToBytesOnPY2` function to lists and dictionaries so that, instead of having to write
```python
[ ToBytesOnPY2( 'a' ), ToBytesOnPY2( 'b' ), ToBytesOnPY2( 'c' ) ]
```
in tests, one can write:
```python
ToBytesOnPY2( [ 'a', 'b', 'c' ] )
```
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2701)
<!-- Reviewable:end -->
[READY] Fix unicode warning when jumping on Python 2
On Python 2, jumping in a file whose path contains non-ASCII characters raises the following warnings:
```
Error detected while processing function <SNR>26_CompleterCommand:
line 18:
python\ycm\vimsupport.py:400: UnicodeWarning: Unicode unequal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
if filename != GetCurrentBufferFilepath():
python\ycm\vimsupport.py:375: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
if win.buffer.name == filepath:
```
This happens because, in the `JumpToLocation` function, we are comparing a unicode object (`filename` and `filepath`) with a Vim buffer name (`vim.current.buffer.name` returned by `GetCurrentBufferFilepath` and `win.buffer.name` where `win` is a Vim window) which is a byte object on Python 2.
For now, this PR adds tests covering the `JumpToLocation` function with unicode paths. They will raise the warnings on Python 2 and will fail because warnings are now treated as errors in tests. I'll update the PR with the fix once the builds are done.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2702)
<!-- Reviewable:end -->
The 'hidden' option is a global option, not a buffer one. If this option is
false, we should check if the 'bufhidden' option, which is local to the buffer,
is set to 'hide'. If so, the buffer can be hidden despite the 'hidden' option
being false.
[READY] Check if server is ready before running tests
We don't need to mock the `IsServerReady` function if we check that the server is ready before running each test.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2700)
<!-- Reviewable:end -->
Correct vim 7.4 patch requirement
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [ ] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
[Please explain **in detail** why the changes in this PR are needed.]
With Valloric/YouCompleteMe#2636 we bumped minimum required vim version to 7.4.1578, but in one place we didn't change the value.
Yes, this PR is just one number change, but inconsitency bothers me.
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2697)
<!-- Reviewable:end -->
[READY] Rewrite completion system
There is a number of issues with the current completion system:
- UI is blocked until the completions are returned by the server, the request timed out, or a key is pressed by the user. This leads to a rough experience when completions take too much time: cursor disappearing and timeout errors (see https://github.com/Valloric/YouCompleteMe/issues/2192 and https://github.com/Valloric/YouCompleteMe/issues/2574). Users even [increase the timeout by manually editing the `completion_request.py` file](https://github.com/Valloric/YouCompleteMe/blob/master/python/ycm/client/completion_request.py#L30) to avoid these errors, which exacerbate the issue and, in some cases, make the plugin unusable (see https://github.com/Valloric/YouCompleteMe/issues/2574).
- no fuzzy matching from omnifunc when forcing semantic completion. See https://github.com/Valloric/YouCompleteMe/issues/961;
- no fuzzy matching when deleting characters. Vim filtering is used instead:
Neovim and MacVim are not affected.
- completion menu disappears after deleting a character and inserting one:
Neovim and MacVim are not affected.
- ignore the start column returned by the server. See PR https://github.com/Valloric/YouCompleteMe/pull/2489.
- subject to flickers. This one depends a lot on the version of Vim. Completion is almost flicker-free in Neovim and MacVim. Not too bad in console Vim (except in fast terminal like [alacritty](https://github.com/jwilm/alacritty)). Awful in gVim GTK2 (a bit better on GTK3) and gVim on Windows.
This PR is an attempt at fixing all of these issues while reducing flickers as best as possible (due to how completion works in Vim, a flicker-free experience is impossible to achieve). Here's how:
- poll for completions using a timer and call `completefunc` once the completions are ready. Use the start column returned by the server in `completefunc`;
- immediately display the last completions on the `TextChangedI` event to prevent the popup menu disappearing while waiting for the completions. This reduces flickering;
- use the `InsertCharPre` event to close the completion menu just before inserting a character. This way the `TextChangedI` event is triggered when the character is inserted (this event is not fired when the completion menu is visible). This replaces the `refresh` option set to `always` in `completefunc` and the `s:cursor_moved` hack;
- remap the backspace key to close the completion menu when deleting a character and thus triggering the `TextChangedI` event;
- send a request with `force_semantic` set to `True` when forcing semantic completion instead of calling the omnifunc. This fixes the issue where there is no fuzzy matching for custom omnifuncs.
Here's a demo where I added a spin animation on the command line while loading the completions to show that it doesn't block the Vim UI:
Fixes#961.
Fixes#1282.
Fixes#1881.
Fixes#2192.
Fixes#2500.
Fixes#2574.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2657)
<!-- Reviewable:end -->
Bring fully asynchronous completion by polling for completions with a timer
then calling completefunc once the completions are ready. Use the start column
returned by the server in completefunc. Immediately display the last completion
on the TextChangedI event to prevent the popup menu disappearing while waiting
for the completions. Handle the TextChangedI event not being triggered while
the completion menu is open by closing the menu when inserting a character
through the InsertCharPre event, and when deleting a character on the <BS> and
<C-h> keys.
[READY] Require UTF-8 encoding
YCM expects the character encoding used by Vim to be UTF-8. If another encoding is used, inserting a non-ASCII character raises a `UnicodeDecodeError` exception resulting in a Python traceback inside Vim. However, supporting other encodings would make the code more complex for no good reason. A file in a different encoding than UTF-8 can be edited in Vim without any issue when the `encoding` option is set to `utf-8`. For instance, editing a file encoded in `latin-1` is as simple as:
```viml
:e ++enc=latin1
```
There is no need to change the `encoding` option for that.
We don't force `encoding` to be `utf-8` because changing this option is not safe after Vim start. From [the `encoding` documentation](http://vimdoc.sourceforge.net/htmldoc/options.html#'encoding'):
```
NOTE: Changing this option will not change the encoding of the
existing text in Vim. It may cause non-ASCII text to become invalid.
It should normally be kept at its default value, or set when Vim
starts up.
```
Instead, we require this option to be set to `utf-8` when loading the plugin. If not, we tell users to put the line:
```viml
set encoding=utf-8
```
in their vimrc.
This change should only affect Vim users on Windows where `encoding` is set to `latin1` by default (`encoding` in Neovim is always `utf-8` and cannot be changed). We update the Windows installation guide accordingly.
Closes https://github.com/Valloric/YouCompleteMe/issues/2416.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2691)
<!-- Reviewable:end -->
[READY] Use codepoint offsets in identifier functions
`CurrentIdentifierFinished` and `LastEnteredCharIsIdentifierChar` incorrectly use byte offsets with unicode lines (`CurrentColumn` returns a byte offset and `CurrentLineContents` a unicode string). This leads to weird bugs when there is a non-ASCII character on the current line:
This is fixed by converting byte offsets to codepoint ones through the `ByteOffsetToCodepointOffset` function.
This changes the behavior of these two functions when the current column position is invalid. Both functions returned false in that case. They now return as if the current column were at the end of the line. In practice, this doesn't really matter since the position of the current column should always be valid.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2690)
<!-- Reviewable:end -->
CurrentIdentifierFinished and LastEnteredCharIsIdentifierChar incorrectly use
byte offsets with unicode lines. Convert those offsets to codepoint offsets.
[Ready] Refactored diag interface, removed dummy sign.
# PR Prelude
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
Removed dummy sign logic, now it places news signs, then removes obsolete ones. I did this in my fork long ago and never experienced any flicker.
Some source code refactoring, which I believe simplifies logic a little.
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2677)
<!-- Reviewable:end -->
