[READY] Improve diagnostic matches display
There are two issues with how we display diagnostic matches. The first issue is that if the current buffer contains diagnostic matches and is split in a new window, all the matches are cleared and only matches in the new window are shown. The second issue is that if a new buffer with no diagnostic support is open in the current window and that window already contained matches then the matches are still displayed. Here's an illustration of both issues (signs are disabled):
The solution is to add an autocommand that updates matches on the `BufEnter` and `WinEnter` events. Here's the result:
Note that it's not perfect as multiple windows of the same buffer won't be updated simultaneously. Supporting that scenario is rather tricky because we would need to go through all the windows to update the matches and switching windows can lead to a lot of issues (like interrupting visual mode) so we don't.
This PR also improves how we update matches by only displaying matches that are not already present and then clearing the remaining ones (similarly to what we do with signs; see PR https://github.com/Valloric/YouCompleteMe/pull/2915) instead of always clearing all the matches then displaying the new ones.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2922)
<!-- Reviewable:end -->
[READY] Add function to capture Vim command output
Since there are now two places where we need to capture the output of a Vim command (for diagnostic signs and for syntax keywords), we should add a function for that.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2921)
<!-- Reviewable:end -->
[READY] Improve diagnostic sign placement
Currently, we store internally the signs that we place in the buffer so that when we receive a new batch of diagnostics, we can efficiently add and remove signs by looking at the line positions of the stored signs. However, there is a flaw in that approach. When lines are added or removed to the buffer, Vim update the lines of the signs accordingly and thus our stored signs may not correspond anymore to the actual signs. Here's an example of the issue:
*Where did my signs go? I want a refund!*
The solution is to not store the signs but instead get them directly from Vim each time we need to update them. This is done by parsing the output of the `:sign place` command (Vim doesn't provide an API to properly do that). The same example as above with the proposed changes:
*Oh, they were there.*
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2915)
<!-- Reviewable:end -->
[READY] Support for additional FixIts on java completions, e.g. for automatic imports
Java completer can include FixIts which are applied when a completion
entry is selected. We use the existing mechanism implemented for c-sharp
to perform these edits using the CompleteDone autocommand.
However, the existing mechanism relies on pattern matching the source to
work out which item was completed. Vim patch 8.0.1493 introduces support
for user_data on completion items, so when available we populate it with
the completion array index of the item and use that to get the exact
element that was selected. This is both a lot faster and a lot more
accirate.
Of course when applying these 'FixIts' we don't interrupt the user with
confirmation or the quickfix list as this would just be annoying. If the
server reports that an edit must be made, we just make the edit.
## Use SplitLines in ReplaceChunk to allow newlines to be inserted.
The Java completer frequently inserts newlines as part of its FixIts. We
previously used the base python splitlines implementation which consumed
terminating newlines and also consumed empty strings.
We can therefore now remove the duplicate newline in InsertNamespace, as
this was only to work around the splitlines behaviour.
In the tests, be clear that replacement_text in ReplaceChunks is Unicode
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [X] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [X] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [X] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [X] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
Automatic addition of import statements is a highly desirable feature when working with languages like java where it is idiomatic to literally import everything by name. Users expect this function: it was the first question I got when i put java live at work.
I was at first reticent to include it due to the irksome complete_done interface, but since Bram merged my PR: 9b56a57cda we can now identify the exact completion selected which makes this a _lot_ more robust.
We can't just remove the old code as nobody will actually have that version yet, but I have tested before and after and it is fully backwardly compatible.
# Test case
A simple way to verify this is with the ycmd tests:
* open `third_party/ycmd/ycmd/tests/java/testdata/simple_eclipse_project/src/com/test/TestFactory.java`
* Invoke completion for `CUTHBERT` on line 19 (replace to end of line)
* select one of the enum values
You should get the enum value automatically imported at the top of the file.
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2907)
<!-- Reviewable:end -->
Java completer can include FixIts which are applied when a completion
entry is selected. We use the existing mechanism implemented for c-sharp
to perform these edits using the CompleteDone autocommand.
However, the existing mechanism relies on pattern matching the source to
work out which item was completed. Vim patch 8.0.1493 introduces support
for user_data on completion items, so when available we populate it with
the completion array index of the item and use that to get the exact
element that was selected. This is both a lot faster and a lot more
accirate.
Of course when applying these 'FixIts' we don't interrupt the user with
confirmation or the quickfix list as this would just be annoying. If the
server reports that an edit must be made, we just make the edit. This is
achieved by adding a silent flag to ReplaceChunks.
[READY] Apply chunks from bottom to top
Without the proposed fix, the test included in that PR fails as follows
```
FAIL: ycm.tests.vimsupport_test.ReplaceChunksInBuffer_LineOverlappingChunks_test
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Python36\lib\site-packages\nose\case.py", line 198, in runTest
self.test(*self.arg)
File "C:\Users\micbou\projects\YouCompleteMe\python\ycm\tests\vimsupport_test.py", line 768, in ReplaceChunksInBuffer_LineOverlappingChunks_test
AssertBuffersAreEqualAsBytes( expected_buffer, result_buffer )
File "C:\Users\micbou\projects\YouCompleteMe\python\ycm\tests\vimsupport_test.py", line 86, in AssertBuffersAreEqualAsBytes
eq_( ToBytes( result_line ), ToBytes( expected_line ) )
AssertionError: b' third line' != b' third '
```
Found the issue while trying to add code formatting support to the language server completer. The test is based on a real-world scenario when formatting with jdt.ls (for some reason, jdt.ls is including the newline of the previous line when fixing indentation).
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2898)
<!-- Reviewable:end -->
[READY] Java support with asynchronous diagnostics and messages
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
This change is required for a better user experience when using native
java support
This implements an asynchronous message system using a long-poll request
to the server.
The server provides an endpoint /receive_messages which blocks until
either a timeout occurs or we receive a batch of asynchronous messages.
We send this request asynchronously and poll it 4 times a second to see
if we have received any messages.
The messages may either be simply for display (such as startup progress)
or diagnostics, which override the diagnostics returned by
OnFileReqdyToParse.
In the former case, we simply display the message, accepting that this
might be overwritten by any other message (indeed, requiring this), and
for the latter we fan out diagnostics to any open buffer for the file in
question.
Unfortunately, Vim has bugs related to timers when there is something
displayed (such as a "confirm" prompt or other), so we suspend
background timers when doing subcommands to avoid vim bugs. NOTE: This
requires a new version of Vim (detected by the presence of the
particular functions used).
NOT_READY because:
- the submodule commit points at my repo and requires https://github.com/Valloric/ycmd/pull/857 to be merged
- my spider sense suggest i have more testing to do...
Notes:
- Part 3 (I think) of the Java support PRs. This one actually adds the minimal changes for working java support
- There are about 2 or 3 other PRs to come to add things like automatic module imports, etc.
[Please explain **in detail** why the changes in this PR are needed.]
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2863)
<!-- Reviewable:end -->
[READY] Avoid traceback when computing extra conf data
Evaluating the values of [the `g:ycm_extra_conf_vim_data` option](https://github.com/Valloric/YouCompleteMe#the-gycm_extra_conf_vim_data-option) may raise a Python exception (e.g. one of the values is not defined). Since that option is evaluated each time a request is sent, such exception can render the editor almost unusable as each key press will print a Python traceback to the user.
A simple way to reproduce the issue is to add the following line in vimrc:
```viml
let g:ycm_extra_conf_vim_data = [ 'undefined_value' ]
```
We fix that by catching and logging the exception.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2899)
<!-- Reviewable:end -->
This implements an asynchronous message system using a long-poll request
to the server.
The server provides an endpoint /receive_messages which blocks until
either a timeout occurs or we receive a batch of asynchronous messages.
We send this request asynchronously and poll it 4 times a second to see
if we have received any messages.
The messages may either be simply for display (such as startup progress)
or diagnostics, which override the diagnostics returned by
OnFileReqdyToParse.
In the former case, we simply display the message, accepting that this
might be overwritten by any other message (indeed, requiring this), and
for the latter we fan out diagnostics to any open buffer for the file in
question.
Unfortunately, Vim has bugs related to timers when there is something
displayed (such as a "confirm" prompt or other), so we suspend
background timers when doing subcommands to avoid vim bugs. NOTE: This
requires a new version of Vim (detected by the presence of the
particular functions used).
Evaluating the values of g:ycm_extra_conf_vim_data may raise a Python exception
(e.g. one of the values is not defined). Since that option is parsed each time
a request is sent, such exception makes the editor almost unusable as each key
press is printing a Python traceback to the user. Catch and log the exception.
Include the following changes:
- add support for Windows flags when --driver-mode=cl is given;
- hide C++ symbols by default;
- add Java support using jdt.ls;
- translate libclang error codes to exceptions;
- support downloading Clang binaries on ARM systems;
- handle zero column diagnostic from OmniSharp;
- specify Platform property when compiling OmniSharp;
- use current working directory in JavaScript completer;
- update Boost to 1.66.0;
- update JediHTTP;
- update Clang to 5.0.1;
- fix building with system libclang on Gentoo amd64;
- drop Python 2.6 and Python 3.3 support;
- calculate the start column when items are not resolved in the language
server completer;
- download Clang binaries through HTTPS;
- do not try to symlink libclang on Windows.
[READY] Drop Python 2.6 and Python 3.3
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
[Please explain **in detail** why the changes in this PR are needed.]
Since we are dropping support for Python 2.6 and 3.3 in Valloric/ycmd#904 we should drop support for them in YCM as well.
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2887)
<!-- Reviewable:end -->
[READY] Fix truncation of messages when len(message) == vim_width
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
[Please explain **in detail** why the changes in this PR are needed.]
When the message is exactly as wide as the vim window, vim still
displays "press enter..." message.
The bug can be demonstrated if we hardcode the `message` to `&columns * 'a'` and forcing `truncate` to `True`.
No tests because I didn't think it was necessary.
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2889)
<!-- Reviewable:end -->
When the message is exactly as wide as the vim window, vim still
displays "press enter..." message.
Wen we want to avoid that we need to cut the message one character
shorter.
[READY] Optimize request building
This PR significantly reduces the time spent to build the request when there are a lot of buffers. This is done by:
- using the `options` property on the buffer object to get the `mod` variable instead of evaluating `getbufvar`;
- not computing the buffer filepath if the buffer is not modified;
- passing the number of the unloaded buffer instead of its filepath on the `BufferUnload` event. Getting the Python buffer object from its number is easier than from its filepath.
Here are the times spent to edit a new empty buffer (i.e. two event notifications `BufferVisit` and `FileReadyToParse`) for different numbers of buffers already open in the editor before and after the changes:
<table>
<tr>
<th>Number of buffers open</th>
<th>Before (ms)</th>
<th>After (ms)</th>
</tr>
<tr>
<td>1</td>
<td>1.2</td>
<td>1.0</td>
</tr>
<tr>
<td>10</td>
<td>1.2</td>
<td>1.0</td>
</tr>
<tr>
<td>100</td>
<td>2.4</td>
<td>1.1</td>
</tr>
<tr>
<td>1000</td>
<td>13.5</td>
<td>2.2</td>
</tr>
<tr>
<td>10000</td>
<td>139.8</td>
<td>17.3</td>
</tr>
</table>
Results obtained with the `prof.py` script from [that branch](https://github.com/micbou/YouCompleteMe/tree/profiling-request-building).
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2871)
<!-- Reviewable:end -->
Reduce the time spent to build the request when there are a lot of buffers by:
- using the options property on the buffer object to get the mod variable
instead of evaluating getbufvar;
- not computing the buffer filepath if the buffer is not modified;
- passing the number of the unloaded buffer instead of its filepath on the
BufferUnload event. Getting the Python buffer object from its number is
easier than from its filepath.
[READY] Signpost the contact details earlier in the README
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
At the request of the nice folk over at `#vim` on free node, this PR signposts the contact details and installation instructions at the top of the readme so that users seeking help use our contact details, rather than asking over there.
Overall it is better for users to seek support from us because:
* we have better knowledge and can solve issues more quickly
* we get better visibility of in-the-wild issues and sticking points
* certain #vim users feel that they are unduly burdened with supporting setup issues for YCM
cc @romainl
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2866)
<!-- Reviewable:end -->
[READY] Use fake filetype for omni completion test
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [X] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [X] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [X] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [X] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
Now that the server supports Java natively (or will soon), we need to use a
filetype that does not have native support to ensure that we call the
mocked omnifunc, but also one for which YCM has a semantic trigger.
Rather than picking another random languagse, which might break in the
future, we invent a language (ycmtest) and supply a manual trigger
mapping to the YouCompleteMe object.
References:
* Part 2 of the java completer client changes
* You probably previously saw this using ruby, but this was always a hack. This solution is permanent.
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2862)
<!-- Reviewable:end -->
Now that the server supports Java natively, we need to use a
filetype that does not have native support to ensure that we call the
mocked omnifunc, but also one for which YCM has a semantic trigger.
Rather than picking another random languagse, which might break in the
future, we invent a language (ycmtest) and supply a manual trigger
mapping to the YouCompleteMe object.
[READY] Always supply working directory
This allows completer servers to detect the correct directory to launch
no matter what request initialises the completer server.
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [X] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [X] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [X] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [X] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
This change ensures that the client supplies the working directory to the server on each request, in particular, event notifications; the java and javascript completers use this to start the server in the `FileReadyToParse` event, rather than on construction of the completer. The subtle difference allows them to use the client's working directory, rather than the _server's_ working directory to do things like project detection.
See also:
* This is PR 1 of the set of changes related to #2827 - client changes required to support Java and other language-server protocols.
* ycmd change for javascript that makes use of this now: https://github.com/Valloric/ycmd/pull/886
* updated API doc: http://puremourning.github.io/ycmd-1
The ycmd API change is simply to allow `working_dir` on all requests.
cc for minor optional API change: @abingham @Qusic @LuckyGeck @mawww @richard1122 @jakeanq @orsonteodoro
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2861)
<!-- Reviewable:end -->
When no argument is given to the YcmToggleLogs command, instead of displaying
the list of available logfiles, prompt the user to open (or close if already
open) one of them.
