[READY] Fix index error when applying a chunk at the end of file
When sending a request to the server, [we add a newline to the buffer contents to match what gets saved to disk](d3381411a0/python/ycm/vimsupport.py (L117)). If the server generates a chunk containing that newline, this chunk goes past the Vim buffer contents since there is actually no new line, which raises an `IndexError` exception when applying the chunk. We fix the issue by recomputing the end position of where the chunk is applied and by removing all trailing characters in the chunk.
Steps to reproduce the issue:
- create a `Test.java` file with the following contents:
```java
package Test;$
$
```
where `$` represents an end of line;
- issue the `:YcmCompleter Format` command.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2951)
<!-- Reviewable:end -->
When sending a request to the server, we add a newline to the buffer contents
to match what gets saved to disk. If the server generates a chunk containing
that newline, this chunk goes past the Vim buffer contents since there is
actually no new line, which raises an IndexError exception when applying the
chunk. We fix the issue by recomputing the end position of where the chunk is
applied and by removing all trailing characters in the chunk.
[READY] Remove 7-Zip dependency from Windows instructions
7-Zip is not required anymore on Windows since we now extract libclang from our `.tar.bz` packages.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2942)
<!-- Reviewable:end -->
[READY] Don't open the quickfix window after FixIt/Refactor
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
When FixIt was originally implemented, I decided to present the quick fix list containing the set of changes made to the buffers as it was thought this would be helpful and would reassure the user that the changes we made to their code made sense and were safe. Some large part of this decision was paranoia.
Since then we're using fixit code in a lot more places and the feature is pretty mature. I personally find the opening of the quick fix window to be a bit of a pain, and I quickly end up closing it manually to reclaim the space. I have literally never used it to jump to a location that was changed.
I have received recent feedback from users that "Why is it when I do a FixIt, this annoying window appears at the bottom of the screen that I can't get rid of?". Most users who are unhappy with the result of a FixIt can, and do, use the undo command as a matter of habit.
So I propose to just not auto-display the quick fix window anymore for FixIts, Renames, etc. We continue to populate the quick fix list, however, so `:copen` can be used to inspect the results, for those power users that know about quick fix lists.
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2926)
<!-- Reviewable:end -->
[READY] Prefix commands with python in Windows instructions
Arguments may be ignored when running the `install.py` script in a `cmd.exe` prompt if the command associated to Python files is incorrect. Update the Windows instructions by prefixing the commands with `python`.
Fixes#2929.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2931)
<!-- Reviewable:end -->
[READY] Improve diagnostic matches display
There are two issues with how we display diagnostic matches. The first issue is that if the current buffer contains diagnostic matches and is split in a new window, all the matches are cleared and only matches in the new window are shown. The second issue is that if a new buffer with no diagnostic support is open in the current window and that window already contained matches then the matches are still displayed. Here's an illustration of both issues (signs are disabled):
The solution is to add an autocommand that updates matches on the `BufEnter` and `WinEnter` events. Here's the result:
Note that it's not perfect as multiple windows of the same buffer won't be updated simultaneously. Supporting that scenario is rather tricky because we would need to go through all the windows to update the matches and switching windows can lead to a lot of issues (like interrupting visual mode) so we don't.
This PR also improves how we update matches by only displaying matches that are not already present and then clearing the remaining ones (similarly to what we do with signs; see PR https://github.com/Valloric/YouCompleteMe/pull/2915) instead of always clearing all the matches then displaying the new ones.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2922)
<!-- Reviewable:end -->
[READY] Add function to capture Vim command output
Since there are now two places where we need to capture the output of a Vim command (for diagnostic signs and for syntax keywords), we should add a function for that.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2921)
<!-- Reviewable:end -->
[READY] Improve diagnostic sign placement
Currently, we store internally the signs that we place in the buffer so that when we receive a new batch of diagnostics, we can efficiently add and remove signs by looking at the line positions of the stored signs. However, there is a flaw in that approach. When lines are added or removed to the buffer, Vim update the lines of the signs accordingly and thus our stored signs may not correspond anymore to the actual signs. Here's an example of the issue:
*Where did my signs go? I want a refund!*
The solution is to not store the signs but instead get them directly from Vim each time we need to update them. This is done by parsing the output of the `:sign place` command (Vim doesn't provide an API to properly do that). The same example as above with the proposed changes:
*Oh, they were there.*
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2915)
<!-- Reviewable:end -->
[READY] Support for additional FixIts on java completions, e.g. for automatic imports
Java completer can include FixIts which are applied when a completion
entry is selected. We use the existing mechanism implemented for c-sharp
to perform these edits using the CompleteDone autocommand.
However, the existing mechanism relies on pattern matching the source to
work out which item was completed. Vim patch 8.0.1493 introduces support
for user_data on completion items, so when available we populate it with
the completion array index of the item and use that to get the exact
element that was selected. This is both a lot faster and a lot more
accirate.
Of course when applying these 'FixIts' we don't interrupt the user with
confirmation or the quickfix list as this would just be annoying. If the
server reports that an edit must be made, we just make the edit.
## Use SplitLines in ReplaceChunk to allow newlines to be inserted.
The Java completer frequently inserts newlines as part of its FixIts. We
previously used the base python splitlines implementation which consumed
terminating newlines and also consumed empty strings.
We can therefore now remove the duplicate newline in InsertNamespace, as
this was only to work around the splitlines behaviour.
In the tests, be clear that replacement_text in ReplaceChunks is Unicode
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [X] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [X] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [X] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [X] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
Automatic addition of import statements is a highly desirable feature when working with languages like java where it is idiomatic to literally import everything by name. Users expect this function: it was the first question I got when i put java live at work.
I was at first reticent to include it due to the irksome complete_done interface, but since Bram merged my PR: 9b56a57cda we can now identify the exact completion selected which makes this a _lot_ more robust.
We can't just remove the old code as nobody will actually have that version yet, but I have tested before and after and it is fully backwardly compatible.
# Test case
A simple way to verify this is with the ycmd tests:
* open `third_party/ycmd/ycmd/tests/java/testdata/simple_eclipse_project/src/com/test/TestFactory.java`
* Invoke completion for `CUTHBERT` on line 19 (replace to end of line)
* select one of the enum values
You should get the enum value automatically imported at the top of the file.
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2907)
<!-- Reviewable:end -->
Java completer can include FixIts which are applied when a completion
entry is selected. We use the existing mechanism implemented for c-sharp
to perform these edits using the CompleteDone autocommand.
However, the existing mechanism relies on pattern matching the source to
work out which item was completed. Vim patch 8.0.1493 introduces support
for user_data on completion items, so when available we populate it with
the completion array index of the item and use that to get the exact
element that was selected. This is both a lot faster and a lot more
accirate.
Of course when applying these 'FixIts' we don't interrupt the user with
confirmation or the quickfix list as this would just be annoying. If the
server reports that an edit must be made, we just make the edit. This is
achieved by adding a silent flag to ReplaceChunks.
[READY] Apply chunks from bottom to top
Without the proposed fix, the test included in that PR fails as follows
```
FAIL: ycm.tests.vimsupport_test.ReplaceChunksInBuffer_LineOverlappingChunks_test
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Python36\lib\site-packages\nose\case.py", line 198, in runTest
self.test(*self.arg)
File "C:\Users\micbou\projects\YouCompleteMe\python\ycm\tests\vimsupport_test.py", line 768, in ReplaceChunksInBuffer_LineOverlappingChunks_test
AssertBuffersAreEqualAsBytes( expected_buffer, result_buffer )
File "C:\Users\micbou\projects\YouCompleteMe\python\ycm\tests\vimsupport_test.py", line 86, in AssertBuffersAreEqualAsBytes
eq_( ToBytes( result_line ), ToBytes( expected_line ) )
AssertionError: b' third line' != b' third '
```
Found the issue while trying to add code formatting support to the language server completer. The test is based on a real-world scenario when formatting with jdt.ls (for some reason, jdt.ls is including the newline of the previous line when fixing indentation).
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2898)
<!-- Reviewable:end -->
[READY] Java support with asynchronous diagnostics and messages
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
This change is required for a better user experience when using native
java support
This implements an asynchronous message system using a long-poll request
to the server.
The server provides an endpoint /receive_messages which blocks until
either a timeout occurs or we receive a batch of asynchronous messages.
We send this request asynchronously and poll it 4 times a second to see
if we have received any messages.
The messages may either be simply for display (such as startup progress)
or diagnostics, which override the diagnostics returned by
OnFileReqdyToParse.
In the former case, we simply display the message, accepting that this
might be overwritten by any other message (indeed, requiring this), and
for the latter we fan out diagnostics to any open buffer for the file in
question.
Unfortunately, Vim has bugs related to timers when there is something
displayed (such as a "confirm" prompt or other), so we suspend
background timers when doing subcommands to avoid vim bugs. NOTE: This
requires a new version of Vim (detected by the presence of the
particular functions used).
NOT_READY because:
- the submodule commit points at my repo and requires https://github.com/Valloric/ycmd/pull/857 to be merged
- my spider sense suggest i have more testing to do...
Notes:
- Part 3 (I think) of the Java support PRs. This one actually adds the minimal changes for working java support
- There are about 2 or 3 other PRs to come to add things like automatic module imports, etc.
[Please explain **in detail** why the changes in this PR are needed.]
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2863)
<!-- Reviewable:end -->
[READY] Avoid traceback when computing extra conf data
Evaluating the values of [the `g:ycm_extra_conf_vim_data` option](https://github.com/Valloric/YouCompleteMe#the-gycm_extra_conf_vim_data-option) may raise a Python exception (e.g. one of the values is not defined). Since that option is evaluated each time a request is sent, such exception can render the editor almost unusable as each key press will print a Python traceback to the user.
A simple way to reproduce the issue is to add the following line in vimrc:
```viml
let g:ycm_extra_conf_vim_data = [ 'undefined_value' ]
```
We fix that by catching and logging the exception.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2899)
<!-- Reviewable:end -->
This implements an asynchronous message system using a long-poll request
to the server.
The server provides an endpoint /receive_messages which blocks until
either a timeout occurs or we receive a batch of asynchronous messages.
We send this request asynchronously and poll it 4 times a second to see
if we have received any messages.
The messages may either be simply for display (such as startup progress)
or diagnostics, which override the diagnostics returned by
OnFileReqdyToParse.
In the former case, we simply display the message, accepting that this
might be overwritten by any other message (indeed, requiring this), and
for the latter we fan out diagnostics to any open buffer for the file in
question.
Unfortunately, Vim has bugs related to timers when there is something
displayed (such as a "confirm" prompt or other), so we suspend
background timers when doing subcommands to avoid vim bugs. NOTE: This
requires a new version of Vim (detected by the presence of the
particular functions used).
Evaluating the values of g:ycm_extra_conf_vim_data may raise a Python exception
(e.g. one of the values is not defined). Since that option is parsed each time
a request is sent, such exception makes the editor almost unusable as each key
press is printing a Python traceback to the user. Catch and log the exception.
