When sending a request to the server, we add a newline to the buffer contents
to match what gets saved to disk. If the server generates a chunk containing
that newline, this chunk goes past the Vim buffer contents since there is
actually no new line, which raises an IndexError exception when applying the
chunk. We fix the issue by recomputing the end position of where the chunk is
applied and by removing all trailing characters in the chunk.
Java completer can include FixIts which are applied when a completion
entry is selected. We use the existing mechanism implemented for c-sharp
to perform these edits using the CompleteDone autocommand.
However, the existing mechanism relies on pattern matching the source to
work out which item was completed. Vim patch 8.0.1493 introduces support
for user_data on completion items, so when available we populate it with
the completion array index of the item and use that to get the exact
element that was selected. This is both a lot faster and a lot more
accirate.
Of course when applying these 'FixIts' we don't interrupt the user with
confirmation or the quickfix list as this would just be annoying. If the
server reports that an edit must be made, we just make the edit. This is
achieved by adding a silent flag to ReplaceChunks.
[READY] Apply chunks from bottom to top
Without the proposed fix, the test included in that PR fails as follows
```
FAIL: ycm.tests.vimsupport_test.ReplaceChunksInBuffer_LineOverlappingChunks_test
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Python36\lib\site-packages\nose\case.py", line 198, in runTest
self.test(*self.arg)
File "C:\Users\micbou\projects\YouCompleteMe\python\ycm\tests\vimsupport_test.py", line 768, in ReplaceChunksInBuffer_LineOverlappingChunks_test
AssertBuffersAreEqualAsBytes( expected_buffer, result_buffer )
File "C:\Users\micbou\projects\YouCompleteMe\python\ycm\tests\vimsupport_test.py", line 86, in AssertBuffersAreEqualAsBytes
eq_( ToBytes( result_line ), ToBytes( expected_line ) )
AssertionError: b' third line' != b' third '
```
Found the issue while trying to add code formatting support to the language server completer. The test is based on a real-world scenario when formatting with jdt.ls (for some reason, jdt.ls is including the newline of the previous line when fixing indentation).
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2898)
<!-- Reviewable:end -->
[READY] Java support with asynchronous diagnostics and messages
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
This change is required for a better user experience when using native
java support
This implements an asynchronous message system using a long-poll request
to the server.
The server provides an endpoint /receive_messages which blocks until
either a timeout occurs or we receive a batch of asynchronous messages.
We send this request asynchronously and poll it 4 times a second to see
if we have received any messages.
The messages may either be simply for display (such as startup progress)
or diagnostics, which override the diagnostics returned by
OnFileReqdyToParse.
In the former case, we simply display the message, accepting that this
might be overwritten by any other message (indeed, requiring this), and
for the latter we fan out diagnostics to any open buffer for the file in
question.
Unfortunately, Vim has bugs related to timers when there is something
displayed (such as a "confirm" prompt or other), so we suspend
background timers when doing subcommands to avoid vim bugs. NOTE: This
requires a new version of Vim (detected by the presence of the
particular functions used).
NOT_READY because:
- the submodule commit points at my repo and requires https://github.com/Valloric/ycmd/pull/857 to be merged
- my spider sense suggest i have more testing to do...
Notes:
- Part 3 (I think) of the Java support PRs. This one actually adds the minimal changes for working java support
- There are about 2 or 3 other PRs to come to add things like automatic module imports, etc.
[Please explain **in detail** why the changes in this PR are needed.]
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2863)
<!-- Reviewable:end -->
This implements an asynchronous message system using a long-poll request
to the server.
The server provides an endpoint /receive_messages which blocks until
either a timeout occurs or we receive a batch of asynchronous messages.
We send this request asynchronously and poll it 4 times a second to see
if we have received any messages.
The messages may either be simply for display (such as startup progress)
or diagnostics, which override the diagnostics returned by
OnFileReqdyToParse.
In the former case, we simply display the message, accepting that this
might be overwritten by any other message (indeed, requiring this), and
for the latter we fan out diagnostics to any open buffer for the file in
question.
Unfortunately, Vim has bugs related to timers when there is something
displayed (such as a "confirm" prompt or other), so we suspend
background timers when doing subcommands to avoid vim bugs. NOTE: This
requires a new version of Vim (detected by the presence of the
particular functions used).
Evaluating the values of g:ycm_extra_conf_vim_data may raise a Python exception
(e.g. one of the values is not defined). Since that option is parsed each time
a request is sent, such exception makes the editor almost unusable as each key
press is printing a Python traceback to the user. Catch and log the exception.
Reduce the time spent to build the request when there are a lot of buffers by:
- using the options property on the buffer object to get the mod variable
instead of evaluating getbufvar;
- not computing the buffer filepath if the buffer is not modified;
- passing the number of the unloaded buffer instead of its filepath on the
BufferUnload event. Getting the Python buffer object from its number is
easier than from its filepath.
The VimExpressionToPythonType function automatically convert a number
represented as a string to an integer. This causes an error when used to
evaluate a filetype set to a number as the result is split on the dot character
to get a list of filetypes and an integer cannot be split. Use vim.eval and
ToUnicode instead.
If no buffer exists for a given filename, the GetBufferNumberForFilename
function will create a buffer for that file by default. This behavior is
unexpected given the name of that function and may lead to performance issues
when ycmd returns diagnostics for a lot of files with no corresponding buffers.
The default behavior for that function should be to not create a buffer.
The 'hidden' option is a global option, not a buffer one. If this option is
false, we should check if the 'bufhidden' option, which is local to the buffer,
is set to 'hide'. If so, the buffer can be hidden despite the 'hidden' option
being false.
Send the request as the unloaded buffer instead of the current buffer
for the BufferUnload event notification. This fixes the issue where
the filetype of the current buffer is not the same as the unloaded
buffer one, making the ycmd server uses the wrong completer when
handling the request.
When an error occurs during completions, a message is displayed on
the status line. If this message is longer than the width of the
current window, Vim will prompt the user to press enter or type a
command to hide the message, interrupting user workflow. We prevent
that by truncating the message to window width.
Merge PostMultiLineNotice, EchoText, and EchoTextVimWidth functions
into PostVimMessage.
When columns are clamped to not be past the contents of the line for
highlighting diagnostics, we need to account for the column end not
being included in the diagnostic range.
Do not convert strings to bytes but instead use plain strings to mimic
Vim buffers returning a list of byte objects on Python 2 and unicode
objects on Python 3.
Open the quickfix window to full width at the bottom of the screen with
its height set to fit all entries. This behavior can be overridden by
using the YcmQuickFixOpened autocommand.
Add a new section for autocommands in the documentation.
Update GoTo and ReplaceChunks tests.
On Windows and Python 2, the full exception message from IOError
in CheckFilename will contain the filepath formatted as a unicode
string. Since the filepath is already added in the RuntimeError
message, use the strerror attribute to only display the error.
Python 3 is much stricter around mixing bytes with unicode (and by
"stricter," I mean it doesn't allow it at all) so we're making
vimsupport only return `unicode` objects (`str` on py3). The idea is
that YCM (and ycmd) internals only ever deal with unicode.
We simply apply the changes to each file in turn. The existing replacement
logic is unchanged, except that it now no longer implicitly assumes we are
talking about the current buffer.
If a buffer is not visible for the requested file name, we open it in
a horizontal split, make the edits, then hide the window. Because this
can cause UI flickering, and leave hidden, modified buffers around, we
issue a warning to the user stating the number of files for which we are
going to do this. We pop up the quickfix list at the end of applying
the edits to allow the user to see what we changed.
If the user opts to abort due to, say, the file being open in another
window, we simply raise an error and give up, as undoing the changes
is too complex to do programatically, but trivial to do manually in such
a rare case.
Mock buffers as a list of buffers where each buffer is represented
as a dictionary containing its filename, its number, and optionally
its associated window.
Test buffer visibility instead of existence.
Mock Vim wipeout command.
Refactor Vim mocks.
We display the detailed info text in the preview window. Vim's preview window is
designed to display actual files, not scratch data. Our approach is to open a
temporary file, even though that file is never written. This way, all of Vim's
existing settings for the preview window (and people's configured mappings) just
work. This is also consistent with showing the documentation in the preview
window during completion.
Other plugins have more complicated functions for this (such as eclim), or
Scratch.vim, but this approach is simple and doesn't require external
dependencies or additional settings.
Tests:
This required fixing a sort-of-bug in which the mock'd Vim module was always
only set once, and could not be changed outside of the module which created it.
This meant that it wasn't easy to have arbitrary tests, because it was dependent
on the order in which the tests execute as to whether the return from
MockVimModule() was actually the one in use.
The solution was to make the mock'd vim module a singleton, and use mock's
patch decorator to assign new MagicMock() instances to those methods in the vim
module which a particular test is interested in.
