[READY] Refactor server exception handling
Having to wrap all request calls with the `HandleServerException` function is inconvenient. Handle server exceptions directly in `BaseRequest`.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2976)
<!-- Reviewable:end -->
Allow users to still trigger Vim's omnifunc through C-Space when the
g:ycm_filetype_specific_completion_to_disable option is set for the current
filetype.
When sending a request to the server, we add a newline to the buffer contents
to match what gets saved to disk. If the server generates a chunk containing
that newline, this chunk goes past the Vim buffer contents since there is
actually no new line, which raises an IndexError exception when applying the
chunk. We fix the issue by recomputing the end position of where the chunk is
applied and by removing all trailing characters in the chunk.
[READY] Improve diagnostic matches display
There are two issues with how we display diagnostic matches. The first issue is that if the current buffer contains diagnostic matches and is split in a new window, all the matches are cleared and only matches in the new window are shown. The second issue is that if a new buffer with no diagnostic support is open in the current window and that window already contained matches then the matches are still displayed. Here's an illustration of both issues (signs are disabled):
The solution is to add an autocommand that updates matches on the `BufEnter` and `WinEnter` events. Here's the result:
Note that it's not perfect as multiple windows of the same buffer won't be updated simultaneously. Supporting that scenario is rather tricky because we would need to go through all the windows to update the matches and switching windows can lead to a lot of issues (like interrupting visual mode) so we don't.
This PR also improves how we update matches by only displaying matches that are not already present and then clearing the remaining ones (similarly to what we do with signs; see PR https://github.com/Valloric/YouCompleteMe/pull/2915) instead of always clearing all the matches then displaying the new ones.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2922)
<!-- Reviewable:end -->
Java completer can include FixIts which are applied when a completion
entry is selected. We use the existing mechanism implemented for c-sharp
to perform these edits using the CompleteDone autocommand.
However, the existing mechanism relies on pattern matching the source to
work out which item was completed. Vim patch 8.0.1493 introduces support
for user_data on completion items, so when available we populate it with
the completion array index of the item and use that to get the exact
element that was selected. This is both a lot faster and a lot more
accirate.
Of course when applying these 'FixIts' we don't interrupt the user with
confirmation or the quickfix list as this would just be annoying. If the
server reports that an edit must be made, we just make the edit. This is
achieved by adding a silent flag to ReplaceChunks.
[READY] Apply chunks from bottom to top
Without the proposed fix, the test included in that PR fails as follows
```
FAIL: ycm.tests.vimsupport_test.ReplaceChunksInBuffer_LineOverlappingChunks_test
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Python36\lib\site-packages\nose\case.py", line 198, in runTest
self.test(*self.arg)
File "C:\Users\micbou\projects\YouCompleteMe\python\ycm\tests\vimsupport_test.py", line 768, in ReplaceChunksInBuffer_LineOverlappingChunks_test
AssertBuffersAreEqualAsBytes( expected_buffer, result_buffer )
File "C:\Users\micbou\projects\YouCompleteMe\python\ycm\tests\vimsupport_test.py", line 86, in AssertBuffersAreEqualAsBytes
eq_( ToBytes( result_line ), ToBytes( expected_line ) )
AssertionError: b' third line' != b' third '
```
Found the issue while trying to add code formatting support to the language server completer. The test is based on a real-world scenario when formatting with jdt.ls (for some reason, jdt.ls is including the newline of the previous line when fixing indentation).
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2898)
<!-- Reviewable:end -->
[READY] Java support with asynchronous diagnostics and messages
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
This change is required for a better user experience when using native
java support
This implements an asynchronous message system using a long-poll request
to the server.
The server provides an endpoint /receive_messages which blocks until
either a timeout occurs or we receive a batch of asynchronous messages.
We send this request asynchronously and poll it 4 times a second to see
if we have received any messages.
The messages may either be simply for display (such as startup progress)
or diagnostics, which override the diagnostics returned by
OnFileReqdyToParse.
In the former case, we simply display the message, accepting that this
might be overwritten by any other message (indeed, requiring this), and
for the latter we fan out diagnostics to any open buffer for the file in
question.
Unfortunately, Vim has bugs related to timers when there is something
displayed (such as a "confirm" prompt or other), so we suspend
background timers when doing subcommands to avoid vim bugs. NOTE: This
requires a new version of Vim (detected by the presence of the
particular functions used).
NOT_READY because:
- the submodule commit points at my repo and requires https://github.com/Valloric/ycmd/pull/857 to be merged
- my spider sense suggest i have more testing to do...
Notes:
- Part 3 (I think) of the Java support PRs. This one actually adds the minimal changes for working java support
- There are about 2 or 3 other PRs to come to add things like automatic module imports, etc.
[Please explain **in detail** why the changes in this PR are needed.]
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2863)
<!-- Reviewable:end -->
This implements an asynchronous message system using a long-poll request
to the server.
The server provides an endpoint /receive_messages which blocks until
either a timeout occurs or we receive a batch of asynchronous messages.
We send this request asynchronously and poll it 4 times a second to see
if we have received any messages.
The messages may either be simply for display (such as startup progress)
or diagnostics, which override the diagnostics returned by
OnFileReqdyToParse.
In the former case, we simply display the message, accepting that this
might be overwritten by any other message (indeed, requiring this), and
for the latter we fan out diagnostics to any open buffer for the file in
question.
Unfortunately, Vim has bugs related to timers when there is something
displayed (such as a "confirm" prompt or other), so we suspend
background timers when doing subcommands to avoid vim bugs. NOTE: This
requires a new version of Vim (detected by the presence of the
particular functions used).
Evaluating the values of g:ycm_extra_conf_vim_data may raise a Python exception
(e.g. one of the values is not defined). Since that option is parsed each time
a request is sent, such exception makes the editor almost unusable as each key
press is printing a Python traceback to the user. Catch and log the exception.
[READY] Drop Python 2.6 and Python 3.3
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
[Please explain **in detail** why the changes in this PR are needed.]
Since we are dropping support for Python 2.6 and 3.3 in Valloric/ycmd#904 we should drop support for them in YCM as well.
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2887)
<!-- Reviewable:end -->
When the message is exactly as wide as the vim window, vim still
displays "press enter..." message.
Wen we want to avoid that we need to cut the message one character
shorter.
Reduce the time spent to build the request when there are a lot of buffers by:
- using the options property on the buffer object to get the mod variable
instead of evaluating getbufvar;
- not computing the buffer filepath if the buffer is not modified;
- passing the number of the unloaded buffer instead of its filepath on the
BufferUnload event. Getting the Python buffer object from its number is
easier than from its filepath.
Now that the server supports Java natively, we need to use a
filetype that does not have native support to ensure that we call the
mocked omnifunc, but also one for which YCM has a semantic trigger.
Rather than picking another random languagse, which might break in the
future, we invent a language (ycmtest) and supply a manual trigger
mapping to the YouCompleteMe object.
[READY] Always supply working directory
This allows completer servers to detect the correct directory to launch
no matter what request initialises the completer server.
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [X] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [X] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [X] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [X] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
This change ensures that the client supplies the working directory to the server on each request, in particular, event notifications; the java and javascript completers use this to start the server in the `FileReadyToParse` event, rather than on construction of the completer. The subtle difference allows them to use the client's working directory, rather than the _server's_ working directory to do things like project detection.
See also:
* This is PR 1 of the set of changes related to #2827 - client changes required to support Java and other language-server protocols.
* ycmd change for javascript that makes use of this now: https://github.com/Valloric/ycmd/pull/886
* updated API doc: http://puremourning.github.io/ycmd-1
The ycmd API change is simply to allow `working_dir` on all requests.
cc for minor optional API change: @abingham @Qusic @LuckyGeck @mawww @richard1122 @jakeanq @orsonteodoro
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2861)
<!-- Reviewable:end -->
When no argument is given to the YcmToggleLogs command, instead of displaying
the list of available logfiles, prompt the user to open (or close if already
open) one of them.
On Windows, Vim returns buffer filepaths with forward slashes instead of
backward ones if the shellslash option is set. Since we always expect backward
slashes from the server, we normalize these paths.
The VimExpressionToPythonType function automatically convert a number
represented as a string to an integer. This causes an error when used to
evaluate a filetype set to a number as the result is split on the dot character
to get a list of filetypes and an integer cannot be split. Use vim.eval and
ToUnicode instead.
If no buffer exists for a given filename, the GetBufferNumberForFilename
function will create a buffer for that file by default. This behavior is
unexpected given the name of that function and may lead to performance issues
when ycmd returns diagnostics for a lot of files with no corresponding buffers.
The default behavior for that function should be to not create a buffer.
On Windows, the options file cannot be deleted while it is still open by
another process. Close the file before starting the server. Do not explicitly
flush the file as closing implies flushing.
[READY] Improve path to python interpreter error handling
YCM returns the cryptic error `YouCompleteMe unavailable: [Errno 2] No such file or directory` when the Python interpreter path used to start the server doesn't exist. See issues https://github.com/Valloric/YouCompleteMe/issues/2773 and https://github.com/Valloric/YouCompleteMe/issues/2775.
This PR improves that by using the `FindExecutable` and `GetExecutable` functions from ycmd to check if the Python interpreter path exists (and is an executable) and by returning a much more helpful message if it doesn't. We use `FindExecutable` instead of `GetExecutable` on the `g:ycm_server_python_interpreter` option so that the user can specify a Python path with just the executable name (e.g. `python`, `python2`, `python3`).
This also fixes a Python traceback in Vim that occurs when, after successfully starting the server, the `g:ycm_server_python_interpreter` option is set to an invalid path and the server is restarted with `:YcmRestartServer`. Steps to reproduce are:
- start Vim with a working YCM;
- type `:let g:ycm_server_python_interpreter = '/invalid/python/path'`;
- type `:YcmRestartServer`.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2787)
<!-- Reviewable:end -->
Calling directly the omnifunc may move the cursor position. This is the case
with the default Vim omnifunc for C-family languages (ccomplete#Complete) which
calls searchdecl to find a declaration. This function is supposed to move the
cursor to the found declaration but it doesn't when called through the omni
completion mapping (CTRL-X CTRL-O). So, we restore the cursor position after
calling the omnifunc.
[READY] Fix unicode warning when jumping on Python 2
On Python 2, jumping in a file whose path contains non-ASCII characters raises the following warnings:
```
Error detected while processing function <SNR>26_CompleterCommand:
line 18:
python\ycm\vimsupport.py:400: UnicodeWarning: Unicode unequal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
if filename != GetCurrentBufferFilepath():
python\ycm\vimsupport.py:375: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
if win.buffer.name == filepath:
```
This happens because, in the `JumpToLocation` function, we are comparing a unicode object (`filename` and `filepath`) with a Vim buffer name (`vim.current.buffer.name` returned by `GetCurrentBufferFilepath` and `win.buffer.name` where `win` is a Vim window) which is a byte object on Python 2.
For now, this PR adds tests covering the `JumpToLocation` function with unicode paths. They will raise the warnings on Python 2 and will fail because warnings are now treated as errors in tests. I'll update the PR with the fix once the builds are done.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2702)
<!-- Reviewable:end -->
The 'hidden' option is a global option, not a buffer one. If this option is
false, we should check if the 'bufhidden' option, which is local to the buffer,
is set to 'hide'. If so, the buffer can be hidden despite the 'hidden' option
being false.
Bring fully asynchronous completion by polling for completions with a timer
then calling completefunc once the completions are ready. Use the start column
returned by the server in completefunc. Immediately display the last completion
on the TextChangedI event to prevent the popup menu disappearing while waiting
for the completions. Handle the TextChangedI event not being triggered while
the completion menu is open by closing the menu when inserting a character
through the InsertCharPre event, and when deleting a character on the <BS> and
<C-h> keys.
[READY] Use codepoint offsets in identifier functions
`CurrentIdentifierFinished` and `LastEnteredCharIsIdentifierChar` incorrectly use byte offsets with unicode lines (`CurrentColumn` returns a byte offset and `CurrentLineContents` a unicode string). This leads to weird bugs when there is a non-ASCII character on the current line:
This is fixed by converting byte offsets to codepoint ones through the `ByteOffsetToCodepointOffset` function.
This changes the behavior of these two functions when the current column position is invalid. Both functions returned false in that case. They now return as if the current column were at the end of the line. In practice, this doesn't really matter since the position of the current column should always be valid.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2690)
<!-- Reviewable:end -->
CurrentIdentifierFinished and LastEnteredCharIsIdentifierChar incorrectly use
byte offsets with unicode lines. Convert those offsets to codepoint offsets.
[READY] Moved change tracking to python. Per buffer diagnostics
Thank you for working on YCM! :)
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
Adds an emulation of Vim buffer object and moves change tracking into python.
Now diagnostics are stored per buffer, which effectively solves #2165 .
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2631)
<!-- Reviewable:end -->
coverage.py 4.4 removed the path from the filename attribute in its reports.
This leads to incorrect coverage from codecov as it relies on this attribute to
find the source file. Require coverage.py older than version 4.4.
[READY] Fix diagnostic highlighting with invalid location extent
ycmd may return a location extent with a starting column of zero (see the test for an example) which is not a valid column (columns are 1-indexed). We should use the location in this case. Also, when the location extent is not valid, we should call `AddDiagnosticSyntaxMatch` with the `is_error` parameter to highlight warnings as such and not as errors (`is_error` is `True` by default).
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2602)
<!-- Reviewable:end -->
