[READY] Avoid evaluating Vim globals in Python
### Problem
See the commit message and issue #2127.
### How to reproduce
Make Vim to use Python 3 for YCM by either using Vim with only Python 3 support or by editing the `s:UsingPython2` function in `autoload/youcompleteme.vim` to always return 0.
Create the following `vimrc`:
```
set nocompatible
set runtimepath+=~/.vim/bundle/YouCompleteMe
set encoding=utf8
filetype plugin indent on
let g:dummy_variable = '€'[0]
```
and start Vim with it. The following error will occur:
```
YouCompleteMe unavailable: 'utf-8' codec can't decode byte 0xe2 in position 0: unexpected end of data
```
with the traceback in `:messages`:
```python
Traceback (most recent call last):
File "<string>", line 24, in <module>
File "C:\\Users\\micbou\\.vim\\bundle\\YouCompleteMe\\autoload\..\python\ycm\setup.py", line 49, in SetUpYCM
base.LoadJsonDefaultsIntoVim()
File "C:\\Users\\micbou\\.vim\\bundle\\YouCompleteMe\\autoload\..\python\ycm\base.py", line 60, in LoadJsonDefaultsIntoVim
vimsupport.LoadDictIntoVimGlobals( vim_defaults, overwrite = False )
File "C:\\Users\\micbou\\.vim\\bundle\\YouCompleteMe\\autoload\..\python\ycm\vimsupport.py", line 305, in LoadDictIntoVimGlobals
extend_option ) )
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe2 in position 0: unexpected end of data
```
### Solution
Do not evaluate the Vim globals when loading the YCM default options into Vim and when building the options for the ycmd server.
Depending on the number of global variables and custom YCM options, this may be slower or faster than the current code but by a negligible margin (~1ms).
Fixes#2127 and #2150.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2151)
<!-- Reviewable:end -->
On Python 3, evaluating a Vim expression will raise a unicode exception
if it contains an invalid sequence of bytes for the current encoding.
We can't really do anything about it because this is the way Vim and
Python 3 interact. However, we can prevent this situation to occur by
not evaluating Vim data that we have no control over: in particular,
the Vim globals. This is done by:
- adding one by one the YCM default options instead of extending the
Vim globals with them;
- only evaluating the Vim global variable names (and not their values)
when building the YCM options for the ycmd server.
Display an error message to the user depending on the status code
returned by the ycmd server.
Remove ycm_core checks in plugin/youcompleteme.vim. These checks are
now done by the ycmd server.
Do not start a separate process to check the core version but rely on
ycmd returning a specific exit code. This slightly improves the Vim
startup time.
[READY] Fix PostComplete test
A refactoring of PostComplete tests made the `GetCompletionsUserMayHaveCompleted_ReturnEmptyIfPendingMatches_OldVim` test identical to
`GetCompletionsUserMayHaveCompleted_ReturnEmptyIfPendingMatches_NewVim`. Revert to original test.
This PR also updates the tests with small changes. See the commit message.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2097)
<!-- Reviewable:end -->
Vim recently has introduced a new feature called 'packages', which
permits, among the rest, to disable automatic loading a plugin at
startup. Plugins that are disabled at startup may be manually loaded
during a session using :packadd (see :h packages, :h :packadd). Of
course, a plugin that is loaded after startup cannot rely on VimEnter
events.
In order to support lazy loading YCM, check whether YCM is sourced
during startup or at a later time. In the former case, define an
autocommand as before; in the latter case, enable YCM immediately.
Add how to build python to avoid startup problems (in particular, pyenv)
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [X] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [X] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [X] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [X] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
I spent a while trying to find where I had written the instructions for building python with pyenv to work with YCM, then I thought the README was a good place for it.
ycmd must use pythons with `--enable-shared` or `--enable-framework` else boom.
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2083)
<!-- Reviewable:end -->
[READY] Rename ycmd python option
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
[Please explain **in detail** why the changes in this PR are needed.]
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md
As requested in https://github.com/Valloric/YouCompleteMe/issues/2052#issuecomment-195631106 I'm sending this PR. I've marked it as WIP because I've renamed only the `ycmd` option and that is because keeping the backward compatibility here was straight forward since is a client only option. For the jedihttp option instead I would like some opinion on how to procede: should we keep the backward compatibility here on the client or should we make the backward compatibility layer on the ycmd side? Keeping it here would be easier but it would mean that other clients should implement this theirself. Thoughts?
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="35" align="absmiddle" alt="Review on Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2054)
<!-- Reviewable:end -->
[READY] Update Windows installation in README
This PR updates the Windows documentation by mentioning both Python versions but recommending Python 3. It also changes the link to the 64-bit Vim by a link to [my own builds](https://bintray.com/micbou/generic/vim/view): they are updated more frequently, are available in both architectures, provide almost all Vim features, and are automatically tested before being released.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2074)
<!-- Reviewable:end -->
[READY] Fix traceback when syntax files contain unicode characters
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [X] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [X] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [ ] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [X] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
Fixes https://github.com/Valloric/YouCompleteMe/issues/2036
I'm submitting this without tests for now because it's probably quite important, and adding tests is going to be quite time consuming because we somehow need to mock out the output of `:syntax list` with unicode chars, etc.
I have tested manually and got people experiencing the issue to also confirm that it fixes it.
I've also surveyed other uses of `vim.eval` that return string and wrapped them in a `ToUnicode` where there is a possibility of it not working. This is much more speculative, so feel free to say ditch this and just fix the issue at hand.
Also, of course feel free to say we can't merge this without tests, which is also valid :)
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2037)
<!-- Reviewable:end -->
vim.eval returns a str() object on py2, but our internal strings are all unicode().
We use vimsupport.VimExpressionToPythonType to wrap the conversion complexities.
Update tern_runtime directory in README
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [ ] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
Was following the guides and saw this referencing a wrong path so I decided to fix it with the proper one.
Should be useful sine this will keep the docs up-to-date!
[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.mdhttps://reviewable.io/reviews/valloric/youcompleteme/2047
[READY] Do not send request in OnFileReadyToParse function if server is down
Even if `ycmd` process is terminated, YCM will try to send the `event_notification` request in `OnFileReadyToParse` function, blocking Vim for one second if the server crashed at the start (because of [this request](https://github.com/Valloric/YouCompleteMe/blob/master/python/ycm/client/base_request.py#L222)) or resulting in a traceback with `ConnectionError` exception otherwise. This will happen for the `InsertLeave`, `CursorMoved`, `CursorHold`, `BufferVisit` events, making Vim almost unusable.
This is fixed by returning early if the process is terminated.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="35" align="absmiddle" alt="Review on Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2041)
<!-- Reviewable:end -->
Do not send an "event_notification" request in OnFileReadyToParse
function if server process is terminated. Otherwise, it blocks Vim
for one second or results in a traceback each time the InsertLeave,
CursorMoved, CursorHold, and BufferVisit events are triggered.
[READY] Fix multiline title entry in FAQ
Apparently, it is not possible to write a header on multiple lines using the `#` syntax in markdown, so we write it on one line.
Update the vim docs too.
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2035)
<!-- Reviewable:end -->
[READY] Update flake8 config
# PR Prelude
Thank you for working on YCM! :)
**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**
- [X] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [X] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [X] I have included tests for the changes in my PR. If not, I have included a
rationale for why I haven't.
- [X] **I understand my PR may be closed if it becomes obvious I didn't
actually perform all of these steps.**
# Why this change is necessary and useful
Python is a dynamic language and as a result shares the dynamic language mentality of "Fail Late". Admittedly, Python fails earlier than most dynamic languages, but in any case it is extremely useful to have static analysis and syntax checking tools available prior to runtime execution.
YCM uses flake8 for this as part of its test runs. However, due to code _style_ being more important to most of the checkers than code _quality_, we have not always taken full advantage (for example, syntax checking was disabled). This PR continues https://github.com/Valloric/ycmd/pull/416 and adds the same set of checks to YCM client.
This benefits developers because it speeds up the code/test cycle (using Syntactic now requires _no_ additional configuration for flake8 to be correct for YCM and ycmd), and benefits users, because we should produce better quality code :)
This PR also updates to the latest ycmd to include https://github.com/Valloric/ycmd/pull/416
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2031)
<!-- Reviewable:end -->
[READY] Catch ReadTimeout exception on requests
In PR #2016, we have changed the way we are catching exceptions on requests by only catching the `ServerError` instead of all exceptions, which is great for debugging purposes. However, due to this change, we don't catch the exception from a timed out request, leading to a ugly traceback in Vim. This PR fixes that.
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2029)
<!-- Reviewable:end -->
