vim/YouCompleteMe
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Auto merge of #2517 - micbou:remove-healthy-check, r=Valloric

Browse Source 
[READY] Remove healthy check

This PR removes the code that, if the server is not yet healthy, tries in a separate thread to send the same request at different intervals until it gets a response or reaches 5 failed attempts.

Main issue with that code is that it blocks Vim if a synchronous request (`semantic_completion_available`, `run_completer_command`, etc.)  is sent while the server is not up. See issue #2071.

Other issues are that it's hard to understand how this code behaves (it combines threads and retries), it becomes useless once the server is healthy (because `SERVER_HEALTHY` is cached and never reset), it depends on arbitrary parameters (why 5 retries with a delay of 0.5s multiplied by 1.5 at each attempt?), and it has a package dependency `retries`.

Now that we catch all server exceptions (PR #2453) and define a very short connection timeout (PR #2514), we can just drop this code.

This change gives a huge improvement in startup time on Windows (don't worry, more improvements are coming for all platforms). The reason is that `requests` (probably `urllib3`) tries for ~1s to check if the server is healthy on Windows while it's almost instantaneous on other platforms.
<table>
  <tr>
    <th rowspan="2">Platform</th>
    <th colspan="2">First run (ms)</th>
    <th colspan="2">Subsequent runs (ms)</th>
  </tr>
  <tr>
    <td>Before</td>
    <td>After</td>
    <td>Before</td>
    <td>After</td>
  </tr>
  <tr>
    <td>Ubuntu 16.04 64-bit</td>
    <td>228</td>
    <td>248</td>
    <td>176</td>
    <td>151</td>
  </tr>
  <tr>
    <td>macOS 10.12</td>
    <td>433</td>
    <td>417</td>
    <td>263</td>
    <td>263</td>
  </tr>
  <tr>
    <td>Windows 10 64-bit</td>
    <td>1861</td>
    <td>844</td>
    <td>814</td>
    <td>292</td>
  </tr>
</table>

These results were obtained by running the `prof.py` script from [this branch](https://github.com/micbou/YouCompleteMe/tree/profiling-startup). The difference between first run and subsequent runs is Python bytecode generation (`*.pyc` files).

Fixes #2071.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2517)
<!-- Reviewable:end -->
This commit is contained in:
Homu 2017-01-29 08:53:07 +09:00
commit f6867340dd
2 changed files with 14 additions and 142 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

77
python/ycm/client/base_request.py
View File

@ -30,7 +30,6 @@ import urllib.parse
import json
from future.utils import native
from base64 import b64decode, b64encode
from retries import retries
from requests_futures.sessions import FuturesSession
from ycm.unsafe_thread_pool_executor import UnsafeThreadPoolExecutor
from ycm import vimsupport
@ -102,43 +101,20 @@ class BaseRequest( object ):
handler,
method,
timeout = _READ_TIMEOUT_SEC ):
def SendRequest( data, handler, method, timeout ):
request_uri = _BuildUri( handler )
if method == 'POST':
sent_data = _ToUtf8Json( data )
return BaseRequest.session.post(
request_uri,
data = sent_data,
headers = BaseRequest._ExtraHeaders( method,
request_uri,
sent_data ),
timeout = ( _CONNECT_TIMEOUT_SEC, timeout ) )
if method == 'GET':
return BaseRequest.session.get(
request_uri,
headers = BaseRequest._ExtraHeaders( method, request_uri ),
timeout = ( _CONNECT_TIMEOUT_SEC, timeout ) )
@retries( 5, delay = 0.5, backoff = 1.5 )
def DelayedSendRequest( data, handler, method ):
request_uri = _BuildUri( handler )
if method == 'POST':
sent_data = _ToUtf8Json( data )
return requests.post(
request_uri,
data = sent_data,
headers = BaseRequest._ExtraHeaders( method,
request_uri,
sent_data ) )
if method == 'GET':
return requests.get(
request_uri,
headers = BaseRequest._ExtraHeaders( method, request_uri ) )
if not _CheckServerIsHealthyWithCache():
return _EXECUTOR.submit( DelayedSendRequest, data, handler, method )
return SendRequest( data, handler, method, timeout )
request_uri = _BuildUri( handler )
if method == 'POST':
sent_data = _ToUtf8Json( data )
return BaseRequest.session.post(
request_uri,
data = sent_data,
headers = BaseRequest._ExtraHeaders( method,
request_uri,
sent_data ),
timeout = ( _CONNECT_TIMEOUT_SEC, timeout ) )
return BaseRequest.session.get(
request_uri,
headers = BaseRequest._ExtraHeaders( method, request_uri ),
timeout = ( _CONNECT_TIMEOUT_SEC, timeout ) )
@staticmethod
@ -271,31 +247,6 @@ def _BuildUri( handler ):
handler ) ) )
SERVER_HEALTHY = False
def _CheckServerIsHealthyWithCache():
global SERVER_HEALTHY
def _ServerIsHealthy():
request_uri = _BuildUri( 'healthy' )
response = requests.get( request_uri,
headers = BaseRequest._ExtraHeaders(
'GET', request_uri, bytes( b'' ) ) )
_ValidateResponseObject( response )
response.raise_for_status()
return response.json()
if SERVER_HEALTHY:
return True
try:
SERVER_HEALTHY = _ServerIsHealthy()
return SERVER_HEALTHY
except:
return False
def MakeServerException( data ):
if data[ 'exception' ][ 'TYPE' ] == UnknownExtraConf.__name__:
return UnknownExtraConf( data[ 'exception' ][ 'extra_conf_file' ] )

79
third_party/retries/retries.py vendored
View File

@ -1,79 +0,0 @@
#!/usr/bin/env python
#
# Copyright 2012 by Jeff Laughlin Consulting LLC
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
import sys
from time import sleep
# Source: https://gist.github.com/n1ywb/2570004
def example_exc_handler(tries_remaining, exception, delay):
"""Example exception handler; prints a warning to stderr.
tries_remaining: The number of tries remaining.
exception: The exception instance which was raised.
"""
print >> sys.stderr, "Caught '%s', %d tries remaining, sleeping for %s seconds" % (exception, tries_remaining, delay)
def retries(max_tries, delay=1, backoff=2, exceptions=(Exception,), hook=None):
"""Function decorator implementing retrying logic.
delay: Sleep this many seconds * backoff * try number after failure
backoff: Multiply delay by this factor after each failure
exceptions: A tuple of exception classes; default (Exception,)
hook: A function with the signature myhook(tries_remaining, exception);
default None
The decorator will call the function up to max_tries times if it raises
an exception.
By default it catches instances of the Exception class and subclasses.
This will recover after all but the most fatal errors. You may specify a
custom tuple of exception classes with the 'exceptions' argument; the
function will only be retried if it raises one of the specified
exceptions.
Additionally you may specify a hook function which will be called prior
to retrying with the number of remaining tries and the exception instance;
see given example. This is primarily intended to give the opportunity to
log the failure. Hook is not called after failure if no retries remain.
"""
def dec(func):
def f2(*args, **kwargs):
mydelay = delay
tries = reversed(range(max_tries))
for tries_remaining in tries:
try:
return func(*args, **kwargs)
except exceptions as e:
if tries_remaining > 0:
if hook is not None:
hook(tries_remaining, e, mydelay)
sleep(mydelay)
mydelay = mydelay * backoff
else:
raise
else:
break
return f2
return dec