Use constants

Fix spelling errors in the README

.travis.yml

@@ -0,0 +1,8 @@
+dist: xenial   # required for Python >= 3.7
+language: python
+python:
+  - "3.7"
+install:
+  - pip install -r requirements.txt
+script:
+  - pytest

PLATFORM_IDs.md

@@ -79,4 +79,4 @@ Platform ID list for GOG Galaxy 2.0 Integrations
 | psvita	| Playstation Vita |
 | nds	| Nintendo DS |
 | 3ds	| Nintendo 3DS |
-
+| pathofexile | Path of Exile |

README.md

@@ -1,35 +1,35 @@
 # GOG Galaxy Integrations Python API
 
-This Python library allows to easily build community integrations for various gaming platforms with GOG Galaxy 2.0.
+This Python library allows developers to easily build community integrations for various gaming platforms with GOG Galaxy 2.0.
 
 - refer to our <a href='https://galaxy-integrations-python-api.readthedocs.io'>documentation</a>
 
 ## Features
 
-Each integration in GOG Galaxy 2.0 comes as a separate Python script, and is launched as a separate process, that which needs to communicate with main instance of GOG Galaxy 2.0.
+Each integration in GOG Galaxy 2.0 comes as a separate Python script and is launched as a separate process that needs to communicate with the main instance of GOG Galaxy 2.0.
 
 The provided features are:
 
-- multistep authorisation using a browser built into GOG Galaxy 2.0
+- multistep authorization using a browser built into GOG Galaxy 2.0
 - support for GOG Galaxy 2.0 features:
-    - importing owned and detecting installed games
+  - importing owned and detecting installed games
-    - installing and launching games
+  - installing and launching games
-    - importing achievements and game time
+  - importing achievements and game time
-    - importing friends lists and statuses
+  - importing friends lists and statuses
-    - importing friends recomendations list
+  - importing friends recommendations list
-    - receiving and sending chat messages
+  - receiving and sending chat messages
 - cache storage
 
 ## Platform Id's
 
 Each integration can implement only one platform. Each integration must declare which platform it's integrating.
 
-[List of possible Platofrm IDs](PLATFORM_IDs.md)
+[List of possible Platform IDs](PLATFORM_IDs.md)
 
 ## Basic usage
 
-Eeach integration should inherit from the :class:`~galaxy.api.plugin.Plugin` class. Supported methods like :meth:`~galaxy.api.plugin.Plugin.get_owned_games` should be overwritten - they are called from the GOG Galaxy client in the appropriate times.
+Each integration should inherit from the :class:`~galaxy.api.plugin.Plugin` class. Supported methods like :meth:`~galaxy.api.plugin.Plugin.get_owned_games` should be overwritten - they are called from the GOG Galaxy client at the appropriate times.
-Each of those method can raise exceptions inherited from the :exc:`~galaxy.api.jsonrpc.ApplicationError`.
+Each of those methods can raise exceptions inherited from the :exc:`~galaxy.api.jsonrpc.ApplicationError`.
 Communication between an integration and the client is also possible with the use of notifications, for example: :meth:`~galaxy.api.plugin.Plugin.update_local_game_status`.
 
 ```python
@@ -61,11 +61,13 @@ if __name__ == "__main__":
 
 ## Deployment
 
-The client has a built-in Python 3.7 interpreter, so the integrations are delivered as python modules.
+The client has a built-in Python 3.7 interpreter, so integrations are delivered as Python modules.
-In order to be found by GOG Galaxy 2.0 an integration folder should be placed in [lookup directory](#deploy-location). Beside all the python files, the integration folder has to contain [manifest.json](#deploy-manifest) and all third-party dependencies. See an [examplary structure](#deploy-structure-example).
+In order to be found by GOG Galaxy 2.0 an integration folder should be placed in [lookup directory](#deploy-location). Beside all the Python files, the integration folder must contain [manifest.json](#deploy-manifest) and all third-party dependencies. See an [exemplary structure](#deploy-structure-example).
 
 ### Lookup directory
+
 <a name="deploy-location"></a>
+
 - Windows:
 
     `%localappdata%\GOG.com\Galaxy\plugins\installed`
@@ -75,8 +77,9 @@ In order to be found by GOG Galaxy 2.0 an integration folder should be placed in
     `~/Library/Application Support/GOG.com/Galaxy/plugins/installed`
 
 ### Manifest
-<a name="deploy-manifest"></a> 
+
-Obligatory JSON file to be placed in a integration folder.
+<a name="deploy-manifest"></a>
+Obligatory JSON file to be placed in an integration folder.
 
 ```json
 {
@@ -91,6 +94,7 @@ Obligatory JSON file to be placed in a integration folder.
     "script": "plugin.py"
 }
 ```
+
 | property      | description |
 |---------------|---|
 | `guid`        |   |
@@ -99,13 +103,15 @@ Obligatory JSON file to be placed in a integration folder.
 | `script`      | path of the entry point module, relative to the integration folder |
 
 ### Dependencies
-All third-party packages (packages not included in Python 3.7 standard library) should be deployed along with plugin files. Use the folowing command structure:
+
+All third-party packages (packages not included in the Python 3.7 standard library) should be deployed along with plugin files. Use the following command structure:
 
 ```pip install DEP --target DIR --implementation cp --python-version 37```
 
-For example plugin that uses *requests* has structure as follows:
+For example, a plugin that uses *requests* could have the following structure:
 
 <a name="deploy-structure-example"></a>
+
 ```bash
 installed
 └── my_integration

jenkins/release.groovy

@@ -1,14 +0,0 @@
-stage('Upload to github')
-{
-    node('ActiveClientMacosxBuilder') {
-        deleteDir()
-        checkout scm
-        withPythonEnv('/usr/local/bin/python3.7') {
-            withCredentials([string(credentialsId: 'github_goggalaxy', variable: 'GITHUB_TOKEN')]) {
-                sh 'pip install -r jenkins/requirements.txt'
-                def version = sh(returnStdout: true, script: 'python setup.py --version').trim()
-                sh "python jenkins/release.py $version"
-            }
-        }
-    }
-}

jenkins/release.py

@@ -1,26 +0,0 @@
-import os
-import sys
-from galaxy.github.exporter import transfer_repo
-
-GITHUB_USERNAME = "goggalaxy"
-GITHUB_EMAIL = "galaxy-sdk@gog.com"
-GITHUB_TOKEN = os.environ["GITHUB_TOKEN"]
-GITHUB_REPO_NAME = "galaxy-integrations-python-api"
-SOURCE_BRANCH = os.environ["GIT_REFSPEC"]
-
-GITLAB_USERNAME = "galaxy-client"
-GITLAB_REPO_NAME = "galaxy-plugin-api"
-
-def version_provider(_):
-    return sys.argv[1]
-
-gh_version = transfer_repo(
-    version_provider=version_provider,
-    source_repo_spec="git@gitlab.gog.com:{}/{}.git".format(GITLAB_USERNAME, GITLAB_REPO_NAME),
-    source_include_elements=["src", "docs", "tests", "requirements.txt", ".readthedocs.yml" ".gitignore", "*.md", "pytest.ini", "setup.py"],
-    source_branch=SOURCE_BRANCH,
-    dest_repo_spec="https://{}:{}@github.com/{}/{}.git".format(GITHUB_USERNAME, GITHUB_TOKEN, "gogcom", GITHUB_REPO_NAME),
-    dest_branch="master",
-    dest_user_email=GITHUB_EMAIL,
-    dest_user_name="GOG Galaxy SDK Team"
-)

jenkins/requirements.txt

@@ -1 +0,0 @@
-git+ssh://git@gitlab.gog.com/galaxy-client/github-exporter.git@v0.1

setup.py

@@ -2,7 +2,7 @@ from setuptools import setup, find_packages
 
 setup(
     name="galaxy.plugin.api",
-    version="0.37",
+    version="0.41",
     description="GOG Galaxy Integrations Python API",
     author='Galaxy team',
     author_email='galaxy@gog.com',

src/galaxy/api/consts.py

@@ -80,6 +80,7 @@ class Platform(Enum):
     PlayStationVita = "psvita"
     NintendoDs = "nds"
     Nintendo3Ds = "3ds"
+    PathOfExile = "pathofexile"
 
 class Feature(Enum):
     """Possible features that can be implemented by an integration.

src/galaxy/api/jsonrpc.py

@@ -5,6 +5,8 @@ import logging
 import inspect
 import json
 
+from galaxy.reader import StreamLineReader
+
 class JsonRpcError(Exception):
     def __init__(self, code, message, data=None):
         self.code = code
@@ -67,13 +69,12 @@ def anonymise_sensitive_params(params, sensitive_params):
 class Server():
     def __init__(self, reader, writer, encoder=json.JSONEncoder()):
         self._active = True
-        self._reader = reader
+        self._reader = StreamLineReader(reader)
         self._writer = writer
         self._encoder = encoder
         self._methods = {}
         self._notifications = {}
         self._eof_listeners = []
-        self._input_buffer = bytes()
 
     def register_method(self, name, callback, internal, sensitive_params=False):
         """
@@ -105,7 +106,7 @@ class Server():
     async def run(self):
         while self._active:
             try:
-                data = await self._readline()
+                data = await self._reader.readline()
                 if not data:
                     self._eof()
                     continue
@@ -115,21 +116,7 @@ class Server():
             data = data.strip()
             logging.debug("Received %d bytes of data", len(data))
             self._handle_input(data)
-
+            await asyncio.sleep(0) # To not starve task queue
-    async def _readline(self):
-        """Like StreamReader.readline but without limit"""
-        while True:
-            chunk = await self._reader.read(1024)
-            if not chunk:
-                return chunk
-            previous_size = len(self._input_buffer)
-            self._input_buffer += chunk
-            it = self._input_buffer.find(b"\n", previous_size)
-            if it < 0:
-                continue
-            line = self._input_buffer[:it]
-            self._input_buffer = self._input_buffer[it+1:]
-            return line
 
     def stop(self):
         self._active = False

src/galaxy/api/plugin.py

@@ -38,6 +38,7 @@ class Plugin:
 
         self._feature_methods = OrderedDict()
         self._active = True
+        self._pass_control_task = None
 
         self._reader, self._writer = reader, writer
         self._handshake_token = handshake_token
@@ -210,15 +211,17 @@ class Plugin:
 
     async def run(self):
         """Plugin's main coroutine."""
-        async def pass_control():
+        await self._server.run()
-            while self._active:
+        if self._pass_control_task is not None:
-                try:
+            await self._pass_control_task
-                    self.tick()
-                except Exception:
-                    logging.exception("Unexpected exception raised in plugin tick")
-                await asyncio.sleep(1)
 
-        await asyncio.gather(pass_control(), self._server.run())
+    async def _pass_control(self):
+        while self._active:
+            try:
+                self.tick()
+            except Exception:
+                logging.exception("Unexpected exception raised in plugin tick")
+            await asyncio.sleep(1)
 
     def _shutdown(self):
         logging.info("Shutting down")
@@ -236,6 +239,7 @@ class Plugin:
     def _initialize_cache(self, data: Dict):
         self._persistent_cache = data
         self.handshake_complete()
+        self._pass_control_task = asyncio.create_task(self._pass_control())
 
     @staticmethod
     def _ping():
@@ -264,6 +268,7 @@ class Plugin:
                 return Authentication(user_data['userId'], user_data['username'])
 
          """
+        self.persistent_cache['credentials'] = credentials
         self._notification_client.notify("store_credentials", credentials, sensitive_params=True)
 
     def add_game(self, game: Game) -> None:

src/galaxy/api/types.py

@@ -204,5 +204,5 @@ class GameTime():
     :param last_time_played: last time the game was played (**unix timestamp**)
     """
     game_id: str
-    time_played: int
+    time_played: Optional[int]
-    last_played_time: int
+    last_played_time: Optional[int]

src/galaxy/http.py

@@ -1,5 +1,6 @@
 import asyncio
 import ssl
+from contextlib import contextmanager
 from http import HTTPStatus
 
 import aiohttp
@@ -12,44 +13,69 @@ from galaxy.api.errors import (
 )
 
 
+DEFAULT_LIMIT = 20
+DEFAULT_TIMEOUT = 60  # seconds
+
+
 class HttpClient:
-    def __init__(self, limit=20, timeout=aiohttp.ClientTimeout(total=60), cookie_jar=None):
+    """Deprecated"""
-        ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
+    def __init__(self, limit=DEFAULT_LIMIT, timeout=aiohttp.ClientTimeout(total=DEFAULT_TIMEOUT), cookie_jar=None):
-        ssl_context.load_verify_locations(certifi.where())
+        connector = create_tcp_connector(limit=limit)
-        connector = aiohttp.TCPConnector(limit=limit, ssl=ssl_context)
+        self._session = create_client_session(connector=connector, timeout=timeout, cookie_jar=cookie_jar)
-        self._session = aiohttp.ClientSession(connector=connector, timeout=timeout, cookie_jar=cookie_jar)
 
     async def close(self):
         await self._session.close()
 
     async def request(self, method, url, *args, **kwargs):
-        try:
+        with handle_exception():
-            response = await self._session.request(method, url, *args, **kwargs)
+            return await self._session.request(method, url, *args, **kwargs)
-        except asyncio.TimeoutError:
+
-            raise BackendTimeout()
+
-        except aiohttp.ServerDisconnectedError:
+def create_tcp_connector(*args, **kwargs):
-            raise BackendNotAvailable()
+    ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
-        except aiohttp.ClientConnectionError:
+    ssl_context.load_verify_locations(certifi.where())
-            raise NetworkError()
+    kwargs.setdefault("ssl", ssl_context)
-        except aiohttp.ContentTypeError:
+    kwargs.setdefault("limit", DEFAULT_LIMIT)
-            raise UnknownBackendResponse()
+    return aiohttp.TCPConnector(*args, **kwargs)
-        except aiohttp.ClientError:
+
-            logging.exception(
+
-                "Caught exception while running {} request for {}".format(method, url))
+def create_client_session(*args, **kwargs):
-            raise UnknownError()
+    kwargs.setdefault("connector", create_tcp_connector())
-        if response.status == HTTPStatus.UNAUTHORIZED:
+    kwargs.setdefault("timeout", aiohttp.ClientTimeout(total=DEFAULT_TIMEOUT))
-            raise AuthenticationRequired()
+    kwargs.setdefault("raise_for_status", True)
-        if response.status == HTTPStatus.FORBIDDEN:
+    return aiohttp.ClientSession(*args, **kwargs)
-            raise AccessDenied()
+
-        if response.status == HTTPStatus.SERVICE_UNAVAILABLE:
+
-            raise BackendNotAvailable()
+@contextmanager
-        if response.status == HTTPStatus.TOO_MANY_REQUESTS:
+def handle_exception():
-            raise TooManyRequests()
+    try:
-        if response.status >= 500:
+        yield
-            raise BackendError()
+    except asyncio.TimeoutError:
-        if response.status >= 400:
+        raise BackendTimeout()
-            logging.warning(
+    except aiohttp.ServerDisconnectedError:
-                "Got status {} while running {} request for {}".format(response.status, method, url))
+        raise BackendNotAvailable()
-            raise UnknownError()
+    except aiohttp.ClientConnectionError:
+        raise NetworkError()
+    except aiohttp.ContentTypeError:
+        raise UnknownBackendResponse()
+    except aiohttp.ClientResponseError as error:
+        if error.status == HTTPStatus.UNAUTHORIZED:
+            raise AuthenticationRequired()
+        if error.status == HTTPStatus.FORBIDDEN:
+            raise AccessDenied()
+        if error.status == HTTPStatus.SERVICE_UNAVAILABLE:
+            raise BackendNotAvailable()
+        if error.status == HTTPStatus.TOO_MANY_REQUESTS:
+            raise TooManyRequests()
+        if error.status >= 500:
+            raise BackendError()
+        if error.status >= 400:
+            logging.warning(
+                "Got status %d while performing %s request for %s",
+                error.status, error.request_info.method, str(error.request_info.url)
+            )
+            raise UnknownError()
+    except aiohttp.ClientError:
+        logging.exception("Caught exception while performing request")
+        raise UnknownError()
 
-        return response

src/galaxy/reader.py

@@ -0,0 +1,28 @@
+from asyncio import StreamReader
+
+
+class StreamLineReader:
+    """Handles StreamReader readline without buffer limit"""
+    def __init__(self, reader: StreamReader):
+        self._reader = reader
+        self._buffer = bytes()
+        self._processed_buffer_it = 0
+
+    async def readline(self):
+        while True:
+            # check if there is no unprocessed data in the buffer
+            if not self._buffer or self._processed_buffer_it != 0:
+                chunk = await self._reader.read(1024)
+                if not chunk:
+                    return bytes() # EOF
+                self._buffer += chunk
+
+            it = self._buffer.find(b"\n", self._processed_buffer_it)
+            if it < 0:
+                self._processed_buffer_it = len(self._buffer)
+                continue
+
+            line = self._buffer[:it]
+            self._buffer = self._buffer[it+1:]
+            self._processed_buffer_it = 0
+            return line

tests/test_chunk_messages.py

@@ -12,6 +12,43 @@ def test_chunked_messages(plugin, read):
 
     message = json.dumps(request).encode() + b"\n"
     read.side_effect = [message[:5], message[5:], b""]
-    plugin.get_owned_games.return_value = None
     asyncio.run(plugin.run())
     plugin.install_game.assert_called_with(game_id="3")
+
+def test_joined_messages(plugin, read):
+    requests = [
+        {
+            "jsonrpc": "2.0",
+            "method": "install_game",
+            "params": {
+                "game_id": "3"
+            }
+        },
+        {
+            "jsonrpc": "2.0",
+            "method": "launch_game",
+            "params": {
+                "game_id": "3"
+            }
+        }
+    ]
+    data = b"".join([json.dumps(request).encode() + b"\n" for request in requests])
+
+    read.side_effect = [data, b""]
+    asyncio.run(plugin.run())
+    plugin.install_game.assert_called_with(game_id="3")
+    plugin.launch_game.assert_called_with(game_id="3")
+
+def test_not_finished(plugin, read):
+    request = {
+        "jsonrpc": "2.0",
+        "method": "install_game",
+        "params": {
+            "game_id": "3"
+        }
+    }
+
+    message = json.dumps(request).encode() # no new line
+    read.side_effect = [message, b""]
+    asyncio.run(plugin.run())
+    plugin.install_game.assert_not_called()

tests/test_game_times.py

@@ -16,7 +16,8 @@ def test_success(plugin, read, write):
     read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_game_times.coro.return_value = [
         GameTime("3", 60, 1549550504),
-        GameTime("5", 10, 1549550502)
+        GameTime("5", 10, None),
+        GameTime("7", None, 1549550502),
     ]
     asyncio.run(plugin.run())
     plugin.get_game_times.assert_called_with()
@@ -35,7 +36,10 @@ def test_success(plugin, read, write):
                 {
                     "game_id": "5",
                     "time_played": 10,
-                    "last_played_time": 1549550502
+                },
+                {
+                    "game_id": "7",
+                     "last_played_time": 1549550502
                 }
             ]
         }

tests/test_http.py

@@ -0,0 +1,37 @@
+import asyncio
+from http import HTTPStatus
+
+import aiohttp
+import pytest
+
+from galaxy.api.errors import (
+    AccessDenied, AuthenticationRequired, BackendTimeout, BackendNotAvailable, BackendError, NetworkError,
+    TooManyRequests, UnknownBackendResponse, UnknownError
+)
+from galaxy.http import handle_exception
+
+request_info = aiohttp.RequestInfo("http://o.pl", "GET", {})
+
+@pytest.mark.parametrize(
+    "aiohttp_exception,expected_exception_type",
+    [
+        (asyncio.TimeoutError(), BackendTimeout),
+        (aiohttp.ServerDisconnectedError(), BackendNotAvailable),
+        (aiohttp.ClientConnectionError(), NetworkError),
+        (aiohttp.ContentTypeError(request_info, []), UnknownBackendResponse),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.UNAUTHORIZED), AuthenticationRequired),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.FORBIDDEN), AccessDenied),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.SERVICE_UNAVAILABLE), BackendNotAvailable),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.TOO_MANY_REQUESTS), TooManyRequests),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.INTERNAL_SERVER_ERROR), BackendError),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.NOT_IMPLEMENTED), BackendError),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.BAD_REQUEST), UnknownError),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.NOT_FOUND), UnknownError),
+        (aiohttp.ClientError(), UnknownError)
+    ]
+)
+def test_handle_exception(aiohttp_exception, expected_exception_type):
+    with pytest.raises(expected_exception_type):
+        with handle_exception():
+            raise aiohttp_exception
+

tests/test_internal.py

@@ -62,7 +62,18 @@ def test_ping(plugin, read, write):
         "result": None
     }
 
-def test_tick(plugin, read):
+def test_tick_before_handshake(plugin, read):
     read.side_effect = [b""]
     asyncio.run(plugin.run())
+    plugin.tick.assert_not_called()
+
+def test_tick_after_handshake(plugin, read):
+    request = {
+        "jsonrpc": "2.0",
+        "id": "6",
+        "method": "initialize_cache",
+        "params": {"data": {}}
+    }
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
+    asyncio.run(plugin.run())
     plugin.tick.assert_called_with()

tests/test_stream_line_reader.py

@@ -0,0 +1,52 @@
+from unittest.mock import MagicMock
+
+import pytest
+
+from galaxy.reader import StreamLineReader
+from galaxy.unittest.mock import AsyncMock
+
+@pytest.fixture()
+def stream_reader():
+    reader = MagicMock()
+    reader.read = AsyncMock()
+    return reader
+
+@pytest.fixture()
+def read(stream_reader):
+    return stream_reader.read
+
+@pytest.fixture()
+def reader(stream_reader):
+    return StreamLineReader(stream_reader)
+
+@pytest.mark.asyncio
+async def test_message(reader, read):
+    read.return_value = b"a\n"
+    assert await reader.readline() == b"a"
+    read.assert_called_once()
+
+@pytest.mark.asyncio
+async def test_separate_messages(reader, read):
+    read.side_effect = [b"a\n", b"b\n"]
+    assert await reader.readline() == b"a"
+    assert await reader.readline() == b"b"
+    assert read.call_count == 2
+
+@pytest.mark.asyncio
+async def test_connected_messages(reader, read):
+    read.return_value = b"a\nb\n"
+    assert await reader.readline() == b"a"
+    assert await reader.readline() == b"b"
+    read.assert_called_once()
+
+@pytest.mark.asyncio
+async def test_cut_message(reader, read):
+    read.side_effect = [b"a", b"b\n"]
+    assert await reader.readline() == b"ab"
+    assert read.call_count == 2
+
+@pytest.mark.asyncio
+async def test_half_message(reader, read):
+    read.side_effect = [b"a", b""]
+    assert await reader.readline() == b""
+    assert read.call_count == 2