Use constants

Fix spelling errors in the README

.gitignore

@@ -1,2 +1,9 @@
 # pytest
 __pycache__/
+.vscode/
+.venv/
+src/galaxy.plugin.api.egg-info/
+docs/build/
+Pipfile
+.idea
+docs/source/_build

.gitlab-ci.yml

@@ -0,0 +1,26 @@
+image: registry-gitlab.gog.com/galaxy-client/gitlab-ci-tools:latest
+
+stages:
+  - test
+  - deploy
+
+test_package:
+  stage: test
+  script:
+    - pip install -r requirements.txt
+    - pytest
+  except:
+    - tags
+
+deploy_package:
+  stage: deploy
+  script:
+    - export VERSION=$(python setup.py --version)
+    - python setup.py sdist --formats=gztar upload -r gog-pypi
+    - curl -X POST --silent --show-error --fail
+        "https://gitlab.gog.com/api/v4/projects/${CI_PROJECT_ID}/repository/tags?tag_name=${VERSION}&ref=${CI_COMMIT_REF_NAME}&private_token=${PACKAGE_DEPLOYER_API_TOKEN}"
+  when: manual
+  only:
+    - master
+  except:
+    - tags

.travis.yml

@@ -0,0 +1,8 @@
+dist: xenial   # required for Python >= 3.7
+language: python
+python:
+  - "3.7"
+install:
+  - pip install -r requirements.txt
+script:
+  - pytest

PLATFORM_IDs.md

@@ -0,0 +1,82 @@
+### PLATFORM ID LIST
+
+Platform ID list for GOG Galaxy 2.0 Integrations
+
+| ID | Name |
+| --- | --- |
+| steam	| Steam |
+| psn	| PlayStation Network |
+| xboxone	| Xbox Live |
+| generic	| Manually added games |
+| origin	| Origin |
+| uplay	| Uplay |
+| battlenet	| Battle.net |
+| epic	| Epic Games Store |
+| bethesda	| Bethesda.net |
+| paradox	| Paradox Plaza |
+| humble	| Humble Bundle |
+| kartridge	| Kartridge |
+| itch	| Itch.io |
+| nswitch	| Nintendo Switch |
+| nwiiu	| Nintendo Wii U |
+| nwii	| Nintendo Wii |
+| ncube	| Nintendo Game Cube |
+| riot	| Riot |
+| wargaming	| Wargaming |
+| ngameboy	| Nintendo Game Boy |
+| atari	| Atari |
+| amiga	| Amiga |
+| snes	| SNES |
+| beamdog	| Beamdog |
+| d2d	| Direct2Drive |
+| discord	| Discord |
+| dotemu	| DotEmu |
+| gamehouse	| GameHouse |
+| gmg	| Green Man Gaming |
+| weplay	| WePlay |
+| zx	| Zx Spectrum PC |
+| vision	| ColecoVision |
+| nes	| NES |
+| sms	| Sega Master System |
+| c64	| Commodore 64 |
+| pce 	| PC Engine |
+| segag	| Sega Genesis |
+| neo	| NeoGeo |
+| sega32	| Sega 32X |
+| segacd	| Sega CD |
+| 3do	| 3DO Interactive  |
+| saturn	| SegaSaturn |
+| psx	| Sony PlayStation |
+| ps2	| Sony PlayStation 2 |
+| n64	| Nintendo64 |
+| jaguar	| Atari Jaguar |
+| dc	| Sega Dreamcast |
+| xboxog	| Original Xbox games |
+| amazon	| Amazon |
+| gg	| GamersGate |
+| egg	| Newegg |
+| bb	| BestBuy |
+| gameuk	| Game UK |
+| fanatical	| Fanatical store |
+| playasia 	| PlayAsia |
+| stadia	| Google Stadia |
+| arc	| ARC |
+| eso	| ESO |
+| glyph	| Trion World |
+| aionl	| Aion: Legions of War |
+| aion	| Aion |
+| blade	| Blade and Soul |
+| gw	| Guild Wars |
+| gw2	| Guild Wars 2 |
+| lin2	| Lineage 2 |
+| ffxi	| Final Fantasy XI |
+| ffxiv	| Final Fantasy XIV |
+| totalwar	| TotalWar |
+| winstore	| Windows Store |
+| elites	| Elite Dangerous |
+| star	| Star Citizen |
+| psp	| Playstation Portable |
+| psvita	| Playstation Vita |
+| nds	| Nintendo DS |
+| 3ds	| Nintendo 3DS |
+| pathofexile | Path of Exile |

README.md

@@ -1,25 +1,35 @@
-# GOG Galaxy - Community Integration - Python API
+# GOG Galaxy Integrations Python API
 
-This Python library allows to easily build community integrations for various gaming platforms with GOG Galaxy 2.0.
+This Python library allows developers to easily build community integrations for various gaming platforms with GOG Galaxy 2.0.
 
-Each integration in GOG Galaxy 2.0 comes as a separate Python script, and is launched as a separate process, that needs to communicate with main instance of GOG Galaxy 2.0.
+- refer to our <a href='https://galaxy-integrations-python-api.readthedocs.io'>documentation</a>
+
+## Features
+
+Each integration in GOG Galaxy 2.0 comes as a separate Python script and is launched as a separate process that needs to communicate with the main instance of GOG Galaxy 2.0.
 
 The provided features are:
 
-- multistep authorisation using a browser built into GOG Galaxy 2.0
+- multistep authorization using a browser built into GOG Galaxy 2.0
 - support for GOG Galaxy 2.0 features:
-    - importing owned and detecting installed games
-    - installing and launching games
-    - importing achievements and game time
-    - importing friends lists and statuses
-    - importing friends recomendations list
-    - receiving and sending chat messages
+  - importing owned and detecting installed games
+  - installing and launching games
+  - importing achievements and game time
+  - importing friends lists and statuses
+  - importing friends recommendations list
+  - receiving and sending chat messages
 - cache storage
 
+## Platform Id's
+
+Each integration can implement only one platform. Each integration must declare which platform it's integrating.
+
+[List of possible Platform IDs](PLATFORM_IDs.md)
+
 ## Basic usage
 
-Eeach integration should inherit from the :class:`~galaxy.api.plugin.Plugin` class. Supported methods like :meth:`~galaxy.api.plugin.Plugin.get_owned_games` should be overwritten - they are called from the GOG Galaxy client in the appropriate times.
-Each of those method can raise exceptions inherited from the :exc:`~galaxy.api.jsonrpc.ApplicationError`.
+Each integration should inherit from the :class:`~galaxy.api.plugin.Plugin` class. Supported methods like :meth:`~galaxy.api.plugin.Plugin.get_owned_games` should be overwritten - they are called from the GOG Galaxy client at the appropriate times.
+Each of those methods can raise exceptions inherited from the :exc:`~galaxy.api.jsonrpc.ApplicationError`.
 Communication between an integration and the client is also possible with the use of notifications, for example: :meth:`~galaxy.api.plugin.Plugin.update_local_game_status`.
 
 ```python
@@ -51,8 +61,25 @@ if __name__ == "__main__":
 
 ## Deployment
 
-The client has a built-in Python 3.7 interpreter, so the integrations are delivered as `.py` files.
-The additional `manifest.json` file is required:
+The client has a built-in Python 3.7 interpreter, so integrations are delivered as Python modules.
+In order to be found by GOG Galaxy 2.0 an integration folder should be placed in [lookup directory](#deploy-location). Beside all the Python files, the integration folder must contain [manifest.json](#deploy-manifest) and all third-party dependencies. See an [exemplary structure](#deploy-structure-example).
+
+### Lookup directory
+
+<a name="deploy-location"></a>
+
+- Windows:
+
+    `%localappdata%\GOG.com\Galaxy\plugins\installed`
+
+- macOS:
+
+    `~/Library/Application Support/GOG.com/Galaxy/plugins/installed`
+
+### Manifest
+
+<a name="deploy-manifest"></a>
+Obligatory JSON file to be placed in an integration folder.
 
 ```json
 {
@@ -67,6 +94,35 @@ The additional `manifest.json` file is required:
     "script": "plugin.py"
 }
 ```
+
+| property      | description |
+|---------------|---|
+| `guid`        |   |
+| `description` |   |
+| `url`         |   |
+| `script`      | path of the entry point module, relative to the integration folder |
+
+### Dependencies
+
+All third-party packages (packages not included in the Python 3.7 standard library) should be deployed along with plugin files. Use the following command structure:
+
+```pip install DEP --target DIR --implementation cp --python-version 37```
+
+For example, a plugin that uses *requests* could have the following structure:
+
+<a name="deploy-structure-example"></a>
+
+```bash
+installed
+└── my_integration
+    ├── galaxy
+    │   └── api
+    ├── requests
+    │   └── ...
+    ├── plugin.py
+    └── manifest.json
+```
+
 ## Legal Notice
 
-By integrating or attempting to integrate any applications or content with or into GOG Galaxy 2.0. you represent that such application or content is your original creation (other than any software made available by GOG) and/or that you have all necessary rights to grant such applicable rights to the relevant community integration to GOG and to GOG Galaxy 2.0 end users for the purpose of use of such community integration and that such community integration comply with any third party license and other requirements including compliance with applicable laws.
+By integrating or attempting to integrate any applications or content with or into GOG Galaxy 2.0 you represent that such application or content is your original creation (other than any software made available by GOG) and/or that you have all necessary rights to grant such applicable rights to the relevant community integration to GOG and to GOG Galaxy 2.0 end users for the purpose of use of such community integration and that such community integration comply with any third party license and other requirements including compliance with applicable laws.

docs/make.py

@@ -1,13 +1,16 @@
 """Builds documentation locally. Use for preview only"""
 
+import pathlib
 import subprocess
 import webbrowser
 
 
-source = "docs/source"
-build = "docs/build"
-master_doc = 'index'
+source = pathlib.Path("docs", "source")
+build = pathlib.Path("docs", "build")
+master_doc = 'index.html'
 
-subprocess.run(['sphinx-build', '-M', 'clean', source, build])
-subprocess.run(['sphinx-build', '-M', 'html', source, build])
-webbrowser.open(f'{build}/html/{master_doc}')
+subprocess.run(['sphinx-build', '-M', 'clean', str(source), str(build)])
+subprocess.run(['sphinx-build', '-M', 'html', str(source), str(build)])
+
+master_path = build / 'html' / master_doc
+webbrowser.open(f'file://{master_path.resolve()}')

docs/source/index.rst

@@ -7,6 +7,7 @@ GOG Galaxy Integrations Python API
 
    Overview <overview>
    API <galaxy.api>
+   Platform ID's <platforms>
 
 Index
 -------------------

docs/source/overview.rst

@@ -1 +1,15 @@
 .. mdinclude:: ../../README.md
+   :end-line: 4
+
+.. excluding self-pointing documentation link
+
+.. mdinclude:: ../../README.md
+   :start-line: 6
+   :end-line: 26
+   
+.. excluding Platforms Id's link
+
+:ref:`platforms-link`
+
+.. mdinclude:: ../../README.md
+   :start-line: 28

docs/source/platforms.rst

@@ -0,0 +1,2 @@
+.. _platforms-link:
+.. mdinclude:: ../../PLATFORM_IDs.md

setup.py

@@ -2,7 +2,7 @@ from setuptools import setup, find_packages
 
 setup(
     name="galaxy.plugin.api",
-    version="0.32.0",
+    version="0.41",
     description="GOG Galaxy Integrations Python API",
     author='Galaxy team',
     author_email='galaxy@gog.com',

src/galaxy/api/consts.py

@@ -13,7 +13,74 @@ class Platform(Enum):
     Uplay = "uplay"
     Battlenet = "battlenet"
     Epic = "epic"
-
+    Bethesda = "bethesda"
+    ParadoxPlaza = "paradox"
+    HumbleBundle = "humble"
+    Kartridge = "kartridge"
+    ItchIo = "itch"
+    NintendoSwitch = "nswitch"
+    NintendoWiiU = "nwiiu"
+    NintendoWii = "nwii"
+    NintendoGameCube = "ncube"
+    RiotGames = "riot"
+    Wargaming = "wargaming"
+    NintendoGameBoy = "ngameboy"
+    Atari = "atari"
+    Amiga = "amiga"
+    SuperNintendoEntertainmentSystem = "snes"
+    Beamdog = "beamdog"
+    Direct2Drive = "d2d"
+    Discord = "discord"
+    DotEmu = "dotemu"
+    GameHouse = "gamehouse"
+    GreenManGaming = "gmg"
+    WePlay = "weplay"
+    ZxSpectrum = "zx"
+    ColecoVision = "vision"
+    NintendoEntertainmentSystem = "nes"
+    SegaMasterSystem = "sms"
+    Commodore64 = "c64"
+    PcEngine = "pce"
+    SegaGenesis = "segag"
+    NeoGeo = "neo"
+    Sega32X = "sega32"
+    SegaCd = "segacd"
+    _3Do = "3do"
+    SegaSaturn = "saturn"
+    PlayStation = "psx"
+    PlayStation2 = "ps2"
+    Nintendo64 = "n64"
+    AtariJaguar = "jaguar"
+    SegaDreamcast = "dc"
+    Xbox = "xboxog"
+    Amazon = "amazon"
+    GamersGate = "gg"
+    Newegg = "egg"
+    BestBuy = "bb"
+    GameUk = "gameuk"
+    Fanatical = "fanatical"
+    PlayAsia = "playasia"
+    Stadia = "stadia"
+    Arc = "arc"
+    ElderScrollsOnline = "eso"
+    Glyph = "glyph"
+    AionLegionsOfWar = "aionl"
+    Aion = "aion"
+    BladeAndSoul = "blade"
+    GuildWars = "gw"
+    GuildWars2 = "gw2"
+    Lineage2 = "lin2"
+    FinalFantasy11 = "ffxi"
+    FinalFantasy14 = "ffxiv"
+    TotalWar = "totalwar"
+    WindowsStore = "winstore"
+    EliteDangerous = "elites"
+    StarCitizen = "star"
+    PlayStationPortable = "psp"
+    PlayStationVita = "psvita"
+    NintendoDs = "nds"
+    Nintendo3Ds = "3ds"
+    PathOfExile = "pathofexile"
 
 class Feature(Enum):
     """Possible features that can be implemented by an integration.

src/galaxy/api/errors.py

@@ -22,6 +22,10 @@ class UnknownBackendResponse(ApplicationError):
     def __init__(self, data=None):
         super().__init__(4, "Backend responded in uknown way", data)
 
+class TooManyRequests(ApplicationError):
+    def __init__(self, data=None):
+        super().__init__(5, "Too many requests. Try again later", data)
+
 class InvalidCredentials(ApplicationError):
     def __init__(self, data=None):
         super().__init__(100, "Invalid credentials", data)
@@ -50,18 +54,6 @@ class AccessDenied(ApplicationError):
     def __init__(self, data=None):
         super().__init__(106, "Access denied", data)
 
-class ParentalControlBlock(ApplicationError):
-    def __init__(self, data=None):
-        super().__init__(107, "Parental control block", data)
-
-class DeviceBlocked(ApplicationError):
-    def __init__(self, data=None):
-        super().__init__(108, "Device blocked", data)
-
-class RegionBlocked(ApplicationError):
-    def __init__(self, data=None):
-        super().__init__(109, "Region blocked", data)
-
 class FailedParsingManifest(ApplicationError):
     def __init__(self, data=None):
         super().__init__(200, "Failed parsing manifest", data)
@@ -80,4 +72,4 @@ class MessageNotFound(ApplicationError):
 
 class ImportInProgress(ApplicationError):
     def __init__(self, data=None):
-        super().__init__(600, "Import already in progress", data)
+        super().__init__(600, "Import already in progress", data)

src/galaxy/api/jsonrpc.py

@@ -5,6 +5,8 @@ import logging
 import inspect
 import json
 
+from galaxy.reader import StreamLineReader
+
 class JsonRpcError(Exception):
     def __init__(self, code, message, data=None):
         self.code = code
@@ -54,22 +56,20 @@ Method = namedtuple("Method", ["callback", "signature", "internal", "sensitive_p
 
 def anonymise_sensitive_params(params, sensitive_params):
     anomized_data = "****"
-    if not sensitive_params:
-        return params
+
+    if isinstance(sensitive_params, bool):
+        if sensitive_params:
+            return {k:anomized_data for k,v in params.items()}
 
     if isinstance(sensitive_params, Iterable):
-        anomized_params = params.copy()
-        for key in anomized_params.keys():
-            if key in sensitive_params:
-                anomized_params[key] = anomized_data
-        return anomized_params
+        return {k: anomized_data if k in sensitive_params else v for k, v in params.items()}
 
-    return anomized_data
+    return params
 
 class Server():
     def __init__(self, reader, writer, encoder=json.JSONEncoder()):
         self._active = True
-        self._reader = reader
+        self._reader = StreamLineReader(reader)
         self._writer = writer
         self._encoder = encoder
         self._methods = {}
@@ -116,6 +116,7 @@ class Server():
             data = data.strip()
             logging.debug("Received %d bytes of data", len(data))
             self._handle_input(data)
+            await asyncio.sleep(0) # To not starve task queue
 
     def stop(self):
         self._active = False

src/galaxy/api/plugin.py

@@ -7,16 +7,18 @@ from enum import Enum
 from collections import OrderedDict
 import sys
 
-from typing import List, Dict
+from typing import Any, List, Dict, Optional, Union
 
 from galaxy.api.types import Achievement, Game, LocalGame, FriendInfo, GameTime, UserInfo, Room
 
 from galaxy.api.jsonrpc import Server, NotificationClient, ApplicationError
 from galaxy.api.consts import Feature
 from galaxy.api.errors import UnknownError, ImportInProgress
+from galaxy.api.types import Authentication, NextStep, Message
+
 
 class JSONEncoder(json.JSONEncoder):
-    def default(self, o): # pylint: disable=method-hidden
+    def default(self, o):  # pylint: disable=method-hidden
         if dataclasses.is_dataclass(o):
             # filter None values
             def dict_factory(elements):
@@ -26,7 +28,8 @@ class JSONEncoder(json.JSONEncoder):
             return o.value
         return super().default(o)
 
-class Plugin():
+
+class Plugin:
     """Use and override methods of this class to create a new platform integration."""
     def __init__(self, platform, version, reader, writer, handshake_token):
         logging.info("Creating plugin for platform %s, version %s", platform.value, version)
@@ -35,6 +38,7 @@ class Plugin():
 
         self._feature_methods = OrderedDict()
         self._active = True
+        self._pass_control_task = None
 
         self._reader, self._writer = reader, writer
         self._handshake_token = handshake_token
@@ -50,9 +54,17 @@ class Plugin():
         self._achievements_import_in_progress = False
         self._game_times_import_in_progress = False
 
+        self._persistent_cache = dict()
+
         # internal
         self._register_method("shutdown", self._shutdown, internal=True)
         self._register_method("get_capabilities", self._get_capabilities, internal=True)
+        self._register_method(
+            "initialize_cache",
+            self._initialize_cache,
+            internal=True,
+            sensitive_params="data"
+        )
         self._register_method("ping", self._ping, internal=True)
 
         # implemented by developer
@@ -156,6 +168,12 @@ class Plugin():
 
         return features
 
+    @property
+    def persistent_cache(self) -> Dict:
+        """The cache is only available after the :meth:`~.handshake_complete()` is called.
+        """
+        return self._persistent_cache
+
     def _implements(self, handlers):
         for handler in handlers:
             if handler.__name__ not in self.__class__.__dict__:
@@ -192,16 +210,18 @@ class Plugin():
             self._feature_methods.setdefault(feature, []).append(handler)
 
     async def run(self):
-        """Plugin main coroutine."""
-        async def pass_control():
-            while self._active:
-                try:
-                    self.tick()
-                except Exception:
-                    logging.exception("Unexpected exception raised in plugin tick")
-                await asyncio.sleep(1)
+        """Plugin's main coroutine."""
+        await self._server.run()
+        if self._pass_control_task is not None:
+            await self._pass_control_task
 
-        await asyncio.gather(pass_control(), self._server.run())
+    async def _pass_control(self):
+        while self._active:
+            try:
+                self.tick()
+            except Exception:
+                logging.exception("Unexpected exception raised in plugin tick")
+            await asyncio.sleep(1)
 
     def _shutdown(self):
         logging.info("Shutting down")
@@ -216,12 +236,17 @@ class Plugin():
             "token": self._handshake_token
         }
 
+    def _initialize_cache(self, data: Dict):
+        self._persistent_cache = data
+        self.handshake_complete()
+        self._pass_control_task = asyncio.create_task(self._pass_control())
+
     @staticmethod
     def _ping():
         pass
 
     # notifications
-    def store_credentials(self, credentials: dict):
+    def store_credentials(self, credentials: Dict[str, Any]) -> None:
         """Notify the client to store authentication credentials.
         Credentials are passed on the next authenticate call.
 
@@ -243,9 +268,10 @@ class Plugin():
                 return Authentication(user_data['userId'], user_data['username'])
 
          """
+        self.persistent_cache['credentials'] = credentials
         self._notification_client.notify("store_credentials", credentials, sensitive_params=True)
 
-    def add_game(self, game: Game):
+    def add_game(self, game: Game) -> None:
         """Notify the client to add game to the list of owned games
         of the currently authenticated user.
 
@@ -264,10 +290,10 @@ class Plugin():
                         self.add_game(game)
 
         """
-        params = {"owned_game" : game}
+        params = {"owned_game": game}
         self._notification_client.notify("owned_game_added", params)
 
-    def remove_game(self, game_id: str):
+    def remove_game(self, game_id: str) -> None:
         """Notify the client to remove game from the list of owned games
         of the currently authenticated user.
 
@@ -286,19 +312,19 @@ class Plugin():
                         self.remove_game(game.game_id)
 
         """
-        params = {"game_id" : game_id}
+        params = {"game_id": game_id}
         self._notification_client.notify("owned_game_removed", params)
 
-    def update_game(self, game: Game):
+    def update_game(self, game: Game) -> None:
         """Notify the client to update the status of a game
         owned by the currently authenticated user.
 
         :param game: Game to update
         """
-        params = {"owned_game" : game}
+        params = {"owned_game": game}
         self._notification_client.notify("owned_game_updated", params)
 
-    def unlock_achievement(self, game_id: str, achievement: Achievement):
+    def unlock_achievement(self, game_id: str, achievement: Achievement) -> None:
         """Notify the client to unlock an achievement for a specific game.
 
         :param game_id: game_id of the game for which to unlock an achievement.
@@ -310,7 +336,7 @@ class Plugin():
         }
         self._notification_client.notify("achievement_unlocked", params)
 
-    def game_achievements_import_success(self, game_id: str, achievements):
+    def game_achievements_import_success(self, game_id: str, achievements: List[Achievement]) -> None:
         """Notify the client that import of achievements for a given game has succeeded.
         This method is called by import_games_achievements.
 
@@ -323,7 +349,7 @@ class Plugin():
         }
         self._notification_client.notify("game_achievements_import_success", params)
 
-    def game_achievements_import_failure(self, game_id: str, error: ApplicationError):
+    def game_achievements_import_failure(self, game_id: str, error: ApplicationError) -> None:
         """Notify the client that import of achievements for a given game has failed.
         This method is called by import_games_achievements.
 
@@ -339,12 +365,12 @@ class Plugin():
         }
         self._notification_client.notify("game_achievements_import_failure", params)
 
-    def achievements_import_finished(self):
+    def achievements_import_finished(self) -> None:
         """Notify the client that importing achievements has finished.
         This method is called by import_games_achievements_task"""
         self._notification_client.notify("achievements_import_finished", None)
 
-    def update_local_game_status(self, local_game: LocalGame):
+    def update_local_game_status(self, local_game: LocalGame) -> None:
         """Notify the client to update the status of a local game.
 
         :param local_game: the LocalGame to update
@@ -367,26 +393,31 @@ class Plugin():
                 if self._check_statuses_task is None or self._check_statuses_task.done():
                     self._check_statuses_task = asyncio.create_task(self._check_statuses())
         """
-        params = {"local_game" : local_game}
+        params = {"local_game": local_game}
         self._notification_client.notify("local_game_status_changed", params)
 
-    def add_friend(self, user: FriendInfo):
+    def add_friend(self, user: FriendInfo) -> None:
         """Notify the client to add a user to friends list of the currently authenticated user.
 
         :param user: FriendInfo of a user that the client will add to friends list
         """
-        params = {"friend_info" : user}
+        params = {"friend_info": user}
         self._notification_client.notify("friend_added", params)
 
-    def remove_friend(self, user_id: str):
+    def remove_friend(self, user_id: str) -> None:
         """Notify the client to remove a user from friends list of the currently authenticated user.
 
         :param user_id: id of the user to remove from friends list
         """
-        params = {"user_id" : user_id}
+        params = {"user_id": user_id}
         self._notification_client.notify("friend_removed", params)
 
-    def update_room(self, room_id: str, unread_message_count=None, new_messages=None):
+    def update_room(
+        self,
+        room_id: str,
+        unread_message_count: Optional[int]=None,
+        new_messages: Optional[List[Message]]=None
+    ) -> None:
         """WIP, Notify the client to update the information regarding
         a chat room that the currently authenticated user is in.
 
@@ -401,24 +432,24 @@ class Plugin():
             params["messages"] = new_messages
         self._notification_client.notify("chat_room_updated", params)
 
-    def update_game_time(self, game_time: GameTime):
+    def update_game_time(self, game_time: GameTime) -> None:
         """Notify the client to update game time for a game.
 
         :param game_time: game time to update
         """
-        params = {"game_time" : game_time}
+        params = {"game_time": game_time}
         self._notification_client.notify("game_time_updated", params)
 
-    def game_time_import_success(self, game_time: GameTime):
+    def game_time_import_success(self, game_time: GameTime) -> None:
         """Notify the client that import of a given game_time has succeeded.
         This method is called by import_game_times.
 
         :param game_time: game_time which was imported
         """
-        params = {"game_time" : game_time}
+        params = {"game_time": game_time}
         self._notification_client.notify("game_time_import_success", params)
 
-    def game_time_import_failure(self, game_id: str, error: ApplicationError):
+    def game_time_import_failure(self, game_id: str, error: ApplicationError) -> None:
         """Notify the client that import of a game time for a given game has failed.
         This method is called by import_game_times.
 
@@ -434,20 +465,36 @@ class Plugin():
         }
         self._notification_client.notify("game_time_import_failure", params)
 
-    def game_times_import_finished(self):
+    def game_times_import_finished(self) -> None:
         """Notify the client that importing game times has finished.
         This method is called by :meth:`~.import_game_times_task`.
         """
         self._notification_client.notify("game_times_import_finished", None)
 
-    def lost_authentication(self):
+    def lost_authentication(self) -> None:
         """Notify the client that integration has lost authentication for the
          current user and is unable to perform actions which would require it.
          """
         self._notification_client.notify("authentication_lost", None)
 
+    def push_cache(self) -> None:
+        """Push local copy of the persistent cache to the GOG Galaxy Client replacing existing one.
+        """
+        self._notification_client.notify(
+            "push_cache",
+            params={"data": self._persistent_cache},
+            sensitive_params="data"
+        )
+
     # handlers
-    def tick(self):
+    def handshake_complete(self) -> None:
+        """This method is called right after the handshake with the GOG Galaxy Client is complete and
+        before any other operations are called by the GOG Galaxy Client.
+        Persistent cache is available when this method is called.
+        Override it if you need to do additional plugin initializations.
+        This method is called internally."""
+
+    def tick(self) -> None:
         """This method is called periodically.
         Override it to implement periodical non-blocking tasks.
         This method is called internally.
@@ -467,17 +514,17 @@ class Plugin():
 
         """
 
-    def shutdown(self):
+    def shutdown(self) -> None:
         """This method is called on integration shutdown.
         Override it to implement tear down.
-        This method is called by the GOG Galaxy client."""
+        This method is called by the GOG Galaxy Client."""
 
     # methods
-    async def authenticate(self, stored_credentials:dict=None):
+    async def authenticate(self, stored_credentials: Optional[Dict] = None) -> Union[NextStep, Authentication]:
         """Override this method to handle user authentication.
         This method should either return :class:`~galaxy.api.types.Authentication` if the authentication is finished
         or :class:`~galaxy.api.types.NextStep` if it requires going to another url.
-        This method is called by the GOG Galaxy client.
+        This method is called by the GOG Galaxy Client.
 
         :param stored_credentials: If the client received any credentials to store locally
          in the previous session they will be passed here as a parameter.
@@ -501,12 +548,13 @@ class Plugin():
         """
         raise NotImplementedError()
 
-    async def pass_login_credentials(self, step: str, credentials: Dict[str, str], cookies: List[Dict[str, str]]):
+    async def pass_login_credentials(self, step: str, credentials: Dict[str, str], cookies: List[Dict[str, str]]) \
+            -> Union[NextStep, Authentication]:
         """This method is called if we return galaxy.api.types.NextStep from authenticate or from pass_login_credentials.
         This method's parameters provide the data extracted from the web page navigation that previous NextStep finished on.
         This method should either return galaxy.api.types.Authentication if the authentication is finished
         or galaxy.api.types.NextStep if it requires going to another cef url.
-        This method is called by the GOG Galaxy client.
+        This method is called by the GOG Galaxy Client.
 
         :param step: deprecated.
         :param credentials: end_uri previous NextStep finished on.
@@ -531,8 +579,8 @@ class Plugin():
         raise NotImplementedError()
 
     async def get_owned_games(self) -> List[Game]:
-        """Override this method to return owned games for currenly logged in user.
-        This method is called by the GOG Galaxy client.
+        """Override this method to return owned games for currently logged in user.
+        This method is called by the GOG Galaxy Client.
 
         Example of possible override of the method:
 
@@ -556,9 +604,9 @@ class Plugin():
         """
         raise NotImplementedError()
 
-    async def start_achievements_import(self, game_ids: List[str]):
+    async def start_achievements_import(self, game_ids: List[str]) -> None:
         """Starts the task of importing achievements.
-        This method is called by the GOG Galaxy client.
+        This method is called by the GOG Galaxy Client.
 
         :param game_ids: ids of the games for which the achievements are imported
         """
@@ -575,14 +623,14 @@ class Plugin():
         asyncio.create_task(import_games_achievements_task(game_ids))
         self._achievements_import_in_progress = True
 
-    async def import_games_achievements(self, game_ids: List[str]):
+    async def import_games_achievements(self, game_ids: List[str]) -> None:
         """
         Override this method to return the unlocked achievements
         of the user that is currently logged in to the plugin.
         Call game_achievements_import_success/game_achievements_import_failure for each game_id on the list.
-        This method is called by the GOG Galaxy client.
+        This method is called by the GOG Galaxy Client.
 
-        :param game_id: ids of the games for which to import unlocked achievements
+        :param game_ids: ids of the games for which to import unlocked achievements
         """
         async def import_game_achievements(game_id):
             try:
@@ -597,7 +645,7 @@ class Plugin():
     async def get_local_games(self) -> List[LocalGame]:
         """Override this method to return the list of
         games present locally on the users pc.
-        This method is called by the GOG Galaxy client.
+        This method is called by the GOG Galaxy Client.
 
         Example of possible override of the method:
 
@@ -616,10 +664,10 @@ class Plugin():
         """
         raise NotImplementedError()
 
-    async def launch_game(self, game_id: str):
+    async def launch_game(self, game_id: str) -> None:
         """Override this method to launch the game
         identified by the provided game_id.
-        This method is called by the GOG Galaxy client.
+        This method is called by the GOG Galaxy Client.
 
         :param str game_id: id of the game to launch
 
@@ -634,10 +682,10 @@ class Plugin():
         """
         raise NotImplementedError()
 
-    async def install_game(self, game_id: str):
+    async def install_game(self, game_id: str) -> None:
         """Override this method to install the game
         identified by the provided game_id.
-        This method is called by the GOG Galaxy client.
+        This method is called by the GOG Galaxy Client.
 
         :param str game_id: id of the game to install
 
@@ -652,10 +700,10 @@ class Plugin():
         """
         raise NotImplementedError()
 
-    async def uninstall_game(self, game_id: str):
+    async def uninstall_game(self, game_id: str) -> None:
         """Override this method to uninstall the game
         identified by the provided game_id.
-        This method is called by the GOG Galaxy client.
+        This method is called by the GOG Galaxy Client.
 
         :param str game_id: id of the game to uninstall
 
@@ -673,7 +721,7 @@ class Plugin():
     async def get_friends(self) -> List[FriendInfo]:
         """Override this method to return the friends list
         of the currently authenticated user.
-        This method is called by the GOG Galaxy client.
+        This method is called by the GOG Galaxy Client.
 
         Example of possible override of the method:
 
@@ -692,48 +740,49 @@ class Plugin():
 
     async def get_users(self, user_id_list: List[str]) -> List[UserInfo]:
         """WIP, Override this method to return the list of users matching the provided ids.
-        This method is called by the GOG Galaxy client.
+        This method is called by the GOG Galaxy Client.
 
         :param user_id_list: list of user ids
         """
         raise NotImplementedError()
 
-    async def send_message(self, room_id: str, message_text: str):
+    async def send_message(self, room_id: str, message_text: str) -> None:
         """WIP, Override this method to send message to a chat room.
-         This method is called by the GOG Galaxy client.
+         This method is called by the GOG Galaxy Client.
 
          :param room_id: id of the room to which the message should be sent
          :param message_text: text which should be sent in the message
          """
         raise NotImplementedError()
 
-    async def mark_as_read(self, room_id: str, last_message_id: str):
+    async def mark_as_read(self, room_id: str, last_message_id: str) -> None:
         """WIP, Override this method to mark messages in a chat room as read up to the id provided in the parameter.
-        This method is called by the GOG Galaxy client.
+        This method is called by the GOG Galaxy Client.
 
         :param room_id: id of the room
-        :param last_message_id: id of the last message; room is marked as read only if this id matches the last message id known to the client
+        :param last_message_id: id of the last message; room is marked as read only if this id matches
+         the last message id known to the client
         """
         raise NotImplementedError()
 
     async def get_rooms(self) -> List[Room]:
         """WIP, Override this method to return the chat rooms in which the user is currently in.
-        This method is called by the GOG Galaxy client
+        This method is called by the GOG Galaxy Client
         """
         raise NotImplementedError()
 
-    async def get_room_history_from_message(self, room_id: str, message_id: str):
+    async def get_room_history_from_message(self, room_id: str, message_id: str) -> List[Message]:
         """WIP, Override this method to return the chat room history since the message provided in parameter.
-        This method is called by the GOG Galaxy client.
+        This method is called by the GOG Galaxy Client.
 
         :param room_id: id of the room
         :param message_id: id of the message since which the history should be retrieved
         """
         raise NotImplementedError()
 
-    async def get_room_history_from_timestamp(self, room_id: str, from_timestamp: int):
+    async def get_room_history_from_timestamp(self, room_id: str, from_timestamp: int) -> List[Message]:
         """WIP, Override this method to return the chat room history since the timestamp provided in parameter.
-        This method is called by the GOG Galaxy client.
+        This method is called by the GOG Galaxy Client.
 
         :param room_id: id of the room
         :param from_timestamp: timestamp since which the history should be retrieved
@@ -747,9 +796,9 @@ class Plugin():
         """
         raise NotImplementedError()
 
-    async def start_game_times_import(self, game_ids: List[str]):
+    async def start_game_times_import(self, game_ids: List[str]) -> None:
         """Starts the task of importing game times
-        This method is called by the GOG Galaxy client.
+        This method is called by the GOG Galaxy Client.
 
         :param game_ids: ids of the games for which the game time is imported
         """
@@ -766,7 +815,7 @@ class Plugin():
         asyncio.create_task(import_game_times_task(game_ids))
         self._game_times_import_in_progress = True
 
-    async def import_game_times(self, game_ids: List[str]):
+    async def import_game_times(self, game_ids: List[str]) -> None:
         """
         Override this method to return game times for
         games owned by the currently authenticated user.
@@ -829,7 +878,7 @@ def create_and_run_plugin(plugin_class, argv):
 
     async def coroutine():
         reader, writer = await asyncio.open_connection("127.0.0.1", port)
-        extra_info = writer.get_extra_info('sockname')
+        extra_info = writer.get_extra_info("sockname")
         logging.info("Using local address: %s:%u", *extra_info)
         plugin = plugin_class(reader, writer, token)
         await plugin.run()

src/galaxy/api/types.py

@@ -204,5 +204,5 @@ class GameTime():
     :param last_time_played: last time the game was played (**unix timestamp**)
     """
     game_id: str
-    time_played: int
-    last_played_time: int
+    time_played: Optional[int]
+    last_played_time: Optional[int]

src/galaxy/http.py

@@ -1,47 +1,81 @@
 import asyncio
 import ssl
+from contextlib import contextmanager
 from http import HTTPStatus
 
 import aiohttp
 import certifi
+import logging
 
 from galaxy.api.errors import (
-    AccessDenied, AuthenticationRequired,
-    BackendTimeout, BackendNotAvailable, BackendError, NetworkError, UnknownBackendResponse, UnknownError
+    AccessDenied, AuthenticationRequired, BackendTimeout, BackendNotAvailable, BackendError, NetworkError,
+    TooManyRequests, UnknownBackendResponse, UnknownError
 )
 
+
+DEFAULT_LIMIT = 20
+DEFAULT_TIMEOUT = 60  # seconds
+
+
 class HttpClient:
-    def __init__(self, limit=20, timeout=aiohttp.ClientTimeout(total=60), cookie_jar=None):
-        ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
-        ssl_context.load_verify_locations(certifi.where())
-        connector = aiohttp.TCPConnector(limit=limit, ssl=ssl_context)
-        self._session = aiohttp.ClientSession(connector=connector, timeout=timeout, cookie_jar=cookie_jar)
+    """Deprecated"""
+    def __init__(self, limit=DEFAULT_LIMIT, timeout=aiohttp.ClientTimeout(total=DEFAULT_TIMEOUT), cookie_jar=None):
+        connector = create_tcp_connector(limit=limit)
+        self._session = create_client_session(connector=connector, timeout=timeout, cookie_jar=cookie_jar)
 
     async def close(self):
         await self._session.close()
 
-    async def request(self, method, *args, **kwargs):
-        try:
-            response = await self._session.request(method, *args, **kwargs)
-        except asyncio.TimeoutError:
-            raise BackendTimeout()
-        except aiohttp.ServerDisconnectedError:
-            raise BackendNotAvailable()
-        except aiohttp.ClientConnectionError:
-            raise NetworkError()
-        except aiohttp.ContentTypeError:
-            raise UnknownBackendResponse()
-        except aiohttp.ClientError:
-            raise UnknownError()
-        if response.status == HTTPStatus.UNAUTHORIZED:
-            raise AuthenticationRequired()
-        if response.status == HTTPStatus.FORBIDDEN:
-            raise AccessDenied()
-        if response.status == HTTPStatus.SERVICE_UNAVAILABLE:
-            raise BackendNotAvailable()
-        if response.status >= 500:
-            raise BackendError()
-        if response.status >= 400:
-            raise UnknownError()
+    async def request(self, method, url, *args, **kwargs):
+        with handle_exception():
+            return await self._session.request(method, url, *args, **kwargs)
+
+
+def create_tcp_connector(*args, **kwargs):
+    ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
+    ssl_context.load_verify_locations(certifi.where())
+    kwargs.setdefault("ssl", ssl_context)
+    kwargs.setdefault("limit", DEFAULT_LIMIT)
+    return aiohttp.TCPConnector(*args, **kwargs)
+
+
+def create_client_session(*args, **kwargs):
+    kwargs.setdefault("connector", create_tcp_connector())
+    kwargs.setdefault("timeout", aiohttp.ClientTimeout(total=DEFAULT_TIMEOUT))
+    kwargs.setdefault("raise_for_status", True)
+    return aiohttp.ClientSession(*args, **kwargs)
+
+
+@contextmanager
+def handle_exception():
+    try:
+        yield
+    except asyncio.TimeoutError:
+        raise BackendTimeout()
+    except aiohttp.ServerDisconnectedError:
+        raise BackendNotAvailable()
+    except aiohttp.ClientConnectionError:
+        raise NetworkError()
+    except aiohttp.ContentTypeError:
+        raise UnknownBackendResponse()
+    except aiohttp.ClientResponseError as error:
+        if error.status == HTTPStatus.UNAUTHORIZED:
+            raise AuthenticationRequired()
+        if error.status == HTTPStatus.FORBIDDEN:
+            raise AccessDenied()
+        if error.status == HTTPStatus.SERVICE_UNAVAILABLE:
+            raise BackendNotAvailable()
+        if error.status == HTTPStatus.TOO_MANY_REQUESTS:
+            raise TooManyRequests()
+        if error.status >= 500:
+            raise BackendError()
+        if error.status >= 400:
+            logging.warning(
+                "Got status %d while performing %s request for %s",
+                error.status, error.request_info.method, str(error.request_info.url)
+            )
+            raise UnknownError()
+    except aiohttp.ClientError:
+        logging.exception("Caught exception while performing request")
+        raise UnknownError()
 
-        return response

src/galaxy/reader.py

@@ -0,0 +1,28 @@
+from asyncio import StreamReader
+
+
+class StreamLineReader:
+    """Handles StreamReader readline without buffer limit"""
+    def __init__(self, reader: StreamReader):
+        self._reader = reader
+        self._buffer = bytes()
+        self._processed_buffer_it = 0
+
+    async def readline(self):
+        while True:
+            # check if there is no unprocessed data in the buffer
+            if not self._buffer or self._processed_buffer_it != 0:
+                chunk = await self._reader.read(1024)
+                if not chunk:
+                    return bytes() # EOF
+                self._buffer += chunk
+
+            it = self._buffer.find(b"\n", self._processed_buffer_it)
+            if it < 0:
+                self._processed_buffer_it = len(self._buffer)
+                continue
+
+            line = self._buffer[:it]
+            self._buffer = self._buffer[it+1:]
+            self._processed_buffer_it = 0
+            return line

tests/conftest.py

@@ -11,7 +11,7 @@ from galaxy.unittest.mock import AsyncMock, coroutine_mock
 @pytest.fixture()
 def reader():
     stream = MagicMock(name="stream_reader")
-    stream.readline = AsyncMock()
+    stream.read = AsyncMock()
     yield stream
 
 @pytest.fixture()
@@ -22,8 +22,8 @@ def writer():
     yield stream
 
 @pytest.fixture()
-def readline(reader):
-    yield reader.readline
+def read(reader):
+    yield reader.read
 
 @pytest.fixture()
 def write(writer):
@@ -33,6 +33,7 @@ def write(writer):
 def plugin(reader, writer):
     """Return plugin instance with all feature methods mocked"""
     async_methods = (
+        "handshake_complete",
         "authenticate",
         "get_owned_games",
         "get_unlocked_achievements",

tests/test_achievements.py

@@ -16,7 +16,7 @@ def test_initialization_no_id_nor_name():
     with raises(AssertionError):
         Achievement(unlock_time=1234567890)
 
-def test_success(plugin, readline, write):
+def test_success(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "3",
@@ -25,7 +25,7 @@ def test_success(plugin, readline, write):
             "game_id": "14"
         }
     }
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_unlocked_achievements.coro.return_value = [
         Achievement(achievement_id="lvl10", unlock_time=1548421241),
         Achievement(achievement_name="Got level 20", unlock_time=1548422395),
@@ -57,7 +57,7 @@ def test_success(plugin, readline, write):
         }
     }
 
-def test_failure(plugin, readline, write):
+def test_failure(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "3",
@@ -67,7 +67,7 @@ def test_failure(plugin, readline, write):
         }
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_unlocked_achievements.coro.side_effect = UnknownError()
     asyncio.run(plugin.run())
     plugin.get_unlocked_achievements.assert_called()

tests/test_authenticate.py

@@ -6,18 +6,17 @@ import pytest
 from galaxy.api.types import Authentication
 from galaxy.api.errors import (
     UnknownError, InvalidCredentials, NetworkError, LoggedInElsewhere, ProtocolError,
-    BackendNotAvailable, BackendTimeout, BackendError, TemporaryBlocked, Banned, AccessDenied,
-    ParentalControlBlock, DeviceBlocked, RegionBlocked
+    BackendNotAvailable, BackendTimeout, BackendError, TemporaryBlocked, Banned, AccessDenied
 )
 
-def test_success(plugin, readline, write):
+def test_success(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "3",
         "method": "init_authentication"
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.authenticate.coro.return_value = Authentication("132", "Zenek")
     asyncio.run(plugin.run())
     plugin.authenticate.assert_called_with()
@@ -44,18 +43,15 @@ def test_success(plugin, readline, write):
     pytest.param(TemporaryBlocked, 104, "Temporary blocked", id="temporary_blocked"),
     pytest.param(Banned, 105, "Banned", id="banned"),
     pytest.param(AccessDenied, 106, "Access denied", id="access_denied"),
-    pytest.param(ParentalControlBlock, 107, "Parental control block", id="parental_control_clock"),
-    pytest.param(DeviceBlocked, 108, "Device blocked", id="device_blocked"),
-    pytest.param(RegionBlocked, 109, "Region blocked", id="region_blocked")
 ])
-def test_failure(plugin, readline, write, error, code, message):
+def test_failure(plugin, read, write, error, code, message):
     request = {
         "jsonrpc": "2.0",
         "id": "3",
         "method": "init_authentication"
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.authenticate.coro.side_effect = error()
     asyncio.run(plugin.run())
     plugin.authenticate.assert_called_with()
@@ -70,7 +66,7 @@ def test_failure(plugin, readline, write, error, code, message):
         }
     }
 
-def test_stored_credentials(plugin, readline, write):
+def test_stored_credentials(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "3",
@@ -81,7 +77,7 @@ def test_stored_credentials(plugin, readline, write):
             }
         }
     }
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.authenticate.coro.return_value = Authentication("132", "Zenek")
     asyncio.run(plugin.run())
     plugin.authenticate.assert_called_with(stored_credentials={"token": "ABC"})
@@ -104,7 +100,7 @@ def test_store_credentials(plugin, write):
         "params": credentials
     }
 
-def test_lost_authentication(plugin, readline, write):
+def test_lost_authentication(plugin, write):
 
     async def couritine():
         plugin.lost_authentication()

tests/test_chat.py

@@ -9,7 +9,7 @@ from galaxy.api.errors import (
     TooManyMessagesSent, IncoherentLastMessage, MessageNotFound
 )
 
-def test_send_message_success(plugin, readline, write):
+def test_send_message_success(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "3",
@@ -20,7 +20,7 @@ def test_send_message_success(plugin, readline, write):
         }
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.send_message.coro.return_value = None
     asyncio.run(plugin.run())
     plugin.send_message.assert_called_with(room_id="14", message="Hello!")
@@ -40,7 +40,7 @@ def test_send_message_success(plugin, readline, write):
     pytest.param(BackendError, 4, "Backend error", id="backend_error"),
     pytest.param(TooManyMessagesSent, 300, "Too many messages sent", id="too_many_messages")
 ])
-def test_send_message_failure(plugin, readline, write, error, code, message):
+def test_send_message_failure(plugin, read, write, error, code, message):
     request = {
         "jsonrpc": "2.0",
         "id": "6",
@@ -51,7 +51,7 @@ def test_send_message_failure(plugin, readline, write, error, code, message):
         }
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.send_message.coro.side_effect = error()
     asyncio.run(plugin.run())
     plugin.send_message.assert_called_with(room_id="15", message="Bye")
@@ -66,7 +66,7 @@ def test_send_message_failure(plugin, readline, write, error, code, message):
         }
     }
 
-def test_mark_as_read_success(plugin, readline, write):
+def test_mark_as_read_success(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "7",
@@ -77,7 +77,7 @@ def test_mark_as_read_success(plugin, readline, write):
         }
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.mark_as_read.coro.return_value = None
     asyncio.run(plugin.run())
     plugin.mark_as_read.assert_called_with(room_id="14", last_message_id="67")
@@ -102,7 +102,7 @@ def test_mark_as_read_success(plugin, readline, write):
         id="incoherent_last_message"
     )
 ])
-def test_mark_as_read_failure(plugin, readline, write, error, code, message):
+def test_mark_as_read_failure(plugin, read, write, error, code, message):
     request = {
         "jsonrpc": "2.0",
         "id": "4",
@@ -113,7 +113,7 @@ def test_mark_as_read_failure(plugin, readline, write, error, code, message):
         }
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.mark_as_read.coro.side_effect = error()
     asyncio.run(plugin.run())
     plugin.mark_as_read.assert_called_with(room_id="18", last_message_id="7")
@@ -128,14 +128,14 @@ def test_mark_as_read_failure(plugin, readline, write, error, code, message):
         }
     }
 
-def test_get_rooms_success(plugin, readline, write):
+def test_get_rooms_success(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "2",
         "method": "import_rooms"
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_rooms.coro.return_value = [
         Room("13", 0, None),
         Room("15", 34, "8")
@@ -162,14 +162,14 @@ def test_get_rooms_success(plugin, readline, write):
         }
     }
 
-def test_get_rooms_failure(plugin, readline, write):
+def test_get_rooms_failure(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "9",
         "method": "import_rooms"
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_rooms.coro.side_effect = UnknownError()
     asyncio.run(plugin.run())
     plugin.get_rooms.assert_called_with()
@@ -184,7 +184,7 @@ def test_get_rooms_failure(plugin, readline, write):
         }
     }
 
-def test_get_room_history_from_message_success(plugin, readline, write):
+def test_get_room_history_from_message_success(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "2",
@@ -195,7 +195,7 @@ def test_get_room_history_from_message_success(plugin, readline, write):
         }
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_room_history_from_message.coro.return_value = [
         Message("13", "149", 1549454837, "Hello"),
         Message("14", "812", 1549454899, "Hi")
@@ -233,7 +233,7 @@ def test_get_room_history_from_message_success(plugin, readline, write):
     pytest.param(BackendError, 4, "Backend error", id="backend_error"),
     pytest.param(MessageNotFound, 500, "Message not found", id="message_not_found")
 ])
-def test_get_room_history_from_message_failure(plugin, readline, write, error, code, message):
+def test_get_room_history_from_message_failure(plugin, read, write, error, code, message):
     request = {
         "jsonrpc": "2.0",
         "id": "7",
@@ -244,7 +244,7 @@ def test_get_room_history_from_message_failure(plugin, readline, write, error, c
         }
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_room_history_from_message.coro.side_effect = error()
     asyncio.run(plugin.run())
     plugin.get_room_history_from_message.assert_called_with(room_id="33", message_id="88")
@@ -259,7 +259,7 @@ def test_get_room_history_from_message_failure(plugin, readline, write, error, c
         }
     }
 
-def test_get_room_history_from_timestamp_success(plugin, readline, write):
+def test_get_room_history_from_timestamp_success(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "7",
@@ -270,7 +270,7 @@ def test_get_room_history_from_timestamp_success(plugin, readline, write):
         }
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_room_history_from_timestamp.coro.return_value = [
         Message("12", "155", 1549454836, "Bye")
     ]
@@ -296,7 +296,7 @@ def test_get_room_history_from_timestamp_success(plugin, readline, write):
         }
     }
 
-def test_get_room_history_from_timestamp_failure(plugin, readline, write):
+def test_get_room_history_from_timestamp_failure(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "3",
@@ -307,7 +307,7 @@ def test_get_room_history_from_timestamp_failure(plugin, readline, write):
         }
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_room_history_from_timestamp.coro.side_effect = UnknownError()
     asyncio.run(plugin.run())
     plugin.get_room_history_from_timestamp.assert_called_with(

tests/test_chunk_messages.py

@@ -0,0 +1,54 @@
+import asyncio
+import json
+
+def test_chunked_messages(plugin, read):
+    request = {
+        "jsonrpc": "2.0",
+        "method": "install_game",
+        "params": {
+            "game_id": "3"
+        }
+    }
+
+    message = json.dumps(request).encode() + b"\n"
+    read.side_effect = [message[:5], message[5:], b""]
+    asyncio.run(plugin.run())
+    plugin.install_game.assert_called_with(game_id="3")
+
+def test_joined_messages(plugin, read):
+    requests = [
+        {
+            "jsonrpc": "2.0",
+            "method": "install_game",
+            "params": {
+                "game_id": "3"
+            }
+        },
+        {
+            "jsonrpc": "2.0",
+            "method": "launch_game",
+            "params": {
+                "game_id": "3"
+            }
+        }
+    ]
+    data = b"".join([json.dumps(request).encode() + b"\n" for request in requests])
+
+    read.side_effect = [data, b""]
+    asyncio.run(plugin.run())
+    plugin.install_game.assert_called_with(game_id="3")
+    plugin.launch_game.assert_called_with(game_id="3")
+
+def test_not_finished(plugin, read):
+    request = {
+        "jsonrpc": "2.0",
+        "method": "install_game",
+        "params": {
+            "game_id": "3"
+        }
+    }
+
+    message = json.dumps(request).encode() # no new line
+    read.side_effect = [message, b""]
+    asyncio.run(plugin.run())
+    plugin.install_game.assert_not_called()

tests/test_friends.py

@@ -5,14 +5,14 @@ from galaxy.api.types import FriendInfo
 from galaxy.api.errors import UnknownError
 
 
-def test_get_friends_success(plugin, readline, write):
+def test_get_friends_success(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "3",
         "method": "import_friends"
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_friends.coro.return_value = [
         FriendInfo("3", "Jan"),
         FriendInfo("5", "Ola")
@@ -33,14 +33,14 @@ def test_get_friends_success(plugin, readline, write):
     }
 
 
-def test_get_friends_failure(plugin, readline, write):
+def test_get_friends_failure(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "3",
         "method": "import_friends"
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_friends.coro.side_effect = UnknownError()
     asyncio.run(plugin.run())
     plugin.get_friends.assert_called_with()

tests/test_game_times.py

@@ -6,17 +6,18 @@ import pytest
 from galaxy.api.types import GameTime
 from galaxy.api.errors import UnknownError, ImportInProgress, BackendError
 
-def test_success(plugin, readline, write):
+def test_success(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "3",
         "method": "import_game_times"
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_game_times.coro.return_value = [
         GameTime("3", 60, 1549550504),
-        GameTime("5", 10, 1549550502)
+        GameTime("5", 10, None),
+        GameTime("7", None, 1549550502),
     ]
     asyncio.run(plugin.run())
     plugin.get_game_times.assert_called_with()
@@ -35,20 +36,23 @@ def test_success(plugin, readline, write):
                 {
                     "game_id": "5",
                     "time_played": 10,
-                    "last_played_time": 1549550502
+                },
+                {
+                    "game_id": "7",
+                     "last_played_time": 1549550502
                 }
             ]
         }
     }
 
-def test_failure(plugin, readline, write):
+def test_failure(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "3",
         "method": "import_game_times"
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_game_times.coro.side_effect = UnknownError()
     asyncio.run(plugin.run())
     plugin.get_game_times.assert_called_with()

tests/test_http.py

@@ -0,0 +1,37 @@
+import asyncio
+from http import HTTPStatus
+
+import aiohttp
+import pytest
+
+from galaxy.api.errors import (
+    AccessDenied, AuthenticationRequired, BackendTimeout, BackendNotAvailable, BackendError, NetworkError,
+    TooManyRequests, UnknownBackendResponse, UnknownError
+)
+from galaxy.http import handle_exception
+
+request_info = aiohttp.RequestInfo("http://o.pl", "GET", {})
+
+@pytest.mark.parametrize(
+    "aiohttp_exception,expected_exception_type",
+    [
+        (asyncio.TimeoutError(), BackendTimeout),
+        (aiohttp.ServerDisconnectedError(), BackendNotAvailable),
+        (aiohttp.ClientConnectionError(), NetworkError),
+        (aiohttp.ContentTypeError(request_info, []), UnknownBackendResponse),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.UNAUTHORIZED), AuthenticationRequired),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.FORBIDDEN), AccessDenied),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.SERVICE_UNAVAILABLE), BackendNotAvailable),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.TOO_MANY_REQUESTS), TooManyRequests),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.INTERNAL_SERVER_ERROR), BackendError),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.NOT_IMPLEMENTED), BackendError),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.BAD_REQUEST), UnknownError),
+        (aiohttp.ClientResponseError(request_info, [], status=HTTPStatus.NOT_FOUND), UnknownError),
+        (aiohttp.ClientError(), UnknownError)
+    ]
+)
+def test_handle_exception(aiohttp_exception, expected_exception_type):
+    with pytest.raises(expected_exception_type):
+        with handle_exception():
+            raise aiohttp_exception
+

tests/test_install_game.py

@@ -1,7 +1,7 @@
 import asyncio
 import json
 
-def test_success(plugin, readline):
+def test_success(plugin, read):
     request = {
         "jsonrpc": "2.0",
         "method": "install_game",
@@ -10,7 +10,7 @@ def test_success(plugin, readline):
         }
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_owned_games.return_value = None
     asyncio.run(plugin.run())
     plugin.install_game.assert_called_with(game_id="3")

tests/test_internal.py

@@ -4,7 +4,7 @@ import json
 from galaxy.api.plugin import Plugin
 from galaxy.api.consts import Platform
 
-def test_get_capabilites(reader, writer, readline, write):
+def test_get_capabilites(reader, writer, read, write):
     class PluginImpl(Plugin): #pylint: disable=abstract-method
         async def get_owned_games(self):
             pass
@@ -16,7 +16,7 @@ def test_get_capabilites(reader, writer, readline, write):
     }
     token = "token"
     plugin = PluginImpl(Platform.Generic, "0.1", reader, writer, token)
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     asyncio.run(plugin.run())
     response = json.loads(write.call_args[0][0])
     assert response == {
@@ -31,13 +31,13 @@ def test_get_capabilites(reader, writer, readline, write):
         }
     }
 
-def test_shutdown(plugin, readline, write):
+def test_shutdown(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "5",
         "method": "shutdown"
     }
-    readline.side_effect = [json.dumps(request)]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     asyncio.run(plugin.run())
     plugin.shutdown.assert_called_with()
     response = json.loads(write.call_args[0][0])
@@ -47,13 +47,13 @@ def test_shutdown(plugin, readline, write):
         "result": None
     }
 
-def test_ping(plugin, readline, write):
+def test_ping(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "7",
         "method": "ping"
     }
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     asyncio.run(plugin.run())
     response = json.loads(write.call_args[0][0])
     assert response == {
@@ -62,7 +62,18 @@ def test_ping(plugin, readline, write):
         "result": None
     }
 
-def test_tick(plugin, readline):
-    readline.side_effect = [""]
+def test_tick_before_handshake(plugin, read):
+    read.side_effect = [b""]
+    asyncio.run(plugin.run())
+    plugin.tick.assert_not_called()
+
+def test_tick_after_handshake(plugin, read):
+    request = {
+        "jsonrpc": "2.0",
+        "id": "6",
+        "method": "initialize_cache",
+        "params": {"data": {}}
+    }
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     asyncio.run(plugin.run())
     plugin.tick.assert_called_with()

tests/test_launch_game.py

@@ -1,7 +1,7 @@
 import asyncio
 import json
 
-def test_success(plugin, readline):
+def test_success(plugin, read):
     request = {
         "jsonrpc": "2.0",
         "method": "launch_game",
@@ -10,7 +10,7 @@ def test_success(plugin, readline):
         }
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_owned_games.return_value = None
     asyncio.run(plugin.run())
     plugin.launch_game.assert_called_with(game_id="3")

tests/test_local_games.py

@@ -7,14 +7,14 @@ from galaxy.api.types import LocalGame
 from galaxy.api.consts import LocalGameState
 from galaxy.api.errors import UnknownError, FailedParsingManifest
 
-def test_success(plugin, readline, write):
+def test_success(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "3",
         "method": "import_local_games"
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
 
     plugin.get_local_games.coro.return_value = [
         LocalGame("1", LocalGameState.Running),
@@ -53,14 +53,14 @@ def test_success(plugin, readline, write):
         pytest.param(FailedParsingManifest, 200, "Failed parsing manifest", id="failed_parsing")
     ],
 )
-def test_failure(plugin, readline, write, error, code, message):
+def test_failure(plugin, read, write, error, code, message):
     request = {
         "jsonrpc": "2.0",
         "id": "3",
         "method": "import_local_games"
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_local_games.coro.side_effect = error()
     asyncio.run(plugin.run())
     plugin.get_local_games.assert_called_with()

tests/test_owned_games.py

@@ -5,14 +5,14 @@ from galaxy.api.types import Game, Dlc, LicenseInfo
 from galaxy.api.consts import LicenseType
 from galaxy.api.errors import UnknownError
 
-def test_success(plugin, readline, write):
+def test_success(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "3",
         "method": "import_owned_games"
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_owned_games.coro.return_value = [
         Game("3", "Doom", None, LicenseInfo(LicenseType.SinglePurchase, None)),
         Game(
@@ -67,14 +67,14 @@ def test_success(plugin, readline, write):
         }
     }
 
-def test_failure(plugin, readline, write):
+def test_failure(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "3",
         "method": "import_owned_games"
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_owned_games.coro.side_effect = UnknownError()
     asyncio.run(plugin.run())
     plugin.get_owned_games.assert_called_with()

tests/test_persistent_cache.py

@@ -0,0 +1,71 @@
+import asyncio
+import json
+
+import pytest
+
+
+def assert_rpc_response(write, response_id, result=None):
+    assert json.loads(write.call_args[0][0]) == {
+        "jsonrpc": "2.0",
+        "id": str(response_id),
+        "result": result
+    }
+
+
+def assert_rpc_request(write, method, params=None):
+    assert json.loads(write.call_args[0][0]) == {
+        "jsonrpc": "2.0",
+        "method": method,
+        "params": {"data": params}
+    }
+
+
+@pytest.fixture
+def cache_data():
+    return {
+        "persistent key": "persistent value",
+        "persistent object": {"answer to everything": 42}
+    }
+
+
+def test_initialize_cache(plugin, read, write, cache_data):
+    request_id = 3
+    request = {
+        "jsonrpc": "2.0",
+        "id": str(request_id),
+        "method": "initialize_cache",
+        "params": {"data": cache_data}
+    }
+    read.side_effect = [json.dumps(request).encode() + b"\n"]
+
+    assert {} == plugin.persistent_cache
+    asyncio.run(plugin.run())
+    plugin.handshake_complete.assert_called_once_with()
+    assert cache_data == plugin.persistent_cache
+    assert_rpc_response(write, response_id=request_id)
+
+
+def test_set_cache(plugin, write, cache_data):
+    async def runner():
+        assert {} == plugin.persistent_cache
+
+        plugin.persistent_cache.update(cache_data)
+        plugin.push_cache()
+
+        assert_rpc_request(write, "push_cache", cache_data)
+        assert cache_data == plugin.persistent_cache
+
+    asyncio.run(runner())
+
+
+def test_clear_cache(plugin, write, cache_data):
+    async def runner():
+        plugin._persistent_cache = cache_data
+
+        plugin.persistent_cache.clear()
+        plugin.push_cache()
+
+        assert_rpc_request(write, "push_cache", {})
+        assert {} == plugin.persistent_cache
+
+    asyncio.run(runner())

tests/test_stream_line_reader.py

@@ -0,0 +1,52 @@
+from unittest.mock import MagicMock
+
+import pytest
+
+from galaxy.reader import StreamLineReader
+from galaxy.unittest.mock import AsyncMock
+
+@pytest.fixture()
+def stream_reader():
+    reader = MagicMock()
+    reader.read = AsyncMock()
+    return reader
+
+@pytest.fixture()
+def read(stream_reader):
+    return stream_reader.read
+
+@pytest.fixture()
+def reader(stream_reader):
+    return StreamLineReader(stream_reader)
+
+@pytest.mark.asyncio
+async def test_message(reader, read):
+    read.return_value = b"a\n"
+    assert await reader.readline() == b"a"
+    read.assert_called_once()
+
+@pytest.mark.asyncio
+async def test_separate_messages(reader, read):
+    read.side_effect = [b"a\n", b"b\n"]
+    assert await reader.readline() == b"a"
+    assert await reader.readline() == b"b"
+    assert read.call_count == 2
+
+@pytest.mark.asyncio
+async def test_connected_messages(reader, read):
+    read.return_value = b"a\nb\n"
+    assert await reader.readline() == b"a"
+    assert await reader.readline() == b"b"
+    read.assert_called_once()
+
+@pytest.mark.asyncio
+async def test_cut_message(reader, read):
+    read.side_effect = [b"a", b"b\n"]
+    assert await reader.readline() == b"ab"
+    assert read.call_count == 2
+
+@pytest.mark.asyncio
+async def test_half_message(reader, read):
+    read.side_effect = [b"a", b""]
+    assert await reader.readline() == b""
+    assert read.call_count == 2

tests/test_uninstall_game.py

@@ -1,7 +1,7 @@
 import asyncio
 import json
 
-def test_success(plugin, readline):
+def test_success(plugin, read):
     request = {
         "jsonrpc": "2.0",
         "method": "uninstall_game",
@@ -10,7 +10,7 @@ def test_success(plugin, readline):
         }
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_owned_games.return_value = None
     asyncio.run(plugin.run())
     plugin.uninstall_game.assert_called_with(game_id="3")

tests/test_users.py

@@ -6,7 +6,7 @@ from galaxy.api.errors import UnknownError
 from galaxy.api.consts import PresenceState
 
 
-def test_get_users_success(plugin, readline, write):
+def test_get_users_success(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "8",
@@ -16,7 +16,7 @@ def test_get_users_success(plugin, readline, write):
         }
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_users.coro.return_value = [
         UserInfo("5", False, "Ula", "http://avatar.png", Presence(PresenceState.Offline))
     ]
@@ -43,7 +43,7 @@ def test_get_users_success(plugin, readline, write):
     }
 
 
-def test_get_users_failure(plugin, readline, write):
+def test_get_users_failure(plugin, read, write):
     request = {
         "jsonrpc": "2.0",
         "id": "12",
@@ -53,7 +53,7 @@ def test_get_users_failure(plugin, readline, write):
         }
     }
 
-    readline.side_effect = [json.dumps(request), ""]
+    read.side_effect = [json.dumps(request).encode() + b"\n", b""]
     plugin.get_users.coro.side_effect = UnknownError()
     asyncio.run(plugin.run())
     plugin.get_users.assert_called_with(user_id_list=["10", "11", "12"])