From 778c84cf17847dae98bed6d25d3484fe605605ff Mon Sep 17 00:00:00 2001 From: David Christofas Date: Tue, 4 Oct 2022 16:38:01 +0200 Subject: [PATCH 1/5] add a readme file for the thumbnails service --- services/thumbnails/README.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 services/thumbnails/README.md diff --git a/services/thumbnails/README.md b/services/thumbnails/README.md new file mode 100644 index 000000000..b9cd01c8b --- /dev/null +++ b/services/thumbnails/README.md @@ -0,0 +1,29 @@ +# Thumbnails service + +The thumbnails service provides methods to generate thumbnails for various files. +It retrieves the sources at the location where the user files are stored and saves the tumbnails at the location where system files are stored. Those locations have defaults but can be manually defined via environment variables. The relevant environment variables are: + +- `OCIS_BASE_DATA_PATH` which will contain system relevant data and +- `STORAGE_USERS_OCIS_ROOT` which will contain the user source data. +- `THUMBNAILS_FILESYSTEMSTORAGE_ROOT` used if thumnails should be separated from system files. + +For details and defaults see the documentation. + +Thumbnails can be generated for the following file types: + +- png +- jpg +- gif +- tiff +- bmp +- txt + +Thumbnails can either be generated as `png`, `jpg` or `gif` files, various resolutions can be defined via `THUMBNAILS_RESOLUTIONS`. + +--- + +**NOTE** + +Since source files need to be loaded into memory when generating thumbnails, large source files could potentially crash this service if there is insufficient memory available. For bigger instances, this service can be dedicated to own servers with more memory when using container orchestration deployment methods. + +--- From 2836395a299a8d05500d2287d59cc697631e1e6d Mon Sep 17 00:00:00 2001 From: mmattel Date: Tue, 25 Oct 2022 17:21:50 +0200 Subject: [PATCH 2/5] update and text restructuring --- services/thumbnails/README.md | 55 ++++++++++++++++++++++++++--------- 1 file changed, 42 insertions(+), 13 deletions(-) diff --git a/services/thumbnails/README.md b/services/thumbnails/README.md index b9cd01c8b..cfc8855e6 100644 --- a/services/thumbnails/README.md +++ b/services/thumbnails/README.md @@ -1,15 +1,28 @@ -# Thumbnails service +# Thumbnails Service -The thumbnails service provides methods to generate thumbnails for various files. -It retrieves the sources at the location where the user files are stored and saves the tumbnails at the location where system files are stored. Those locations have defaults but can be manually defined via environment variables. The relevant environment variables are: +The thumbnails service provides methods to generate thumbnails for various files and resolutions based on requests. It retrieves the sources at the location where the user files are stored and saves the tumbnails where system files are stored. Those locations have defaults but can be manually defined via environment variables. -- `OCIS_BASE_DATA_PATH` which will contain system relevant data and -- `STORAGE_USERS_OCIS_ROOT` which will contain the user source data. -- `THUMBNAILS_FILESYSTEMSTORAGE_ROOT` used if thumnails should be separated from system files. +## File Locations Overview -For details and defaults see the documentation. +The relevant environment variables defining file locations are: -Thumbnails can be generated for the following file types: +- (1) `OCIS_BASE_DATA_PATH` +- (2) `STORAGE_USERS_OCIS_ROOT` +- (3) `THUMBNAILS_FILESYSTEMSTORAGE_ROOT` + +(1) ... Having a default set by the ocis code, but if defined, used as base path for other services. +(2) ... Source files, defaults to (1) plus path component, but can be freely defined if required. +(3) ... Target files, defaults to (1) plus path component, but can be freely defined if required. + +For details and defaults for these environment variables see the ocis admin documentation. + +## Thumbnail Location + +It may be benificial to define the location of the thumbnails other than at system files. This is due the fact that storing thumbnails can consume a lot of space over time which not necessarily needs to reside on the same partition or mount or expensive drives. + +## Thumbnail Source File Types + +Thumbnails can be generated from the following source file types: - png - jpg @@ -18,12 +31,28 @@ Thumbnails can be generated for the following file types: - bmp - txt -Thumbnails can either be generated as `png`, `jpg` or `gif` files, various resolutions can be defined via `THUMBNAILS_RESOLUTIONS`. +The thumbnail service retrieves source files using the information provided by the backend. The Linux backend identifies source files usually based on the extension. ---- +If a file type was not properly assigned or the type identification failed, thumbnail generation will fail and an error will be logged. -**NOTE** +## Thumbnail Target File Types -Since source files need to be loaded into memory when generating thumbnails, large source files could potentially crash this service if there is insufficient memory available. For bigger instances, this service can be dedicated to own servers with more memory when using container orchestration deployment methods. +Thumbnails can either be generated as `png`, `jpg` or `gif` files. These types are hardcoded and no other types can be requested. A requestor like another service or a client can request one of the avialable types to be generated. If more than one type is required, each type must be requested individaully. ---- +## Thumbnail Resolution + +Various resolutions can be defined via `THUMBNAILS_RESOLUTIONS`. A requestor can request any arbitrary resolution and the thumbnail service will use the one closest to the requested resolution. If more than one resolution is required, each resolution must be requested individaully. + +**Example:** + +Requested: 18x12 +Available: 30x20, 15x10, 9x6 +Returned: 15x10 + +## Deleting Thumbnails + +As of now, there is no automatism deleting thumbnails automatically. This is especially true when a source file gets deleted or moved. This situation will be solved in a later stage. For the time being, if you run short on physical thumbnails space, you have to manually delete the thumbnail store to free space. Thumbnails will then be recreated on request. + +## Memory Considerations + +Since source files need to be loaded into memory when generating thumbnails, large source files could potentially crash this service if there is insufficient memory available. For bigger instances when using container orchestration deployment methods, this service can be dedicated to own servers with more memory. From 9436dfe4d196a4c73509e8c108ed520f3e6db903 Mon Sep 17 00:00:00 2001 From: Edith Parzefall Date: Wed, 26 Oct 2022 09:10:18 +0200 Subject: [PATCH 3/5] Apply suggestions from code review Co-authored-by: Phil Davis --- services/thumbnails/README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/services/thumbnails/README.md b/services/thumbnails/README.md index cfc8855e6..1772bd318 100644 --- a/services/thumbnails/README.md +++ b/services/thumbnails/README.md @@ -1,6 +1,6 @@ # Thumbnails Service -The thumbnails service provides methods to generate thumbnails for various files and resolutions based on requests. It retrieves the sources at the location where the user files are stored and saves the tumbnails where system files are stored. Those locations have defaults but can be manually defined via environment variables. +The thumbnails service provides methods to generate thumbnails for various files and resolutions based on requests. It retrieves the sources at the location where the user files are stored and saves the thumbnails where system files are stored. Those locations have defaults but can be manually defined via environment variables. ## File Locations Overview @@ -18,7 +18,7 @@ For details and defaults for these environment variables see the ocis admin docu ## Thumbnail Location -It may be benificial to define the location of the thumbnails other than at system files. This is due the fact that storing thumbnails can consume a lot of space over time which not necessarily needs to reside on the same partition or mount or expensive drives. +It may be beneficial to define the location of the thumbnails to be other than the default (with system files). This is due the fact that storing thumbnails can consume a lot of space over time which not necessarily needs to reside on the same partition or mount or expensive drives. ## Thumbnail Source File Types @@ -37,11 +37,11 @@ If a file type was not properly assigned or the type identification failed, thum ## Thumbnail Target File Types -Thumbnails can either be generated as `png`, `jpg` or `gif` files. These types are hardcoded and no other types can be requested. A requestor like another service or a client can request one of the avialable types to be generated. If more than one type is required, each type must be requested individaully. +Thumbnails can either be generated as `png`, `jpg` or `gif` files. These types are hardcoded and no other types can be requested. A requestor, like another service or a client, can request one of the available types to be generated. If more than one type is required, each type must be requested individually. ## Thumbnail Resolution -Various resolutions can be defined via `THUMBNAILS_RESOLUTIONS`. A requestor can request any arbitrary resolution and the thumbnail service will use the one closest to the requested resolution. If more than one resolution is required, each resolution must be requested individaully. +Various resolutions can be defined via `THUMBNAILS_RESOLUTIONS`. A requestor can request any arbitrary resolution and the thumbnail service will use the one closest to the requested resolution. If more than one resolution is required, each resolution must be requested individually. **Example:** @@ -51,8 +51,8 @@ Returned: 15x10 ## Deleting Thumbnails -As of now, there is no automatism deleting thumbnails automatically. This is especially true when a source file gets deleted or moved. This situation will be solved in a later stage. For the time being, if you run short on physical thumbnails space, you have to manually delete the thumbnail store to free space. Thumbnails will then be recreated on request. +As of now, there is no automated thumbnail deletion. This is especially true when a source file gets deleted or moved. This situation will be solved at a later stage. For the time being, if you run short on physical thumbnails space, you have to manually delete the thumbnail store to free space. Thumbnails will then be recreated on request. ## Memory Considerations -Since source files need to be loaded into memory when generating thumbnails, large source files could potentially crash this service if there is insufficient memory available. For bigger instances when using container orchestration deployment methods, this service can be dedicated to own servers with more memory. +Since source files need to be loaded into memory when generating thumbnails, large source files could potentially crash this service if there is insufficient memory available. For bigger instances when using container orchestration deployment methods, this service can be dedicated to its own server(s) with more memory. From 152967320d68cf74aaef3465e8ab8f306d9b9786 Mon Sep 17 00:00:00 2001 From: Edith Parzefall Date: Wed, 26 Oct 2022 09:16:05 +0200 Subject: [PATCH 4/5] Update services/thumbnails/README.md --- services/thumbnails/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/services/thumbnails/README.md b/services/thumbnails/README.md index 1772bd318..d45b64bdb 100644 --- a/services/thumbnails/README.md +++ b/services/thumbnails/README.md @@ -10,7 +10,7 @@ The relevant environment variables defining file locations are: - (2) `STORAGE_USERS_OCIS_ROOT` - (3) `THUMBNAILS_FILESYSTEMSTORAGE_ROOT` -(1) ... Having a default set by the ocis code, but if defined, used as base path for other services. +(1) ... Having a default set by the Infinite Scale code, but if defined, used as base path for other services. (2) ... Source files, defaults to (1) plus path component, but can be freely defined if required. (3) ... Target files, defaults to (1) plus path component, but can be freely defined if required. From c67b45730873bcf93404d1bb2a06f1af417ccc0a Mon Sep 17 00:00:00 2001 From: Martin Date: Wed, 26 Oct 2022 09:17:14 +0200 Subject: [PATCH 5/5] Update services/thumbnails/README.md --- services/thumbnails/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/services/thumbnails/README.md b/services/thumbnails/README.md index d45b64bdb..5592a601f 100644 --- a/services/thumbnails/README.md +++ b/services/thumbnails/README.md @@ -43,7 +43,7 @@ Thumbnails can either be generated as `png`, `jpg` or `gif` files. These types a Various resolutions can be defined via `THUMBNAILS_RESOLUTIONS`. A requestor can request any arbitrary resolution and the thumbnail service will use the one closest to the requested resolution. If more than one resolution is required, each resolution must be requested individually. -**Example:** +Example: Requested: 18x12 Available: 30x20, 15x10, 9x6