From 4b86bd09213e932ba5f1568436ef80b89910067b Mon Sep 17 00:00:00 2001 From: Martin Date: Fri, 3 Nov 2023 14:46:02 +0100 Subject: [PATCH] [docs-only] Update search README.md (#7655) References: #7553 (enhancement: improve content extraction stop word cleaning) Making the term `stop word` and the use of the envvar more clear. --- services/search/README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/services/search/README.md b/services/search/README.md index 739fc3a677..520d89516d 100644 --- a/services/search/README.md +++ b/services/search/README.md @@ -70,8 +70,7 @@ When the search service can reach Tika, it begins to read out the content on dem Content extraction and handling the extracted content can be very resource intensive. Content extraction is therefore limited to files with a certain file size. The default limit is 20MB and can be configured using the `SEARCH_CONTENT_EXTRACTION_SIZE_LIMIT` variable. -When extracting the content you can specify whether filler words are ignored or not. -To keep them, the environment variable `SEARCH_EXTRACTOR_TIKA_CLEAN_STOP_WORDS` must be set to false. +When extracting content, you can specify whether [stop words](https://en.wikipedia.org/wiki/Stop_word) like `I`, `you`, `the` are ignored or not. Noramlly, these stop words are removed automatically. To keep them, the environment variable `SEARCH_EXTRACTOR_TIKA_CLEAN_STOP_WORDS` must be set to `false`. When using the Tika container and docker-compose, consider the following: