mirror/opencloud
1
0
Fork 0
mirror of https://github.com/opencloud-eu/opencloud.git synced 2026-01-20 20:08:07 -05:00
Code Issues Packages Projects Releases Wiki Activity

docs(test): update testing docs as per the new changes

Browse Source 
Signed-off-by: Saw-jan <saw.jan.grg3e@gmail.com>
This commit is contained in:
Saw-jan
2026-01-16 17:55:40 +05:45
parent 287cb04aa9
commit fb01cd8d79
1 changed files with 122 additions and 184 deletions
Download Patch File Download Diff File Expand all files Collapse all files

306
tests/README.md
View File

@@ -4,183 +4,119 @@ To run tests in the test suite you have two options. You may go the easy way and
Both ways to run tests with the test suites are described here.
## Table of Contents
- [Running Test Suite in Docker](#running-test-suite-in-docker)
- [Different Command Examples](#different-command-examples)
- [Running Tests](#running-tests)
- [Skip Local Image Build While Running Tests](#skip-local-image-build-while-running-tests)
- [Check Test Logs](#check-test-logs)
- [Cleanup the Setup](#cleanup-the-setup)
- [Running Test Suite in Local Environment](#running-test-suite-in-local-environment)
- [Running Tests With And Without `remote.php`](#running-tests-with-and-without-remotephp)
- [Running ENV Config Tests (@env-Config)](#running-env-config-tests-env-config)
- [Running Test Suite With Email Service (@email)](#running-test-suite-with-email-service-email)
- [Running Test Suite With Tika Service (@tikaServiceNeeded)](#running-test-suite-with-tika-service-tikaserviceneeded)
- [Running Test Suite With Antivirus Service (@antivirus)](#running-test-suite-with-antivirus-service-antivirus)
- [Running Test Suite With Federated Sharing (@ocm)](#running-test-suite-with-federated-sharing-ocm)
- [Running Text Preview Tests Containing Unicode Characters](#running-text-preview-tests-containing-unicode-characters)
- [Running All API Tests Locally](#running-all-api-tests-locally)
- [Running WOPI Validator Tests](#running-wopi-validator-tests)
## Running Test Suite in Docker
Let's see what is available. Invoke the following command from within the root of the OpenCloud repository.
Check the available commands and environment variables with:
```bash
make -C tests/acceptance/docker help
```
Basically we have two sources for feature tests and test suites:
### Different Command Examples
- [OpenCloud feature test and test suites](https://github.com/opencloud-eu/opencloud/tree/main/tests/acceptance/features)
- [tests and test suites transferred from core, they have prefix coreApi](https://github.com/opencloud-eu/opencloud/tree/main/tests/acceptance/features)
#### Running Tests
At the moment, both can be applied to OpenCloud.
We can run a single feature or a single test suite with different storage drivers.
As a storage backend, we support the OpenCloud native storage, also called `decomposed`. This stores files directly on disk. Along with that we also provide `decomposeds3`, `posix` storage drivers.
1. Run a specific feature file:
You can invoke two types of test suite runs:
```bash
BEHAT_FEATURE='tests/acceptance/features/apiGraphUserGroup/createUser.feature' \
make -C tests/acceptance/docker run-api-tests
```
- run a full test suite, which consists of multiple feature tests
- run a single feature or single scenario in a feature
or a single scenario in a feature:
### Run Full Test Suite
```bash
BEHAT_FEATURE='tests/acceptance/features/apiGraphUserGroup/createUser.feature:24' \
make -C tests/acceptance/docker run-api-tests
```
#### Local OpenCloud Tests (prefix `api`)
2. Run a specific test suite:
The names of the full test suite make targets have the same naming as in the CI pipeline. See the available local OpenCloud specific test suites [here](https://github.com/opencloud-eu/opencloud/tree/main/tests/acceptance/features). They can be run with `decomposed` storage, `decomposeds3` storage and `posix` storage
```bash
BEHAT_SUITE='apiGraphUserGroup' \
make -C tests/acceptance/docker run-api-tests
```
For example, command:
3. Run with different storage driver (default is `posix`):
```bash
STORAGE_DRIVER='decomposed' \
BEHAT_SUITE='apiGraphUserGroup' \
make -C tests/acceptance/docker run-api-tests
```
4. Run the tests that require an email server (tests tagged with `@email`). Provide `START_EMAIL=true` while running the tests:
```bash
START_EMAIL=true \
BEHAT_FEATURE='tests/acceptance/features/apiNotification/emailNotification.feature' \
make -C tests/acceptance/docker run-api-tests
```
5. Run the tests that require tika service (tests tagged with `@tikaServiceNeeded`). Provide `START_TIKA=true` while running the tests:
```bash
START_TIKA=true \
BEHAT_FEATURE='tests/acceptance/features/apiSearchContent/contentSearch.feature' \
make -C tests/acceptance/docker run-api-tests
```
6. Run the tests that require an antivirus service (tests tagged with `@antivirus`). Provide `START_ANTIVIRUS=true` while running the tests:
```bash
START_ANTIVIRUS=true \
BEHAT_FEATURE='tests/acceptance/features/apiAntivirus/antivirus.feature' \
make -C tests/acceptance/docker run-api-tests
```
7. Run the wopi tests. Provide `ENABLE_WOPI=true` while running the tests:
```bash
ENABLE_WOPI=true \
BEHAT_FEATURE='tests/acceptance/features/apiCollaboration/checkFileInfo.feature' \
make -C tests/acceptance/docker run-api-tests
```
If the tests fail due to the servers not being ready or any other reasons, you can re-run the tests using the following command:
> Here, you can utilize `BEHAT_FEATURE` and `BEHAT_SUITE` environment variables to specify the feature file or test suite you want to run again.
```bash
make -C tests/acceptance/docker localApiTests-apiGraph-decomposed
BEHAT_FEATURE='tests/acceptance/features/apiGraphUserGroup/createUser.feature:24' \
make -C tests/acceptance/docker run-api-tests
```
runs the same tests as the `localApiTests-apiGraph-decomposed` CI pipeline, which runs the OpenCloud test suite "apiGraph" against the OpenCloud server with `decomposed` storage.
#### Skip Local Image Build While Running Tests
command:
```bash
make -C tests/acceptance/docker localApiTests-apiGraph-decomposeds3
```
runs the OpenCloud test suite `apiGraph` against the OpenCloud server with `decomposeds3` storage.
And command:
```bash
make -C tests/acceptance/docker localApiTests-apiGraph-posix
```
runs the OpenCloud test suite `apiGraph` against the OpenCloud server with `posix` storage.
Note:
While running the tests, OpenCloud server is started with [ocwrapper](https://github.com/opencloud-eu/opencloud/blob/main/tests/ocwrapper/README.md) (i.e. `WITH_WRAPPER=true`) by default. In order to run the tests without ocwrapper, provide `WITH_WRAPPER=false` when running the tests. For example:
```bash
WITH_WRAPPER=false \
BEHAT_FEATURE='tests/acceptance/features/apiGraphUserGroup/createUser.feature:26' \
make -C tests/acceptance/docker test-opencloud-feature-decomposed-storage
```
But some test suites that are tagged with `@env-config` require the OpenCloud server to be run with ocwrapper. So, running those tests require `WITH_WRAPPER=true` (default setting).
Note:
To run the tests that require an email server (tests tagged with `@email`), you need to provide `START_EMAIL=true` while running the tests.
```bash
START_EMAIL=true \
BEHAT_FEATURE='tests/acceptance/features/apiNotification/emailNotification.feature' \
make -C tests/acceptance/docker test-opencloud-feature-decomposed-storage
```
Note:
To run the tests that require tika service (tests tagged with `@tikaServiceNeeded`), you need to provide `START_TIKA=true` while running the tests.
```bash
START_TIKA=true \
BEHAT_FEATURE='tests/acceptance/features/apiSearchContent/contentSearch.feature' \
make -C tests/acceptance/docker test-opencloud-feature-decomposed-storage
```
Note:
To run the tests that require an antivirus service (tests tagged with `@antivirus`), you need to provide the following environment variables while running the tests.
```bash
START_ANTIVIRUS=true \
OC_ASYNC_UPLOADS=true \
OC_ADD_RUN_SERVICES=antivirus \
POSTPROCESSING_STEPS=virusscan \
BEHAT_FEATURE='tests/acceptance/features/apiAntivirus/antivirus.feature' \
make -C tests/acceptance/docker test-opencloud-feature-decomposed-storage
```
#### Tests Transferred From Core (prefix `coreApi`)
Command `make -C tests/acceptance/docker Core-API-Tests-decomposed-storage-3` runs the same tests as the `Core-API-Tests-decomposed-storage-3` CI pipeline, which runs the third (out of ten) test suite groups transferred from core against the OpenCloud server with `decomposed` storage.
And `make -C tests/acceptance/docker Core-API-Tests-decomposeds3-storage-3` runs the third (out of ten) test suite groups transferred from core against the OpenCloud server with `decomposeds3` storage.
### Run Single Feature Test
The tests for a single feature (a feature file) can also be run against the different storage backends. To do that, multiple make targets with the schema **test-_\<test-source\>_-feature-_\<storage-backend\>_** are available. To select a single feature you have to add an additional `BEHAT_FEATURE=<path-to-feature-file>` parameter when invoking the make command.
For example;
```bash
BEHAT_FEATURE='tests/acceptance/features/apiGraphUserGroup/createUser.feature' \
make -C tests/acceptance/docker test-opencloud-feature-decomposed-storage
```
Note:
`BEHAT_FEATURE` must be pointing to a valid feature file
And to run a single scenario in a feature, you can do:
Note:
A specific scenario from a feature can be run by adding `:<line-number>` at the end of the feature file path. For example, to run the scenario at line 26 of the feature file `apiGraphUserGroup/createUser.feature`, simply add the line number like this: `apiGraphUserGroup/createUser.feature:26`. Note that the line numbers mentioned in the examples might not always point to a scenario, so always check the line numbers before running the test.
```bash
BEHAT_FEATURE='tests/acceptance/features/apiGraphUserGroup/createUser.feature:26' \
make -C tests/acceptance/docker test-opencloud-feature-decomposed-storage
```
Similarly, with `decomposeds3` storage;
```bash
# run a whole feature
BEHAT_FEATURE='tests/acceptance/features/apiGraphUserGroup/createUser.feature' \
make -C tests/acceptance/docker test-opencloud-feature-decomposeds3-storage
# run a single scenario
BEHAT_FEATURE='tests/acceptance/features/apiGraphUserGroup/createUser.feature:26' \
make -C tests/acceptance/docker test-opencloud-feature-decomposeds3-storage
```
In the same way, tests transferred from core can be run as:
```bash
# run a whole feature
BEHAT_FEATURE='tests/acceptance/features/coreApiAuth/webDavAuth.feature' \
make -C tests/acceptance/docker test-core-feature-decomposed-storage
# run a single scenario
BEHAT_FEATURE='tests/acceptance/features/coreApiAuth/webDavAuth.feature:15' \
make -C tests/acceptance/docker test-core-feature-decomposed-storage
```
Note:
The test suites transferred from core have `coreApi` prefixed
### Running Collaboration API Tests Locally
Running collaboration API tests requires extra services. You can enable the required services by providing the environment variable `ENABLE_WOPI=true` while running the tests.
```bash
ENABLE_WOPI=true \
BEHAT_FEATURE='tests/acceptance/features/apiCollaboration/checkFileInfo.feature' \
make -C tests/acceptance/docker test-opencloud-feature-posix-storage
```
If the tests fail due to the servers not being ready, you can re-run the tests using the following command:
```bash
BEHAT_FEATURE='tests/acceptance/features/apiCollaboration/checkFileInfo.feature' \
make -C tests/acceptance/docker run-test
```
### OpenCloud Image to Be Tested (Skip Local Image Build)
By default, the tests will be run against the docker image built from your current working state of the OpenCloud repository. For some purposes it might also be handy to use an OpenCloud image from Docker Hub. Therefore, you can provide the optional flag `OC_IMAGE_TAG=...` which must contain an available docker tag of the [opencloud-eu/opencloud registry on Docker Hub](https://hub.docker.com/r/opencloud-eu/opencloud) (e.g. 'latest').
While running the tests, opencloud docker image is built with `opencloudeu/opencloud:dev` tag. If you want to skip building the local image, you can use `OC_IMAGE_TAG` env which must contain an available docker tag of the [opencloudeu/opencloud registry on Docker Hub](https://hub.docker.com/r/opencloudeu/opencloud) (e.g. 'latest').
```bash
OC_IMAGE_TAG=latest \
make -C tests/acceptance/docker localApiTests-apiGraph-opencloud
BEHAT_FEATURE='tests/acceptance/features/apiGraphUserGroup/createUser.feature' \
make -C tests/acceptance/docker run-api-tests
```
### Test Log Output
#### Check Test Logs
While a test is running or when it is finished, you can attach to the logs generated by the tests.
@@ -188,15 +124,15 @@ While a test is running or when it is finished, you can attach to the logs gener
make -C tests/acceptance/docker show-test-logs
```
Note:
The log output is opened in `less`. You can navigate up and down with your cursors. By pressing "F" you can follow the latest line of the output.
> [!NOTE]
> The log output is opened in `less`. You can navigate up and down with your cursors. By pressing "F" you can follow the latest line of the output.
### Cleanup
#### Cleanup the Setup
During testing we start a redis and OpenCloud docker container. These will not be stopped automatically. You can stop them with:
Run the following command to clean all the resources created while running the tests:
```bash
make -C tests/acceptance/docker clean
make -C tests/acceptance/docker clean-all
```
## Running Test Suite in Local Environment
@@ -286,7 +222,7 @@ If you want to work on a specific issue
5. remove those tests from the expected failures file
6. make a PR that has the fixed code, and the relevant lines removed from the expected failures file.
## Running Tests With And Without `remote.php`
### Running Tests With And Without `remote.php`
By default, the tests are run with `remote.php` enabled. If you want to run the tests without `remote.php`, you can disable it by setting the environment variable `WITH_REMOTE_PHP=false` while running the tests.
@@ -296,11 +232,11 @@ TEST_SERVER_URL="https://localhost:9200" \
make test-acceptance-api
```
## Running ENV Config Tests (@env-Config)
### Running ENV Config Tests (@env-Config)
Test suites tagged with `@env-config` are used to test the environment variables that are used to configure OpenCloud. These tests are special tests that require the OpenCloud server to be run using [ocwrapper](https://github.com/opencloud-eu/opencloud/blob/main/tests/ocwrapper/README.md).
### Run OpenCloud With ocwrapper
#### Run OpenCloud With ocwrapper
```bash
# working dir: OpenCloud repo root dir
@@ -318,7 +254,7 @@ PROXY_ENABLE_BASIC_AUTH=true \
./bin/ocwrapper serve --bin=../../opencloud/bin/opencloud
```
### Run the Tests
#### Run the Tests
```bash
OC_WRAPPER_URL=http://localhost:5200 \
@@ -327,7 +263,7 @@ BEHAT_FEATURE=tests/acceptance/features/apiAsyncUpload/delayPostprocessing.featu
make test-acceptance-api
```
### Writing New ENV Config Tests
#### Writing New ENV Config Tests
While writing tests for a new OpenCloud ENV configuration, please make sure to follow these guidelines:
@@ -335,11 +271,11 @@ While writing tests for a new OpenCloud ENV configuration, please make sure to f
2. Use `OcConfigHelper.php` for helper functions - provides functions to reconfigure the running OpenCloud instance.
3. Recommended: add the new step implementations in `OcConfigContext.php`
## Running Test Suite With Email Service (@email)
### Running Test Suite With Email Service (@email)
Test suites that are tagged with `@email` require an email service. We use inbucket as the email service in our tests.
### Setup Inbucket
#### Setup Inbucket
Run the following command to setup inbucket
@@ -347,7 +283,7 @@ Run the following command to setup inbucket
docker run -d -p9000:9000 -p2500:2500 --name inbucket inbucket/inbucket
```
### Run OpenCloud
#### Run OpenCloud
Documentation for environment variables is available [here](https://docs.opencloud.eu/services/notifications/#environment-variables)
@@ -366,7 +302,7 @@ NOTIFICATIONS_SMTP_SENDER="OpenCloud <noreply@example.com>" \
opencloud/bin/opencloud server
```
### Run the Acceptance Test
#### Run the Acceptance Test
Run the acceptance test with the following command:
@@ -378,11 +314,11 @@ BEHAT_FEATURE="tests/acceptance/features/apiNotification/emailNotification.featu
make test-acceptance-api
```
## Running Test Suite With Tika Service (@tikaServiceNeeded)
### Running Test Suite With Tika Service (@tikaServiceNeeded)
Test suites that are tagged with `@tikaServiceNeeded` require tika service.
### Setup Tika Service
#### Setup Tika Service
Run the following docker command to setup tika service
@@ -390,7 +326,7 @@ Run the following docker command to setup tika service
docker run -d -p 127.0.0.1:9998:9998 apache/tika
```
### Run OpenCloud
#### Run OpenCloud
TODO: Documentation related to the content based search and tika extractor will be added later.
@@ -408,7 +344,7 @@ SEARCH_EXTRACTOR_CS3SOURCE_INSECURE=true \
opencloud/bin/opencloud server
```
### Run the Acceptance Test
#### Run the Acceptance Test
Run the acceptance test with the following command:
@@ -418,15 +354,15 @@ BEHAT_FEATURE="tests/acceptance/features/apiSearchContent/contentSearch.feature"
make test-acceptance-api
```
## Running Test Suite With Antivirus Service (@antivirus)
### Running Test Suite With Antivirus Service (@antivirus)
Test suites that are tagged with `@antivirus` require antivirus service. TODO The available antivirus and the configuration related to them will be added latert. This documentation is only going to use `clamav` as antivirus.
### Setup clamAV
#### Setup clamAV
#### 1. Setup Locally
**Option 1. Setup Locally**
##### Linux OS user
Linux OS user:
Run the following command to set up calmAV and clamAV daemon
@@ -443,7 +379,7 @@ sudo service clamav-daemon status
Note:
The commands are ubuntu specific and may differ according to your system. You can find information related to installation of clamAV in their official documentation [here](https://docs.clamav.net/manual/Installing/Packages.html).
##### Mac OS user
Mac OS user:
Install ClamAV using [here](https://gist.github.com/mendozao/3ea393b91f23a813650baab9964425b9)
Start ClamAV daemon
@@ -452,7 +388,7 @@ Start ClamAV daemon
/your/location/to/brew/Cellar/clamav/1.1.0/sbin/clamd
```
#### 2. Setup clamAV With Docker
**Option 2. Setup clamAV With Docker**
Run `clamAV` through docker
@@ -460,7 +396,7 @@ Run `clamAV` through docker
docker run -d -p 3310:3310 opencloudeu/clamav-ci:latest
```
### Run OpenCloud
#### Run OpenCloud
As `antivirus` service is not enabled by default we need to enable the service while running OpenCloud server. We also need to enable `async upload` and as virus scan is performed in post-processing step, we need to set it as well. Documentation for environment variables related to antivirus is available [here](https://docs.opencloud.eu/services/antivirus/#environment-variables)
@@ -486,7 +422,7 @@ For antivirus running localy on Linux OS, use `ANTIVIRUS_CLAMAV_SOCKET= "/var/ru
For antivirus running localy on Mac OS, use `ANTIVIRUS_CLAMAV_SOCKET= "/tmp/clamd.sock"`.
For antivirus running with docker, use `ANTIVIRUS_CLAMAV_SOCKET= "tcp://host.docker.internal:3310"`
### Create virus files
#### Create Virus Files
The antivirus tests require EICAR test files which are not stored in the repository
They are generated dynamically when needed for testing.
@@ -505,11 +441,11 @@ BEHAT_FEATURE="tests/acceptance/features/apiAntivirus/antivirus.feature" \
make test-acceptance-api
```
## Running Test Suite With Federated Sharing (@ocm)
### Running Test Suite With Federated Sharing (@ocm)
Test suites that are tagged with `@ocm` require running two different OpenCloud instances. TODO More detailed information and configuration related to it will be added later.
### Setup First OpenCloud Instance
#### Setup First OpenCloud Instance
```bash
# init OpenCloud
@@ -531,15 +467,15 @@ opencloud/bin/opencloud server
The first OpenCloud instance should be available at: https://localhost:9200/
### Setup Second OpenCloud Instance
#### Setup Second OpenCloud Instance
You can run the second OpenCloud instance in two ways:
#### Using `.vscode/launch.json`
**Option 1. Using `.vscode/launch.json`**
From the `Run and Debug` panel of VSCode, select `Fed OpenCloud Server` and start the debugger.
#### Using env file
**Option 2. Using env file**
```bash
# init OpenCloud
@@ -567,7 +503,7 @@ BEHAT_FEATURE="tests/acceptance/features/apiOcm/ocm.feature" \
make test-acceptance-api
```
## Running Text Preview Tests Containing Unicode Characters
### Running Text Preview Tests Containing Unicode Characters
There are some tests that check the text preview of files containing Unicode characters. The OpenCloud server by default cannot generate the thumbnail of such files correctly but it provides an environment variable to allow the use of custom fonts that support Unicode characters. So to run such tests successfully, we have to run the OpenCloud server with this environment variable.
@@ -627,7 +563,9 @@ STORAGE_DRIVER=posix ./tests/acceptance/run_api_tests.sh
TEST_GROUP=BaseWopiViewing docker compose -f tests/acceptance/docker/src/wopi-validator-test.yml up -d
```
### for macOS use arm image
### Run Test (macOS)
Use the arm image for macOS to run the validator tests.
```bash
WOPI_VALIDATOR_IMAGE=scharfvi/wopi-validator \