opencloud/tests

Acceptance Testing

To run tests in the test suite you have two options. You may go the easy way and just run the test suite in docker. But for some tasks you could also need to install the test suite natively, which requires a little more setup since PHP and some dependencies need to be installed.

Both ways to run tests with the test suites are described here.

Table of Contents

• Running Test Suite in Docker
  • Running API Tests
    • Run Tests
    • Skip Local Image Build While Running Tests
    • Check Test Logs
    • Cleanup the Setup
  • Running WOPI Validator Tests
• Running Test Suite in Local Environment
  • Running Tests With And Without remote.php
  • Running ENV Config Tests (@env-Config)
  • Running Test Suite With Email Service (@email)
  • Running Test Suite With Tika Service (@tikaServiceNeeded)
  • Running Test Suite With Antivirus Service (@antivirus)
  • Running Test Suite With Federated Sharing (@ocm)
  • Running Text Preview Tests Containing Unicode Characters
• Running All API Tests Locally

Running Test Suite in Docker

Check the available commands and environment variables with:

make -C tests/acceptance/docker help

Running API Tests

Run Tests

We can run a single feature or a single test suite with different storage drivers.

1. Run a specific feature file:

   BEHAT_FEATURE='tests/acceptance/features/apiGraphUserGroup/createUser.feature' \
   make -C tests/acceptance/docker run-api-tests
   

   or a single scenario in a feature:

   BEHAT_FEATURE='tests/acceptance/features/apiGraphUserGroup/createUser.feature:24' \
   make -C tests/acceptance/docker run-api-tests
   

2. Run a specific test suite:

   BEHAT_SUITE='apiGraphUserGroup' \
   make -C tests/acceptance/docker run-api-tests
   

3. Run with different storage driver (default is posix):

   STORAGE_DRIVER='posix' \
   BEHAT_SUITE='apiGraphUserGroup' \
   make -C tests/acceptance/docker run-api-tests
   

4. Run the tests that require an email server (tests tagged with @email). Provide START_EMAIL=true while running the tests:

   START_EMAIL=true \
   BEHAT_FEATURE='tests/acceptance/features/apiNotification/emailNotification.feature' \
   make -C tests/acceptance/docker run-api-tests
   

5. Run the tests that require tika service (tests tagged with @tikaServiceNeeded). Provide START_TIKA=true while running the tests:

   START_TIKA=true \
   BEHAT_FEATURE='tests/acceptance/features/apiSearchContent/contentSearch.feature' \
   make -C tests/acceptance/docker run-api-tests
   

6. Run the tests that require an antivirus service (tests tagged with @antivirus). Provide START_ANTIVIRUS=true while running the tests:

   START_ANTIVIRUS=true \
   BEHAT_FEATURE='tests/acceptance/features/apiAntivirus/antivirus.feature' \
   make -C tests/acceptance/docker run-api-tests
   

7. Run the wopi tests. Provide ENABLE_WOPI=true while running the tests:

   ENABLE_WOPI=true \
   BEHAT_FEATURE='tests/acceptance/features/apiCollaboration/checkFileInfo.feature' \
   make -C tests/acceptance/docker run-api-tests
   

If the tests fail due to the servers not being ready or any other reasons, you can re-run the tests using the following command:

Here, you can utilize BEHAT_FEATURE and BEHAT_SUITE environment variables to specify the feature file or test suite you want to run again.

BEHAT_FEATURE='tests/acceptance/features/apiGraphUserGroup/createUser.feature:24' \
make -C tests/acceptance/docker run-api-tests

Skip Local Image Build While Running Tests

While running the tests, opencloud docker image is built with opencloudeu/opencloud:dev tag. If you want to skip building the local image, you can use OC_IMAGE_TAG env which must contain an available docker tag of the opencloudeu/opencloud registry on Docker Hub (e.g. 'latest').

OC_IMAGE_TAG=latest \
BEHAT_FEATURE='tests/acceptance/features/apiGraphUserGroup/createUser.feature' \
make -C tests/acceptance/docker run-api-tests

Check Test Logs

While a test is running or when it is finished, you can attach to the logs generated by the tests.

make -C tests/acceptance/docker show-test-logs

Note

The log output is opened in less. You can navigate up and down with your cursors. By pressing "F" you can follow the latest line of the output.

Cleanup the Setup

Run the following command to clean all the resources created while running the tests:

make -C tests/acceptance/docker clean-all

Running WOPI Validator Tests

Available Test Groups

  BaseWopiViewing
  CheckFileInfoSchema
  EditFlows
  Locks
  AccessTokens
  GetLock
  ExtendedLockLength
  FileVersion
  Features
  PutRelativeFile
  RenameFileIfCreateChildFileIsNotSupported

Run Test

TEST_GROUP=BaseWopiViewing docker compose -f tests/acceptance/docker/src/wopi-validator-test.yml up -d

Run Test (macOS)

Use the arm image for macOS to run the validator tests.

WOPI_VALIDATOR_IMAGE=scharfvi/wopi-validator \
TEST_GROUP=BaseWopiViewing \
docker compose -f tests/acceptance/docker/src/wopi-validator-test.yml up -d

Running Test Suite in Local Environment

Run OpenCloud

Create an up-to-date OpenCloud binary by [building OpenCloud]({{< ref "build" >}})

To start OpenCloud:

IDM_ADMIN_PASSWORD=admin \
opencloud/bin/opencloud init --insecure true

OC_INSECURE=true PROXY_ENABLE_BASIC_AUTH=true \
opencloud/bin/opencloud server

PROXY_ENABLE_BASIC_AUTH will allow the acceptance tests to make requests against the provisioning api (and other endpoints) using basic auth.

Run Local OpenCloud Tests (prefix api) and Tests Transferred From Core (prefix coreApi)

make test-acceptance-api \
TEST_SERVER_URL=https://localhost:9200 \

Useful environment variables:

TEST_SERVER_URL: OpenCloud server url. Please, adjust the server url according to your setup.

BEHAT_FEATURE: to run a single feature

Note: A specific scenario from a feature can be run by adding :<line-number> at the end of the feature file path. For example, to run the scenario at line 26 of the feature file apiGraphUserGroup/createUser.feature, simply add the line number like this: apiGraphUserGroup/createUser.feature:26. Note that the line numbers mentioned in the examples might not always point to a scenario, so always check the line numbers before running the test.

Example:

BEHAT_FEATURE=tests/acceptance/features/apiGraphUserGroup/createUser.feature

Or

BEHAT_FEATURE=tests/acceptance/features/apiGraphUserGroup/createUser.feature:13

BEHAT_SUITE: to run a single suite

Example:

BEHAT_SUITE=apiGraph

STORAGE_DRIVER: to run tests with a different user storage driver. Available options are decomposed (default), owncloudsql and decomposeds3

Example:

STORAGE_DRIVER=owncloudsql

STOP_ON_FAILURE: to stop running tests after the first failure

Example:

STOP_ON_FAILURE=true

Use Existing Tests for BDD

As a lot of scenarios are written for core, we can use those tests for Behaviour driven development in OpenCloud. Every scenario that does not work in OpenCloud with decomposed storage, is listed in tests/acceptance/expected-failures-decomposed-storage.md with a link to the related issue.

Those scenarios are run in the ordinary acceptance test pipeline in CI. The scenarios that fail are checked against the expected failures. If there are any differences then the CI pipeline fails.

If you want to work on a specific issue

1. locally run each of the tests marked with that issue in the expected failures file.

   E.g.:

   make test-acceptance-api \
   TEST_SERVER_URL=https://localhost:9200 \
   STORAGE_DRIVER=decomposed \
   BEHAT_FEATURE='tests/acceptance/features/coreApiVersions/fileVersions.feature:141'
   

2. the tests will fail, try to understand how and why they are failing

3. fix the code

4. go back to 1. and repeat till the tests are passing.

5. remove those tests from the expected failures file

6. make a PR that has the fixed code, and the relevant lines removed from the expected failures file.

Running Tests With And Without remote.php

By default, the tests are run with remote.php enabled. If you want to run the tests without remote.php, you can disable it by setting the environment variable WITH_REMOTE_PHP=false while running the tests.

WITH_REMOTE_PHP=false \
TEST_SERVER_URL="https://localhost:9200" \
make test-acceptance-api

Running ENV Config Tests (@env-Config)

Test suites tagged with @env-config are used to test the environment variables that are used to configure OpenCloud. These tests are special tests that require the OpenCloud server to be run using ocwrapper.

Run OpenCloud With ocwrapper

# working dir: OpenCloud repo root dir

# init OpenCloud
IDM_ADMIN_PASSWORD=admin \
opencloud/bin/opencloud init --insecure true

# build the wrapper
cd tests/ocwrapper
make build

# run OpenCloud
PROXY_ENABLE_BASIC_AUTH=true \
./bin/ocwrapper serve --bin=../../opencloud/bin/opencloud

Run the Tests

OC_WRAPPER_URL=http://localhost:5200 \
TEST_SERVER_URL="https://localhost:9200" \
BEHAT_FEATURE=tests/acceptance/features/apiAsyncUpload/delayPostprocessing.feature \
make test-acceptance-api

Writing New ENV Config Tests

While writing tests for a new OpenCloud ENV configuration, please make sure to follow these guidelines:

1. Tag the test suite (or test scenarios) with @env-config
2. Use OcConfigHelper.php for helper functions - provides functions to reconfigure the running OpenCloud instance.
3. Recommended: add the new step implementations in OcConfigContext.php

Running Test Suite With Email Service (@email)

Test suites that are tagged with @email require an email service. We use inbucket as the email service in our tests.

Setup Inbucket

Run the following command to setup inbucket

docker run -d -p9000:9000 -p2500:2500 --name inbucket inbucket/inbucket

Run OpenCloud

Documentation for environment variables is available here

# init OpenCloud
IDM_ADMIN_PASSWORD=admin \
opencloud/bin/opencloud init --insecure true

# run OpenCloud
PROXY_ENABLE_BASIC_AUTH=true \
OC_ADD_RUN_SERVICES=notifications \
NOTIFICATIONS_SMTP_HOST=localhost \
NOTIFICATIONS_SMTP_PORT=2500 \
NOTIFICATIONS_SMTP_INSECURE=true \
NOTIFICATIONS_SMTP_SENDER="OpenCloud <noreply@example.com>" \
opencloud/bin/opencloud server

Run the Acceptance Test

Run the acceptance test with the following command:

TEST_SERVER_URL="https://localhost:9200" \
EMAIL_HOST="localhost" \
EMAIL_PORT=9000 \
BEHAT_FEATURE="tests/acceptance/features/apiNotification/emailNotification.feature" \
make test-acceptance-api

Running Test Suite With Tika Service (@tikaServiceNeeded)

Test suites that are tagged with @tikaServiceNeeded require tika service.

Setup Tika Service

Run the following docker command to setup tika service

docker run -d -p 127.0.0.1:9998:9998 apache/tika

Run OpenCloud

TODO: Documentation related to the content based search and tika extractor will be added later.

# init OpenCloud
IDM_ADMIN_PASSWORD=admin \
opencloud/bin/opencloud init --insecure true

# run OpenCloud
PROXY_ENABLE_BASIC_AUTH=true \
OC_INSECURE=true \
SEARCH_EXTRACTOR_TYPE=tika \
SEARCH_EXTRACTOR_TIKA_TIKA_URL=http://localhost:9998 \
SEARCH_EXTRACTOR_CS3SOURCE_INSECURE=true \
opencloud/bin/opencloud server

Run the Acceptance Test

Run the acceptance test with the following command:

TEST_SERVER_URL="https://localhost:9200" \
BEHAT_FEATURE="tests/acceptance/features/apiSearchContent/contentSearch.feature" \
make test-acceptance-api

Running Test Suite With Antivirus Service (@antivirus)

Test suites that are tagged with @antivirus require antivirus service. TODO The available antivirus and the configuration related to them will be added latert. This documentation is only going to use clamav as antivirus.

Setup clamAV

Option 1. Setup Locally

Linux OS user:

Run the following command to set up calmAV and clamAV daemon

sudo apt install clamav clamav-daemon -y

Make sure that the clamAV daemon is up and running

sudo service clamav-daemon status

Note: The commands are ubuntu specific and may differ according to your system. You can find information related to installation of clamAV in their official documentation here.

Mac OS user:

Install ClamAV using here Start ClamAV daemon

/your/location/to/brew/Cellar/clamav/1.1.0/sbin/clamd

Option 2. Setup clamAV With Docker

Run clamAV through docker

docker run -d -p 3310:3310 opencloudeu/clamav-ci:latest

Run OpenCloud

As antivirus service is not enabled by default we need to enable the service while running OpenCloud server. We also need to enable async upload and as virus scan is performed in post-processing step, we need to set it as well. Documentation for environment variables related to antivirus is available here

# init OpenCloud
IDM_ADMIN_PASSWORD=admin \
opencloud/bin/opencloud init --insecure true

# run OpenCloud
PROXY_ENABLE_BASIC_AUTH=true \
ANTIVIRUS_SCANNER_TYPE="clamav" \
ANTIVIRUS_CLAMAV_SOCKET="tcp://host.docker.internal:3310" \
POSTPROCESSING_STEPS="virusscan" \
OC_ASYNC_UPLOADS=true \
OC_ADD_RUN_SERVICES="antivirus" \
opencloud/bin/opencloud server

Note: The value for ANTIVIRUS_CLAMAV_SOCKET is an example which needs adaption according your OS.

For antivirus running localy on Linux OS, use ANTIVIRUS_CLAMAV_SOCKET= "/var/run/clamav/clamd.ctl". For antivirus running localy on Mac OS, use ANTIVIRUS_CLAMAV_SOCKET= "/tmp/clamd.sock". For antivirus running with docker, use ANTIVIRUS_CLAMAV_SOCKET= "tcp://host.docker.internal:3310"

Create Virus Files

The antivirus tests require EICAR test files which are not stored in the repository They are generated dynamically when needed for testing.

tests/acceptance/scripts/generate-virus-files.sh

Run the Acceptance Test

Run the acceptance test with the following command:

TEST_SERVER_URL="https://localhost:9200" \
BEHAT_FEATURE="tests/acceptance/features/apiAntivirus/antivirus.feature" \
make test-acceptance-api

Running Test Suite With Federated Sharing (@ocm)

Test suites that are tagged with @ocm require running two different OpenCloud instances. TODO More detailed information and configuration related to it will be added later.

Setup First OpenCloud Instance

# init OpenCloud
IDM_ADMIN_PASSWORD=admin \
opencloud/bin/opencloud init --insecure true

# run OpenCloud
OC_URL="https://localhost:9200" \
PROXY_ENABLE_BASIC_AUTH=true \
OC_ENABLE_OCM=true \
OCM_OCM_PROVIDER_AUTHORIZER_PROVIDERS_FILE="tests/config/local/providers.json" \
OC_ADD_RUN_SERVICES="ocm" \
OCM_OCM_INVITE_MANAGER_INSECURE=true \
OCM_OCM_SHARE_PROVIDER_INSECURE=true \
OCM_OCM_STORAGE_PROVIDER_INSECURE=true \
WEB_UI_CONFIG_FILE="tests/config/local/opencloud-web.json" \
opencloud/bin/opencloud server

The first OpenCloud instance should be available at: https://localhost:9200/

Setup Second OpenCloud Instance

You can run the second OpenCloud instance in two ways:

Option 1. Using .vscode/launch.json

From the Run and Debug panel of VSCode, select Fed OpenCloud Server and start the debugger.

Option 2. Using env file

# init OpenCloud
source tests/config/local/.env-federation && opencloud/bin/opencloud init

# run OpenCloud
opencloud/bin/opencloud server

The second OpenCloud instance should be available at: https://localhost:10200/

Note: To enable ocm in the web interface, you need to set the following envs: OC_ENABLE_OCM="true" OC_ADD_RUN_SERVICES="ocm"

Run the Acceptance Test

Run the acceptance test with the following command:

TEST_SERVER_URL="https://localhost:9200" \
TEST_SERVER_FED_URL="https://localhost:10200" \
BEHAT_FEATURE="tests/acceptance/features/apiOcm/ocm.feature" \
make test-acceptance-api

Running Text Preview Tests Containing Unicode Characters

There are some tests that check the text preview of files containing Unicode characters. The OpenCloud server by default cannot generate the thumbnail of such files correctly but it provides an environment variable to allow the use of custom fonts that support Unicode characters. So to run such tests successfully, we have to run the OpenCloud server with this environment variable.

...
THUMBNAILS_TXT_FONTMAP_FILE="/path/to/fontsMap.json"
opencloud/bin/opencloud server

The sample fontsMap.json file is located in tests/config/drone/fontsMap.json.

{
  "defaultFont": "/path/to/opencloud/tests/config/drone/NotoSans.ttf"
}

Running All API Tests Locally

Build dev docker

make -C opencloud dev-docker

Choose STORAGE_DRIVER

By default, the system uses posix storage. However, you can override this by setting the STORAGE_DRIVER environment variable.

Run a script that starts the openCloud server in the docker and runs the API tests locally (for debugging purposes)

STORAGE_DRIVER=posix ./tests/acceptance/run_api_tests.sh