mirror of
https://github.com/opencloud-eu/opencloud.git
synced 2026-09-09 04:08:48 -04:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d695d283f6 | ||
|
|
553c85b7ff | ||
|
|
19e44f9b64 | ||
|
|
c93f50dc37 | ||
|
|
bb4cd1ebf5 | ||
|
|
e12966a4fd | ||
|
|
b350b355c0 | ||
|
|
9df3881128 | ||
|
|
557adfc327 | ||
|
|
97ad525d5b | ||
|
|
0fd02c5ecf | ||
|
|
27a77567dd | ||
|
|
57a615d6ce | ||
|
|
deb3247b99 | ||
|
|
b9adb507f8 | ||
|
|
c6fba8543f | ||
|
|
7c2cb14a80 | ||
|
|
b125f0681f | ||
|
|
20d16c5275 | ||
|
|
639e3657cd | ||
|
|
559b10e90a | ||
|
|
e63f1d542a | ||
|
|
51089765d0 | ||
|
|
54cced7c56 | ||
|
|
3d546693c6 | ||
|
|
40ced9997e | ||
|
|
3bd188122e | ||
|
|
9b5849f470 | ||
|
|
942cca8eef | ||
|
|
a9f39b85d5 | ||
|
|
15fc44be94 | ||
|
|
2eb1fe069c | ||
|
|
a82b6223aa | ||
|
|
433dea41e8 | ||
|
|
e26a009fc1 | ||
|
|
49a81c9c1f | ||
|
|
7e8179a704 | ||
|
|
d50d8d8900 | ||
|
|
243514ec17 | ||
|
|
be75d8fe95 | ||
|
|
bb8cefa33b | ||
|
|
0a44bbae9e | ||
|
|
fb387a16b6 | ||
|
|
e358cb6e6e | ||
|
|
341ebdf6cc | ||
|
|
6d2d6c38ec | ||
|
|
d27b7ef40a | ||
|
|
27bdda265f | ||
|
|
7d59a83c18 | ||
|
|
9d472290f2 | ||
|
|
e29d961be3 | ||
|
|
809d32c4d8 | ||
|
|
046f46ce8d | ||
|
|
51dd107767 | ||
|
|
fec92d2f22 | ||
|
|
00dd4f1b7f |
No files matched your search
@@ -0,0 +1,153 @@
|
||||
---
|
||||
name: bumping-reva
|
||||
description: Use when the user asks to bump, update, or upgrade the OpenCloud reva dependency to a specific version (e.g. "reva bump to 2.48.0", "bump reva to v2.48.0"). Covers editing go.mod, re-vendoring, bumping the OpenCloud LatestTag, the single commit, and opening the PR against main.
|
||||
---
|
||||
|
||||
# Bumping Reva
|
||||
|
||||
## Overview
|
||||
|
||||
Bumping reva updates the pinned [opencloud-eu/reva](https://github.com/opencloud-eu/reva) dependency (`github.com/opencloud-eu/reva/v2`) to a tagged release, re-vendors the module graph, and bumps the OpenCloud dev version. It touches `go.mod`, `go.sum`, `vendor/**` and `pkg/version/version.go`, lands as one commit, and ships as a PR whose body is the reva changelog for that version.
|
||||
|
||||
Template PR: https://github.com/opencloud-eu/opencloud/pull/3127
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **The reva release must already be tagged.** A reva release is cut by merging its release PR (title `🎉 Release X.Y.Z`, branch `next-release/main`). The **user merges that PR themselves** — this skill starts *after* the tag `vX.Y.Z` exists. Verify the tag before doing anything (step 1); if it's missing, stop and ask the user to merge the reva release PR first.
|
||||
- **`gh` (GitHub CLI), authenticated** — tag lookup and PR creation go through `gh api` / `gh pr`. Verify with `gh auth status`; if it fails, ask the user to run `gh auth login` (suggest `! gh auth login` so it runs in-session).
|
||||
- `gh` needs the system keyring, so run all `gh` commands with the sandbox disabled.
|
||||
- **Go toolchain + network** — `go get` / `go mod tidy` / `go mod vendor` hit the Go module proxy. Run them with the sandbox disabled (network access).
|
||||
- `git` and `base64` (decoding the changelog) — standard on macOS/Linux.
|
||||
|
||||
## Inputs
|
||||
|
||||
- `REVA_VERSION` — the new reva tag, always normalized to a leading `v` (e.g. `v2.48.0`). If the user didn't give it, ask (or take the latest reva release tag).
|
||||
- `OC_VERSION` — the OpenCloud target for `LatestTag`, **without** the `+dev` suffix (e.g. `7.4.0`). **Do not ask or guess** — derive it from the open OpenCloud release PR (step 3). It can be a new major (e.g. `8.0.0`) when release-please picked up a breaking change.
|
||||
|
||||
## What changes
|
||||
|
||||
| File | What |
|
||||
| ------------------------ | ------------------------------------------------------------------- |
|
||||
| `go.mod` | `github.com/opencloud-eu/reva/v2` → `REVA_VERSION` (+ indirect deps pulled by `go mod tidy`) |
|
||||
| `go.sum` | updated by `go get` / `go mod tidy` |
|
||||
| `vendor/**` | re-vendored by `go mod vendor` (incl. `vendor/modules.txt`) |
|
||||
| `pkg/version/version.go` | `LatestTag = "<OC_VERSION>+dev"` |
|
||||
|
||||
## Procedure
|
||||
|
||||
### 1. Verify the reva tag exists
|
||||
|
||||
```bash
|
||||
gh api repos/opencloud-eu/reva/commits/$REVA_VERSION --jq '.sha'
|
||||
```
|
||||
|
||||
- Success (a sha) → the release is tagged, continue.
|
||||
- `422`/`404` → the tag does not exist yet. The reva release PR (`🎉 Release X.Y.Z`) is probably not merged. **Stop** and tell the user to merge it first. Helpful checks:
|
||||
```bash
|
||||
gh api repos/opencloud-eu/reva/releases/latest --jq '.tag_name' # current latest tag
|
||||
gh pr list --repo opencloud-eu/reva --search '🎉 Release in:title' --state open --json number,title
|
||||
```
|
||||
|
||||
### 2. Bump the dependency and re-vendor
|
||||
|
||||
```bash
|
||||
go get github.com/opencloud-eu/reva/v2@$REVA_VERSION
|
||||
go mod tidy
|
||||
go mod vendor
|
||||
```
|
||||
|
||||
(Sandbox disabled — these need network.) Notes:
|
||||
- Right after a fresh tag the module proxy can lag; if `go get` reports the version as unknown, retry, or use `GOPROXY=direct go get github.com/opencloud-eu/reva/v2@$REVA_VERSION`.
|
||||
- `go mod tidy` will also bump indirect dependencies that reva pulled in — that is expected (the template PR did the same).
|
||||
|
||||
### 3. Bump the OpenCloud dev version
|
||||
|
||||
`OC_VERSION` is the version of the **open OpenCloud release PR** — the release-please PR from branch `next-release/main`, titled `🎉 Release X.Y.Z` (e.g. #3143). Derive it, don't ask:
|
||||
|
||||
```bash
|
||||
gh pr list --repo opencloud-eu/opencloud --head next-release/main --state open \
|
||||
--json title --jq '.[0].title' | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1
|
||||
```
|
||||
|
||||
This is the next release target and tracks breaking changes — it may be a new major (e.g. `8.0.0`), not just a minor bump. If no such PR is open, stop and ask the user.
|
||||
|
||||
Edit `pkg/version/version.go`:
|
||||
|
||||
```go
|
||||
LatestTag = "<OC_VERSION>+dev" // e.g. "7.4.0+dev"
|
||||
```
|
||||
|
||||
### 4. Fetch the reva changelog for the PR body
|
||||
|
||||
```bash
|
||||
gh api "repos/opencloud-eu/reva/contents/CHANGELOG.md?ref=$REVA_VERSION" --jq '.content' | base64 -d
|
||||
```
|
||||
|
||||
Take **only the section for this version** and trim it exactly like the web bump: start at the first content heading (`### 🐛 Bug Fixes` / `### 📈 Enhancement` / `### 💥 Breaking changes`), drop the `# Changelog` title, the `## [x.y.z] - date` header and the `### ❤️ Thanks to all contributors!` block, and stop before the next `## [...]` version header.
|
||||
|
||||
Then **prepend two summary bullets** so the final PR body is:
|
||||
|
||||
```
|
||||
- bump opencloud version to <OC_VERSION>
|
||||
- reva bump <REVA_VERSION without the leading v>
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
...trimmed reva changelog...
|
||||
```
|
||||
|
||||
Write this to a file for `--body-file` (e.g. in the scratchpad).
|
||||
|
||||
### 5. Confirm before committing
|
||||
|
||||
The `vendor/` diff is huge — do **not** dump it. Show the user:
|
||||
|
||||
```bash
|
||||
git diff go.mod pkg/version/version.go # the meaningful edits
|
||||
git diff --stat | tail -1 # vendor churn summary
|
||||
```
|
||||
|
||||
Confirm the reva line in `go.mod` is exactly `github.com/opencloud-eu/reva/v2 REVA_VERSION`, show the target branch (`main`), and the PR body. **Do not commit until the user approves.**
|
||||
|
||||
### 6. Commit, push, open PR
|
||||
|
||||
- Create a branch (do not commit on `main`), e.g. `reva-bump-2.48.0`.
|
||||
- Stage everything the bump touched: `git add go.mod go.sum vendor pkg/version/version.go`.
|
||||
- One commit, conventional-commits format, **empty body**:
|
||||
```
|
||||
chore: reva bump -2.48.0
|
||||
```
|
||||
- PR base: **`main`** (reva bumps always target main).
|
||||
- PR title: `[full-ci] chore: reva bump -2.48.0` (the commit message prefixed with `[full-ci] `).
|
||||
- PR body: the file from step 4.
|
||||
- Add the label `Type:Maintenance`.
|
||||
|
||||
```bash
|
||||
gh pr create --base main \
|
||||
--title "[full-ci] chore: reva bump -$REVA_VERSION_NO_V" \
|
||||
--label "Type:Maintenance" \
|
||||
--body-file <body-file>
|
||||
```
|
||||
|
||||
(`gh` commands need the sandbox disabled — they require the system keyring.)
|
||||
|
||||
## Quick reference
|
||||
|
||||
```bash
|
||||
REVA_VERSION=v2.48.0
|
||||
gh api repos/opencloud-eu/reva/commits/$REVA_VERSION --jq '.sha' # verify tag exists
|
||||
OC_VERSION=$(gh pr list --repo opencloud-eu/opencloud --head next-release/main --state open \
|
||||
--json title --jq '.[0].title' | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1) # from open release PR
|
||||
go get github.com/opencloud-eu/reva/v2@$REVA_VERSION && go mod tidy && go mod vendor # bump + re-vendor
|
||||
# edit pkg/version/version.go -> LatestTag = "$OC_VERSION+dev"
|
||||
gh api "repos/opencloud-eu/reva/contents/CHANGELOG.md?ref=$REVA_VERSION" --jq '.content' | base64 -d # changelog
|
||||
```
|
||||
|
||||
## Common mistakes
|
||||
|
||||
- Running the bump before the reva release PR is merged — the tag won't exist and `go get` will fail. Verify the tag first (step 1).
|
||||
- Forgetting to re-run `go mod vendor` after `go mod tidy`, leaving `vendor/` out of sync with `go.mod`.
|
||||
- Asking for or hardcoding `OC_VERSION` — always derive it from the open `next-release/main` release PR (step 3); it can even be a new major after a breaking change.
|
||||
- Reading the changelog from reva `main` instead of the tag (`?ref=$REVA_VERSION`). Always pin to the tag.
|
||||
- Missing the `[full-ci] ` prefix in the PR title, the `Type:Maintenance` label, or the two summary bullets at the top of the body.
|
||||
- Dumping the full `vendor/` diff at the confirmation step instead of `go.mod` + `version.go` + a `--stat` summary.
|
||||
- Committing before the user confirms the diff.
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
exclude_paths:
|
||||
- '.github/**'
|
||||
- '.agents/**'
|
||||
- 'CHANGELOG.md'
|
||||
- '**/CHANGELOG.md'
|
||||
- 'changelog/**'
|
||||
|
||||
+1
-1
@@ -1,3 +1,3 @@
|
||||
# The test runner source for UI tests
|
||||
WEB_COMMITID=7bc6f96066e0618c301ee34cb7fe219efd47e3c0
|
||||
WEB_COMMITID=52b3c79f772feae23b7e77058c89c4151087502e
|
||||
WEB_BRANCH=main
|
||||
@@ -1,5 +1,54 @@
|
||||
# Changelog
|
||||
|
||||
## [7.4.0](https://github.com/opencloud-eu/opencloud/releases/tag/v7.4.0) - 2026-08-03
|
||||
|
||||
### ❤️ Thanks to all contributors! ❤️
|
||||
|
||||
@AlexAndBear, @JammingBen, @aduffeck, @dschmidt, @fschade, @michaelstingl, @pbleser-oc, @rhafer, @schweigisito, @v-scharf
|
||||
|
||||
### 📈 Enhancement
|
||||
|
||||
- Do not check ignored paths [[#3233](https://github.com/opencloud-eu/opencloud/pull/3233)]
|
||||
- Extend posixfs consistency check [[#3220](https://github.com/opencloud-eu/opencloud/pull/3220)]
|
||||
- Improve reindex command [[#3213](https://github.com/opencloud-eu/opencloud/pull/3213)]
|
||||
- Reindex spaces concurrently [[#3207](https://github.com/opencloud-eu/opencloud/pull/3207)]
|
||||
- feat: add announcement banner [[#3189](https://github.com/opencloud-eu/opencloud/pull/3189)]
|
||||
- feat: add space viewer with versions role [[#2961](https://github.com/opencloud-eu/opencloud/pull/2961)]
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- fix(runtime): log service startup errors instead of printing them beside the log [[#3140](https://github.com/opencloud-eu/opencloud/pull/3140)]
|
||||
- fix(posixfs scan): Setup logger for scan command [[#3185](https://github.com/opencloud-eu/opencloud/pull/3185)]
|
||||
|
||||
### ✅ Tests
|
||||
|
||||
- api-test: mark group last-manager removal scenario as flaky [[#3194](https://github.com/opencloud-eu/opencloud/pull/3194)]
|
||||
- rerun flaky tests [[#3183](https://github.com/opencloud-eu/opencloud/pull/3183)]
|
||||
- api-test: fix removeAccessToDrive.feature:145 [[#3179](https://github.com/opencloud-eu/opencloud/pull/3179)]
|
||||
- api-test: cover additional unified roles in acceptance tests [[#3169](https://github.com/opencloud-eu/opencloud/pull/3169)]
|
||||
- test(apiAuthApp): fix flaky token pattern [[#3163](https://github.com/opencloud-eu/opencloud/pull/3163)]
|
||||
- test(apiArchiver): the single-resource archive is named after the resource [[#3080](https://github.com/opencloud-eu/opencloud/pull/3080)]
|
||||
- test(coreApiWebdavUploadTUS): assert etag and permissions on the finalizing TUS chunk [[#3078](https://github.com/opencloud-eu/opencloud/pull/3078)]
|
||||
- test(coreApiWebdavOperations): download a file with a literal "%" via its oc:downloadURL [[#3079](https://github.com/opencloud-eu/opencloud/pull/3079)]
|
||||
|
||||
### 📚 Documentation
|
||||
|
||||
- maint: clean-up auth-app documentation [[#3155](https://github.com/opencloud-eu/opencloud/pull/3155)]
|
||||
- ci: sync tests/README.md to docs [[#3164](https://github.com/opencloud-eu/opencloud/pull/3164)]
|
||||
|
||||
### 📦️ Dependencies
|
||||
|
||||
- [full-ci] chore: bump web to v7.3.0 [[#3223](https://github.com/opencloud-eu/opencloud/pull/3223)]
|
||||
- build(deps): bump github.com/open-policy-agent/opa from 1.18.2 to 1.19.0 [[#3231](https://github.com/opencloud-eu/opencloud/pull/3231)]
|
||||
- build(deps): bump google.golang.org/grpc from 1.82.0 to 1.83.0 [[#3232](https://github.com/opencloud-eu/opencloud/pull/3232)]
|
||||
- chore(idp): bump dependencies [[#3226](https://github.com/opencloud-eu/opencloud/pull/3226)]
|
||||
- build(deps): bump github.com/nats-io/nats-server/v2 from 2.14.3 to 2.14.4 [[#3221](https://github.com/opencloud-eu/opencloud/pull/3221)]
|
||||
- build(deps): bump github.com/go-ldap/ldap/v3 from 3.4.13 to 3.4.14 [[#3222](https://github.com/opencloud-eu/opencloud/pull/3222)]
|
||||
- build(deps): bump github.com/prometheus/client_golang from 1.23.2 to 1.24.1 [[#3218](https://github.com/opencloud-eu/opencloud/pull/3218)]
|
||||
- build(deps): bump github.com/gabriel-vasile/mimetype from 1.4.13 to 1.4.15 [[#3217](https://github.com/opencloud-eu/opencloud/pull/3217)]
|
||||
- build(deps): bump golang.org/x/net from 0.56.0 to 0.57.0 [[#3136](https://github.com/opencloud-eu/opencloud/pull/3136)]
|
||||
- build(deps): bump github.com/beevik/etree from 1.6.0 to 1.7.0 [[#3134](https://github.com/opencloud-eu/opencloud/pull/3134)]
|
||||
|
||||
## [7.3.0](https://github.com/opencloud-eu/opencloud/releases/tag/v7.3.0) - 2026-07-14
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
@@ -10,7 +10,7 @@ require (
|
||||
github.com/MicahParks/keyfunc/v2 v2.1.0
|
||||
github.com/Nerzal/gocloak/v13 v13.9.0
|
||||
github.com/bbalet/stopwords v1.0.0
|
||||
github.com/beevik/etree v1.6.0
|
||||
github.com/beevik/etree v1.7.0
|
||||
github.com/blevesearch/bleve/v2 v2.6.0
|
||||
github.com/cenkalti/backoff v2.2.1+incompatible
|
||||
github.com/coreos/go-oidc/v3 v3.20.0
|
||||
@@ -18,12 +18,12 @@ require (
|
||||
github.com/davidbyttow/govips/v2 v2.18.0
|
||||
github.com/dhowden/tag v0.0.0-20240417053706-3d75831295e8
|
||||
github.com/dutchcoders/go-clamd v0.0.0-20170520113014-b970184f4d9e
|
||||
github.com/gabriel-vasile/mimetype v1.4.13
|
||||
github.com/gabriel-vasile/mimetype v1.4.15
|
||||
github.com/ggwhite/go-masker v1.1.0
|
||||
github.com/go-chi/chi/v5 v5.3.1
|
||||
github.com/go-chi/render v1.0.3
|
||||
github.com/go-jose/go-jose/v3 v3.0.5
|
||||
github.com/go-ldap/ldap/v3 v3.4.13
|
||||
github.com/go-ldap/ldap/v3 v3.4.14
|
||||
github.com/go-ldap/ldif v0.0.0-20200320164324-fd88d9b715b3
|
||||
github.com/go-micro/plugins/v4/client/grpc v1.2.1
|
||||
github.com/go-micro/plugins/v4/logger/zerolog v1.2.0
|
||||
@@ -55,26 +55,27 @@ require (
|
||||
github.com/libregraph/lico v0.67.0
|
||||
github.com/mna/pigeon v1.3.0
|
||||
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826
|
||||
github.com/nats-io/nats-server/v2 v2.14.3
|
||||
github.com/nats-io/nats-server/v2 v2.14.4
|
||||
github.com/nats-io/nats.go v1.52.0
|
||||
github.com/olekukonko/tablewriter v1.1.4
|
||||
github.com/onsi/ginkgo v1.16.5
|
||||
github.com/onsi/ginkgo/v2 v2.32.0
|
||||
github.com/onsi/gomega v1.42.1
|
||||
github.com/open-policy-agent/opa v1.18.2
|
||||
github.com/open-policy-agent/opa v1.19.0
|
||||
github.com/opencloud-eu/icap-client v0.0.0-20250930132611-28a2afe62d89
|
||||
github.com/opencloud-eu/libre-graph-api-go v1.0.8-0.20260310090739-853d972b282d
|
||||
github.com/opencloud-eu/reva/v2 v2.47.0
|
||||
github.com/opencloud-eu/reva/v2 v2.48.0
|
||||
github.com/opensearch-project/opensearch-go/v4 v4.6.0
|
||||
github.com/orcaman/concurrent-map v1.0.0
|
||||
github.com/pkg/errors v0.9.1
|
||||
github.com/pkg/xattr v0.4.12
|
||||
github.com/prometheus/client_golang v1.23.2
|
||||
github.com/prometheus/client_golang v1.24.1
|
||||
github.com/r3labs/sse/v2 v2.10.0
|
||||
github.com/riandyrn/otelchi v0.12.3
|
||||
github.com/rogpeppe/go-internal v1.15.0
|
||||
github.com/rs/cors v1.11.1
|
||||
github.com/rs/zerolog v1.35.1
|
||||
github.com/shamaton/msgpack/v2 v2.4.1
|
||||
github.com/sirupsen/logrus v1.9.4
|
||||
github.com/spf13/afero v1.15.0
|
||||
github.com/spf13/cobra v1.10.2
|
||||
@@ -84,7 +85,6 @@ require (
|
||||
github.com/test-go/testify v1.1.4
|
||||
github.com/testcontainers/testcontainers-go v0.43.0
|
||||
github.com/testcontainers/testcontainers-go/modules/opensearch v0.43.0
|
||||
github.com/theckman/yacspin v0.13.12
|
||||
github.com/thejerf/suture/v4 v4.0.6
|
||||
github.com/tidwall/gjson v1.19.0
|
||||
github.com/tidwall/sjson v1.2.5
|
||||
@@ -102,16 +102,16 @@ require (
|
||||
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.44.0
|
||||
go.opentelemetry.io/otel/sdk v1.44.0
|
||||
go.opentelemetry.io/otel/trace v1.44.0
|
||||
golang.org/x/crypto v0.53.0
|
||||
golang.org/x/crypto v0.54.0
|
||||
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f
|
||||
golang.org/x/image v0.44.0
|
||||
golang.org/x/net v0.56.0
|
||||
golang.org/x/net v0.57.0
|
||||
golang.org/x/oauth2 v0.36.0
|
||||
golang.org/x/sync v0.22.0
|
||||
golang.org/x/term v0.45.0
|
||||
golang.org/x/text v0.40.0
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa
|
||||
google.golang.org/grpc v1.82.0
|
||||
google.golang.org/grpc v1.83.0
|
||||
google.golang.org/protobuf v1.36.11
|
||||
gopkg.in/yaml.v2 v2.4.0
|
||||
gopkg.in/yaml.v3 v3.0.1
|
||||
@@ -135,7 +135,7 @@ require (
|
||||
github.com/ajg/form v1.5.1 // indirect
|
||||
github.com/alexedwards/argon2id v1.0.0 // indirect
|
||||
github.com/amoghe/go-crypt v0.0.0-20220222110647-20eada5f5964 // indirect
|
||||
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op // indirect
|
||||
github.com/antithesishq/antithesis-sdk-go v0.7.2 // indirect
|
||||
github.com/armon/go-radix v1.0.0 // indirect
|
||||
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
|
||||
github.com/beorn7/perks v1.0.1 // indirect
|
||||
@@ -203,14 +203,14 @@ require (
|
||||
github.com/fsnotify/fsnotify v1.10.1 // indirect
|
||||
github.com/gdexlab/go-render v1.0.1 // indirect
|
||||
github.com/go-acme/lego/v4 v4.4.0 // indirect
|
||||
github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667 // indirect
|
||||
github.com/go-asn1-ber/asn1-ber v1.5.8 // indirect
|
||||
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
|
||||
github.com/go-git/go-billy/v5 v5.9.0 // indirect
|
||||
github.com/go-git/go-git/v5 v5.19.1 // indirect
|
||||
github.com/go-jose/go-jose/v4 v4.1.4 // indirect
|
||||
github.com/go-kit/log v0.2.1 // indirect
|
||||
github.com/go-logfmt/logfmt v0.5.1 // indirect
|
||||
github.com/go-logr/logr v1.4.3 // indirect
|
||||
github.com/go-logr/logr v1.4.4 // indirect
|
||||
github.com/go-logr/stdr v1.2.2 // indirect
|
||||
github.com/go-micro/plugins/v4/events/natsjs v1.2.2 // indirect
|
||||
github.com/go-micro/plugins/v4/store/nats-js v1.2.1 // indirect
|
||||
@@ -255,7 +255,7 @@ require (
|
||||
github.com/json-iterator/go v1.1.12 // indirect
|
||||
github.com/juliangruber/go-intersect v1.1.0 // indirect
|
||||
github.com/kevinburke/ssh_config v1.2.0 // indirect
|
||||
github.com/klauspost/compress v1.18.6 // indirect
|
||||
github.com/klauspost/compress v1.19.1 // indirect
|
||||
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
|
||||
github.com/klauspost/crc32 v1.3.0 // indirect
|
||||
github.com/kovidgoyal/go-parallel v1.1.1 // indirect
|
||||
@@ -274,10 +274,10 @@ require (
|
||||
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
|
||||
github.com/magiconair/properties v1.8.10 // indirect
|
||||
github.com/mattermost/xml-roundtrip-validator v0.1.0 // indirect
|
||||
github.com/mattn/go-colorable v0.1.14 // indirect
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/mattn/go-runewidth v0.0.23 // indirect
|
||||
github.com/mattn/go-sqlite3 v1.14.42 // indirect
|
||||
github.com/mattn/go-colorable v0.1.15 // indirect
|
||||
github.com/mattn/go-isatty v0.0.22 // indirect
|
||||
github.com/mattn/go-runewidth v0.0.24 // indirect
|
||||
github.com/mattn/go-sqlite3 v1.14.49 // indirect
|
||||
github.com/maxymania/go-system v0.0.0-20170110133659-647cc364bf0b // indirect
|
||||
github.com/mendsley/gojwk v0.0.0-20141217222730-4d5ec6e58103 // indirect
|
||||
github.com/miekg/dns v1.1.68 // indirect
|
||||
@@ -325,8 +325,8 @@ require (
|
||||
github.com/pquerna/cachecontrol v0.2.0 // indirect
|
||||
github.com/prometheus/alertmanager v0.33.1 // indirect
|
||||
github.com/prometheus/client_model v0.6.2 // indirect
|
||||
github.com/prometheus/common v0.67.5 // indirect
|
||||
github.com/prometheus/procfs v0.20.1 // indirect
|
||||
github.com/prometheus/common v0.70.1 // indirect
|
||||
github.com/prometheus/procfs v0.21.1 // indirect
|
||||
github.com/prometheus/statsd_exporter v0.22.8 // indirect
|
||||
github.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9 // indirect
|
||||
github.com/rs/xid v1.6.0 // indirect
|
||||
@@ -342,9 +342,8 @@ require (
|
||||
github.com/segmentio/ksuid v1.0.4 // indirect
|
||||
github.com/sercand/kuberesolver/v5 v5.1.1 // indirect
|
||||
github.com/sergi/go-diff v1.4.0 // indirect
|
||||
github.com/sethvargo/go-diceware v0.5.0 // indirect
|
||||
github.com/sethvargo/go-password v0.3.1 // indirect
|
||||
github.com/shamaton/msgpack/v2 v2.4.1 // indirect
|
||||
github.com/sethvargo/go-diceware v0.6.0 // indirect
|
||||
github.com/sethvargo/go-password v0.4.0 // indirect
|
||||
github.com/shirou/gopsutil/v4 v4.26.5 // indirect
|
||||
github.com/skeema/knownhosts v1.3.1 // indirect
|
||||
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 // indirect
|
||||
@@ -363,7 +362,7 @@ require (
|
||||
github.com/trustelem/zxcvbn v1.0.1 // indirect
|
||||
github.com/urfave/cli/v2 v2.27.7 // indirect
|
||||
github.com/valyala/fastjson v1.6.10 // indirect
|
||||
github.com/vektah/gqlparser/v2 v2.5.34 // indirect
|
||||
github.com/vektah/gqlparser/v2 v2.5.36 // indirect
|
||||
github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
|
||||
github.com/wk8/go-ordered-map v1.0.0 // indirect
|
||||
github.com/xanzy/ssh-agent v0.3.3 // indirect
|
||||
@@ -390,7 +389,7 @@ require (
|
||||
golang.org/x/time v0.15.0 // indirect
|
||||
golang.org/x/tools v0.47.0 // indirect
|
||||
google.golang.org/genproto v0.0.0-20260319201613-d00831a3d3e7 // indirect
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa // indirect
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260729162451-8efbd57d26e0 // indirect
|
||||
gopkg.in/cenkalti/backoff.v1 v1.1.0 // indirect
|
||||
gopkg.in/ini.v1 v1.67.3 // indirect
|
||||
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
|
||||
|
||||
@@ -115,8 +115,8 @@ github.com/amoghe/go-crypt v0.0.0-20220222110647-20eada5f5964 h1:I9YN9WMo3SUh7p/
|
||||
github.com/amoghe/go-crypt v0.0.0-20220222110647-20eada5f5964/go.mod h1:eFiR01PwTcpbzXtdMces7zxg6utvFM5puiWHpWB8D/k=
|
||||
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8=
|
||||
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be/go.mod h1:ySMOLuWl6zY27l47sB3qLNK6tF2fkHG55UZxx8oIVo4=
|
||||
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
|
||||
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
|
||||
github.com/antithesishq/antithesis-sdk-go v0.7.2 h1:oEEedg1Xgi8drRjqB0f9tfjhLoInE0IYZfZ6zAhQUbY=
|
||||
github.com/antithesishq/antithesis-sdk-go v0.7.2/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
|
||||
github.com/apache/thrift v0.12.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
|
||||
github.com/arbovm/levenshtein v0.0.0-20160628152529-48b4e1c0c4d0 h1:jfIu9sQUG6Ig+0+Ap1h4unLjW6YQJpKZVmUzxsD4E/Q=
|
||||
github.com/arbovm/levenshtein v0.0.0-20160628152529-48b4e1c0c4d0/go.mod h1:t2tdKJDJF9BV14lnkjHmOQgcvEKgtqs5a1N3LNdJhGE=
|
||||
@@ -132,8 +132,8 @@ github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2/go.mod h1:W
|
||||
github.com/aws/aws-sdk-go v1.37.27/go.mod h1:hcU610XS61/+aQV88ixoOzUoG7v3b31pl2zKMmprdro=
|
||||
github.com/bbalet/stopwords v1.0.0 h1:0TnGycCtY0zZi4ltKoOGRFIlZHv0WqpoIGUsObjztfo=
|
||||
github.com/bbalet/stopwords v1.0.0/go.mod h1:sAWrQoDMfqARGIn4s6dp7OW7ISrshUD8IP2q3KoqPjc=
|
||||
github.com/beevik/etree v1.6.0 h1:u8Kwy8pp9D9XeITj2Z0XtA5qqZEmtJtuXZRQi+j03eE=
|
||||
github.com/beevik/etree v1.6.0/go.mod h1:bh4zJxiIr62SOf9pRzN7UUYaEDa9HEKafK25+sLc0Gc=
|
||||
github.com/beevik/etree v1.7.0 h1:xjBk9O4p4x7D1YajePjfLzdaFC4/uYUENA7P0pv6gXA=
|
||||
github.com/beevik/etree v1.7.0/go.mod h1:bh4zJxiIr62SOf9pRzN7UUYaEDa9HEKafK25+sLc0Gc=
|
||||
github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
|
||||
github.com/beorn7/perks v0.0.0-20160804104726-4c0e84591b9a/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
|
||||
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
|
||||
@@ -195,8 +195,6 @@ github.com/bufbuild/protocompile v0.14.1 h1:iA73zAf/fyljNjQKwYzUHD6AD4R8KMasmwa/
|
||||
github.com/bufbuild/protocompile v0.14.1/go.mod h1:ppVdAIhbr2H8asPk6k4pY7t9zB1OU5DoEw9xY/FUi1c=
|
||||
github.com/butonic/go-micro/v4 v4.11.1-0.20241115112658-b5d4de5ed9b3 h1:h8Z0hBv5tg/uZMKu8V47+DKWYVQg0lYP8lXDQq7uRpE=
|
||||
github.com/butonic/go-micro/v4 v4.11.1-0.20241115112658-b5d4de5ed9b3/go.mod h1:eE/tD53n3KbVrzrWxKLxdkGw45Fg1qaNLWjpJMvIUF4=
|
||||
github.com/bytecodealliance/wasmtime-go/v44 v44.0.0 h1:WRZXnLPIer/TWs5aYPaMlmVcOlzmR6Ur6wjLRIQOhTQ=
|
||||
github.com/bytecodealliance/wasmtime-go/v44 v44.0.0/go.mod h1:GP93piU+39CoFVCQ5xfHrPOUtL0APlMnkbblJ2d3YY0=
|
||||
github.com/c-bata/go-prompt v0.2.5/go.mod h1:vFnjEGDIIA/Lib7giyE4E9c50Lvl8j0S+7FVlAwDAVw=
|
||||
github.com/cenkalti/backoff v2.2.1+incompatible h1:tNowT99t7UNflLxfYYSlKYsBpXdEet03Pg2g16Swow4=
|
||||
github.com/cenkalti/backoff v2.2.1+incompatible/go.mod h1:90ReRw6GdpyfrHakVjL/QHaoyV4aDUVVkXQJJJ3NXXM=
|
||||
@@ -281,8 +279,8 @@ github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.1/go.mod h1:ZXNYxsqcloTdSy/rNShjY
|
||||
github.com/deepmap/oapi-codegen v1.3.11/go.mod h1:suMvK7+rKlx3+tpa8ByptmvoXbAV70wERKTOGH3hLp0=
|
||||
github.com/desertbit/timer v0.0.0-20180107155436-c41aec40b27f h1:U5y3Y5UE0w7amNe7Z5G/twsBW0KEalRQXZzf8ufSh9I=
|
||||
github.com/desertbit/timer v0.0.0-20180107155436-c41aec40b27f/go.mod h1:xH/i4TFMt8koVQZ6WFms69WAsDWr2XsYL3Hkl7jkoLE=
|
||||
github.com/dgraph-io/badger/v4 v4.9.2 h1:Wb5qw8gElqwV1a8msHTeQKova9b1V10heFKMIiPd80E=
|
||||
github.com/dgraph-io/badger/v4 v4.9.2/go.mod h1:nJjaJTUOSsQEBhsq209FmwCvMJzEA3e74RjZw6V2pQI=
|
||||
github.com/dgraph-io/badger/v4 v4.9.4 h1:bcw+waCpzRZ2nmcSPbnPvDVhiEsn98TKmvnAhK7r7LM=
|
||||
github.com/dgraph-io/badger/v4 v4.9.4/go.mod h1:nJjaJTUOSsQEBhsq209FmwCvMJzEA3e74RjZw6V2pQI=
|
||||
github.com/dgraph-io/ristretto v0.2.0 h1:XAfl+7cmoUDWW/2Lx8TGZQjjxIQ2Ley9DSf52dru4WE=
|
||||
github.com/dgraph-io/ristretto v0.2.0/go.mod h1:8uBHCU/PBV4Ag0CJrP47b9Ofby5dqWNh4FicAdoqFNU=
|
||||
github.com/dgraph-io/ristretto/v2 v2.2.0 h1:bkY3XzJcXoMuELV8F+vS8kzNgicwQFAaGINAEJdWGOM=
|
||||
@@ -350,8 +348,8 @@ github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMo
|
||||
github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
|
||||
github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
|
||||
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
|
||||
github.com/gabriel-vasile/mimetype v1.4.13 h1:46nXokslUBsAJE/wMsp5gtO500a4F3Nkz9Ufpk2AcUM=
|
||||
github.com/gabriel-vasile/mimetype v1.4.13/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s=
|
||||
github.com/gabriel-vasile/mimetype v1.4.15 h1:05iP/CYtZ/w455R/KZM6rZ5ieAdh99UPtd+d3YzLmaI=
|
||||
github.com/gabriel-vasile/mimetype v1.4.15/go.mod h1:azpTcoLcDZRNgFou5j+APrqQx9HqVPWa6ijYQIIVswQ=
|
||||
github.com/gdexlab/go-render v1.0.1 h1:rxqB3vo5s4n1kF0ySmoNeSPRYkEsyHgln4jFIQY7v0U=
|
||||
github.com/gdexlab/go-render v1.0.1/go.mod h1:wRi5nW2qfjiGj4mPukH4UV0IknS1cHD4VgFTmJX5JzM=
|
||||
github.com/getkin/kin-openapi v0.13.0/go.mod h1:WGRs2ZMM1Q8LR1QBEwUxC6RJEfaBcD0s+pcEVXFuAjw=
|
||||
@@ -370,8 +368,8 @@ github.com/go-acme/lego/v4 v4.4.0 h1:uHhU5LpOYQOdp3aDU+XY2bajseu8fuExphTL1Ss6/Fc
|
||||
github.com/go-acme/lego/v4 v4.4.0/go.mod h1:l3+tFUFZb590dWcqhWZegynUthtaHJbG2fevUpoOOE0=
|
||||
github.com/go-asn1-ber/asn1-ber v1.3.1/go.mod h1:hEBeB/ic+5LoWskz+yKT7vGhhPYkProFKoKdwZRWMe0=
|
||||
github.com/go-asn1-ber/asn1-ber v1.4.1/go.mod h1:hEBeB/ic+5LoWskz+yKT7vGhhPYkProFKoKdwZRWMe0=
|
||||
github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667 h1:BP4M0CvQ4S3TGls2FvczZtj5Re/2ZzkV9VwqPHH/3Bo=
|
||||
github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667/go.mod h1:hEBeB/ic+5LoWskz+yKT7vGhhPYkProFKoKdwZRWMe0=
|
||||
github.com/go-asn1-ber/asn1-ber v1.5.8 h1:H9AZkK22UOmfX8J84ubyaZxKJZ3FMHVwn8swoMML7iQ=
|
||||
github.com/go-asn1-ber/asn1-ber v1.5.8/go.mod h1:hEBeB/ic+5LoWskz+yKT7vGhhPYkProFKoKdwZRWMe0=
|
||||
github.com/go-chi/chi v4.0.2+incompatible/go.mod h1:eB3wogJHnLi3x/kFX2A+IbTBlXxmMeXJVKy9tTv1XzQ=
|
||||
github.com/go-chi/chi/v5 v5.3.1 h1:3j4HZLGZQ3JpMCrPJF/Jl3mYJfWLKBfNJ6quurUGCf8=
|
||||
github.com/go-chi/chi/v5 v5.3.1/go.mod h1:R+tYY2hNuVUUjxoPtqUdgBqevM9s9njzkTLutVsOCto=
|
||||
@@ -402,8 +400,8 @@ github.com/go-kit/log v0.2.0/go.mod h1:NwTd00d/i8cPZ3xOwwiv2PO5MOcx78fFErGNcVmBj
|
||||
github.com/go-kit/log v0.2.1 h1:MRVx0/zhvdseW+Gza6N9rVzU/IVzaeE1SFI4raAhmBU=
|
||||
github.com/go-kit/log v0.2.1/go.mod h1:NwTd00d/i8cPZ3xOwwiv2PO5MOcx78fFErGNcVmBjv0=
|
||||
github.com/go-ldap/ldap/v3 v3.1.7/go.mod h1:5Zun81jBTabRaI8lzN7E1JjyEl1g6zI6u9pd8luAK4Q=
|
||||
github.com/go-ldap/ldap/v3 v3.4.13 h1:+x1nG9h+MZN7h/lUi5Q3UZ0fJ1GyDQYbPvbuH38baDQ=
|
||||
github.com/go-ldap/ldap/v3 v3.4.13/go.mod h1:LxsGZV6vbaK0sIvYfsv47rfh4ca0JXokCoKjZxsszv0=
|
||||
github.com/go-ldap/ldap/v3 v3.4.14 h1:D6PYdEgsaVzsXyr6w/yDC06Ria4uUhWm+Rb+er8lfAs=
|
||||
github.com/go-ldap/ldap/v3 v3.4.14/go.mod h1:S4eJUMUNjDkE0ZJtIZdybwyb03sGGLW6gxXT1Hs8VKA=
|
||||
github.com/go-ldap/ldif v0.0.0-20200320164324-fd88d9b715b3 h1:sfz1YppV05y4sYaW7kXZtrocU/+vimnIWt4cxAYh7+o=
|
||||
github.com/go-ldap/ldif v0.0.0-20200320164324-fd88d9b715b3/go.mod h1:ZXFhGda43Z2TVbfGZefXyMJzsDHhCh0go3bZUcwTx7o=
|
||||
github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
|
||||
@@ -412,8 +410,8 @@ github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG
|
||||
github.com/go-logfmt/logfmt v0.5.1 h1:otpy5pqBCBZ1ng9RQ0dPu4PN7ba75Y/aA+UpowDyNVA=
|
||||
github.com/go-logfmt/logfmt v0.5.1/go.mod h1:WYhtIu8zTZfxdn5+rREduYbwxfcBr/Vr6KEVveWlfTs=
|
||||
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
|
||||
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
|
||||
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||
github.com/go-logr/logr v1.4.4 h1:tG4xh9yMsRCAiodLVTxyrkzSZ9+o0L1Kg/+cPVcbP/8=
|
||||
github.com/go-logr/logr v1.4.4/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
|
||||
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
|
||||
github.com/go-micro/plugins/v4/client/grpc v1.2.1 h1:7xAwZRCO6mdUtBHsYIQs1/eCTdhCrnjF70GB+AVd6L0=
|
||||
@@ -715,8 +713,8 @@ github.com/kevinburke/ssh_config v1.2.0/go.mod h1:CT57kijsi8u/K/BOFA39wgDQJ9CxiF
|
||||
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
|
||||
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
|
||||
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
|
||||
github.com/klauspost/compress v1.18.6 h1:2jupLlAwFm95+YDR+NwD2MEfFO9d4z4Prjl1XXDjuao=
|
||||
github.com/klauspost/compress v1.18.6/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
|
||||
github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk=
|
||||
github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
|
||||
github.com/klauspost/cpuid/v2 v2.0.1/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
|
||||
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
|
||||
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
|
||||
@@ -801,23 +799,23 @@ github.com/mattn/go-colorable v0.1.7/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope
|
||||
github.com/mattn/go-colorable v0.1.8/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
|
||||
github.com/mattn/go-colorable v0.1.9/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
|
||||
github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=
|
||||
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
|
||||
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
|
||||
github.com/mattn/go-colorable v0.1.15 h1:+u9SLTRGnXv73cEsnsmoZBom+dMU88B2M0aDcWy0/jY=
|
||||
github.com/mattn/go-colorable v0.1.15/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
|
||||
github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
|
||||
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
|
||||
github.com/mattn/go-isatty v0.0.9/go.mod h1:YNRxwqDuOph6SZLI9vUUz6OYw3QyUt7WiY2yME+cCiQ=
|
||||
github.com/mattn/go-isatty v0.0.10/go.mod h1:qgIWMr58cqv1PHHyhnkY9lrL7etaEgOFcMEpPG5Rm84=
|
||||
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
|
||||
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
|
||||
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
||||
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
||||
github.com/mattn/go-isatty v0.0.22 h1:j8l17JJ9i6VGPUFUYoTUKPSgKe/83EYU2zBC7YNKMw4=
|
||||
github.com/mattn/go-isatty v0.0.22/go.mod h1:ZXfXG4SQHsB/w3ZeOYbR0PrPwLy+n6xiMrJlRFqopa4=
|
||||
github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
|
||||
github.com/mattn/go-runewidth v0.0.6/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
|
||||
github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
|
||||
github.com/mattn/go-runewidth v0.0.23 h1:7ykA0T0jkPpzSvMS5i9uoNn2Xy3R383f9HDx3RybWcw=
|
||||
github.com/mattn/go-runewidth v0.0.23/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
|
||||
github.com/mattn/go-sqlite3 v1.14.42 h1:MigqEP4ZmHw3aIdIT7T+9TLa90Z6smwcthx+Azv4Cgo=
|
||||
github.com/mattn/go-sqlite3 v1.14.42/go.mod h1:pjEuOr8IwzLJP2MfGeTb0A35jauH+C2kbHKBr7yXKVQ=
|
||||
github.com/mattn/go-runewidth v0.0.24 h1:cpokDiIn0MGnhdHwuWnJBITySJ20QyNGnY2kR/ay2DU=
|
||||
github.com/mattn/go-runewidth v0.0.24/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
|
||||
github.com/mattn/go-sqlite3 v1.14.49 h1:B8jBHC3xhxZgxztrgruTuLucebnULQnx4W7cF7SAE9w=
|
||||
github.com/mattn/go-sqlite3 v1.14.49/go.mod h1:6JTjA44L93a0QCyJef5YvlPoKXntQPjzWv5gtm9sB6w=
|
||||
github.com/mattn/go-tty v0.0.0-20180219170247-931426f7535a/go.mod h1:XPvLUNfbS4fJH25nqRHfWLMa1ONC8Amw+mIA639KxkE=
|
||||
github.com/mattn/go-tty v0.0.3/go.mod h1:ihxohKRERHTVzN+aSVRwACLCeqIoZAWpoICkkvrWyR0=
|
||||
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
|
||||
@@ -894,8 +892,8 @@ github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRW
|
||||
github.com/namedotcom/go v0.0.0-20180403034216-08470befbe04/go.mod h1:5sN+Lt1CaY4wsPvgQH/jsuJi4XO2ssZbdsIizr4CVC8=
|
||||
github.com/nats-io/jwt/v2 v2.8.2 h1:XXRgB60MSTnqsRwejQurVDs/hcv2dkt+86GjI+I/bMc=
|
||||
github.com/nats-io/jwt/v2 v2.8.2/go.mod h1:Ag/56sq9OblL4JgdYufDd16Egb17Kr/8WwwuO/forVc=
|
||||
github.com/nats-io/nats-server/v2 v2.14.3 h1:+xjydPt7rkit67G+04TN0mcO2n+8nveZE7tK/PPV53A=
|
||||
github.com/nats-io/nats-server/v2 v2.14.3/go.mod h1:5IlCtBzfwyzQzPMjmoJ9W2/LKmnJRtNyuOs/OT+NHDY=
|
||||
github.com/nats-io/nats-server/v2 v2.14.4 h1:efgjZ8cdExAKRuqSg8UPJFprb+l7NlBtSDPhDlw3rO4=
|
||||
github.com/nats-io/nats-server/v2 v2.14.4/go.mod h1:BltdpOYestjbtQSnVO2zGHdg5SGBZjt+GYTgB9LZq/I=
|
||||
github.com/nats-io/nats.go v1.52.0 h1:n3avV4VBsCgsdwh71TppsTwtv+QdPs7ntSKM8qJLGsc=
|
||||
github.com/nats-io/nats.go v1.52.0/go.mod h1:26HypzazeOkyO3/mqd1zZd53STJN0EjCYF9Uy2ZOBno=
|
||||
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
|
||||
@@ -936,16 +934,16 @@ github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7J
|
||||
github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=
|
||||
github.com/onsi/gomega v1.42.1 h1:iN1rCUX+44NZ1Dc97MPoeFYbFR0vh8zxoxMFwKdyZ6I=
|
||||
github.com/onsi/gomega v1.42.1/go.mod h1:REff/hsDsodHoKlWsP2mAPhu1+5/6hVYNf9rIEBpeSg=
|
||||
github.com/open-policy-agent/opa v1.18.2 h1:VBiLJpioTuk7XTW1JoQi4ILo+FVxD2/8uD8iP9/OcxY=
|
||||
github.com/open-policy-agent/opa v1.18.2/go.mod h1:9GY+hER4ZEXtxPlMjftVbqJJY9xLtCD3Q0oufRCfAKo=
|
||||
github.com/open-policy-agent/opa v1.19.0 h1:+j2OCsjMezZEML2T1lI9giJdGJS/PL1XFKgkHPGIhpo=
|
||||
github.com/open-policy-agent/opa v1.19.0/go.mod h1:pb6Y6klyf7X7X8uXNDflruA9dQC2gMqWROXI5w/kvv0=
|
||||
github.com/opencloud-eu/go-micro-plugins/v4/store/nats-js-kv v0.0.0-20250512152754-23325793059a h1:Sakl76blJAaM6NxylVkgSzktjo2dS504iDotEFJsh3M=
|
||||
github.com/opencloud-eu/go-micro-plugins/v4/store/nats-js-kv v0.0.0-20250512152754-23325793059a/go.mod h1:pjcozWijkNPbEtX5SIQaxEW/h8VAVZYTLx+70bmB3LY=
|
||||
github.com/opencloud-eu/icap-client v0.0.0-20250930132611-28a2afe62d89 h1:W1ms+lP5lUUIzjRGDg93WrQfZJZCaV1ZP3KeyXi8bzY=
|
||||
github.com/opencloud-eu/icap-client v0.0.0-20250930132611-28a2afe62d89/go.mod h1:vigJkNss1N2QEceCuNw/ullDehncuJNFB6mEnzfq9UI=
|
||||
github.com/opencloud-eu/libre-graph-api-go v1.0.8-0.20260310090739-853d972b282d h1:JcqGDiyrcaQwVyV861TUyQgO7uEmsjkhfm7aQd84dOw=
|
||||
github.com/opencloud-eu/libre-graph-api-go v1.0.8-0.20260310090739-853d972b282d/go.mod h1:pzatilMEHZFT3qV7C/X3MqOa3NlRQuYhlRhZTL+hN6Q=
|
||||
github.com/opencloud-eu/reva/v2 v2.47.0 h1:bYul45qS8GmmN9PKplSZ78ZTZ6A+9xp/FfogqFVud18=
|
||||
github.com/opencloud-eu/reva/v2 v2.47.0/go.mod h1:zdpEKIMDT14w+MGUOWAi+rh+PAZBPUlb7AtIGFWx7Ds=
|
||||
github.com/opencloud-eu/reva/v2 v2.48.0 h1:G/4Jbv0DWWOfA5u5DtV0CB75pi9Wwtj7JkJQOEBvErs=
|
||||
github.com/opencloud-eu/reva/v2 v2.48.0/go.mod h1:ZCo/xQM6if+upZa7rJCmdifZ/Y5XHLCrscHytC39yI4=
|
||||
github.com/opencloud-eu/secure v0.0.0-20260312082735-b6f5cb2244e4 h1:l2oB/RctH+t8r7QBj5p8thfEHCM/jF35aAY3WQ3hADI=
|
||||
github.com/opencloud-eu/secure v0.0.0-20260312082735-b6f5cb2244e4/go.mod h1:BmF5hyM6tXczk3MpQkFf1hpKSRqCyhqcbiQtiAF7+40=
|
||||
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
|
||||
@@ -1016,8 +1014,8 @@ github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqr
|
||||
github.com/prometheus/client_golang v1.12.1/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY=
|
||||
github.com/prometheus/client_golang v1.12.2/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY=
|
||||
github.com/prometheus/client_golang v1.13.0/go.mod h1:vTeo+zgvILHsnnj/39Ou/1fPN5nJFOEMgftOUOmlvYQ=
|
||||
github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o=
|
||||
github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg=
|
||||
github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU=
|
||||
github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE=
|
||||
github.com/prometheus/client_model v0.0.0-20170216185247-6f3806018612/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
|
||||
github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
|
||||
github.com/prometheus/client_model v0.0.0-20190115171406-56726106282f/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
|
||||
@@ -1037,8 +1035,8 @@ github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9
|
||||
github.com/prometheus/common v0.32.1/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls=
|
||||
github.com/prometheus/common v0.35.0/go.mod h1:phzohg0JFMnBEFGxTDbfu3QyL5GI8gTQJFhYO5B3mfA=
|
||||
github.com/prometheus/common v0.37.0/go.mod h1:phzohg0JFMnBEFGxTDbfu3QyL5GI8gTQJFhYO5B3mfA=
|
||||
github.com/prometheus/common v0.67.5 h1:pIgK94WWlQt1WLwAC5j2ynLaBRDiinoAb86HZHTUGI4=
|
||||
github.com/prometheus/common v0.67.5/go.mod h1:SjE/0MzDEEAyrdr5Gqc6G+sXI67maCxzaT3A2+HqjUw=
|
||||
github.com/prometheus/common v0.70.1 h1:1HvjP4D5oL3t8RsPlwxA9onvvStjtIHYE5XuuwOi/PY=
|
||||
github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
|
||||
github.com/prometheus/procfs v0.0.0-20170703101242-e645f4e5aaa8/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
|
||||
github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
|
||||
github.com/prometheus/procfs v0.0.0-20190117184657-bf6a532e95b1/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
|
||||
@@ -1049,8 +1047,8 @@ github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4O
|
||||
github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
|
||||
github.com/prometheus/procfs v0.7.3/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
|
||||
github.com/prometheus/procfs v0.8.0/go.mod h1:z7EfXMXOkbkqb9IINtpCn86r/to3BnA0uaxHdg830/4=
|
||||
github.com/prometheus/procfs v0.20.1 h1:XwbrGOIplXW/AU3YhIhLODXMJYyC1isLFfYCsTEycfc=
|
||||
github.com/prometheus/procfs v0.20.1/go.mod h1:o9EMBZGRyvDrSPH1RqdxhojkuXstoe4UlK79eF5TGGo=
|
||||
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
|
||||
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
|
||||
github.com/prometheus/statsd_exporter v0.22.7/go.mod h1:N/TevpjkIh9ccs6nuzY3jQn9dFqnUakOjnEuMPJJJnI=
|
||||
github.com/prometheus/statsd_exporter v0.22.8 h1:Qo2D9ZzaQG+id9i5NYNGmbf1aa/KxKbB9aKfMS+Yib0=
|
||||
github.com/prometheus/statsd_exporter v0.22.8/go.mod h1:/DzwbTEaFTE0Ojz5PqcSk6+PFHOPWGxdXVr6yC8eFOM=
|
||||
@@ -1102,10 +1100,10 @@ github.com/sercand/kuberesolver/v5 v5.1.1 h1:CYH+d67G0sGBj7q5wLK61yzqJJ8gLLC8aep
|
||||
github.com/sercand/kuberesolver/v5 v5.1.1/go.mod h1:Fs1KbKhVRnB2aDWN12NjKCB+RgYMWZJ294T3BtmVCpQ=
|
||||
github.com/sergi/go-diff v1.4.0 h1:n/SP9D5ad1fORl+llWyN+D6qoUETXNZARKjyY2/KVCw=
|
||||
github.com/sergi/go-diff v1.4.0/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
|
||||
github.com/sethvargo/go-diceware v0.5.0 h1:exrQ7GpaBo00GqRVM1N8ChXSsi3oS7tjQiIehsD+yR0=
|
||||
github.com/sethvargo/go-diceware v0.5.0/go.mod h1:Lg1SyPS7yQO6BBgTN5r4f2MUDkqGfLWsOjHPY0kA8iw=
|
||||
github.com/sethvargo/go-password v0.3.1 h1:WqrLTjo7X6AcVYfC6R7GtSyuUQR9hGyAj/f1PYQZCJU=
|
||||
github.com/sethvargo/go-password v0.3.1/go.mod h1:rXofC1zT54N7R8K/h1WDUdkf9BOx5OptoxrMBcrXzvs=
|
||||
github.com/sethvargo/go-diceware v0.6.0 h1:B3nhMhbBP7KwtTQ7hHRIOmv5FqeD8bJs77RFrV24iWk=
|
||||
github.com/sethvargo/go-diceware v0.6.0/go.mod h1:lHmdB0xuWaJ06KCraW6bztRT+71Dp+lsXQvborhhsBc=
|
||||
github.com/sethvargo/go-password v0.4.0 h1:eSidVKQw5C7CmTDAtH3RipBTSjdU1ZRxQaynD2GWLVU=
|
||||
github.com/sethvargo/go-password v0.4.0/go.mod h1:PO3nYHwUpcHPR0F9woy7a4abZPvzRuqJr0GaeIYTm3k=
|
||||
github.com/shamaton/msgpack/v2 v2.4.1 h1:JtJ141QoQ3NqgPDsjq2v9VXlaON8SiQOwEaoNLEK/MQ=
|
||||
github.com/shamaton/msgpack/v2 v2.4.1/go.mod h1:6khjYnkx73f7VQU7wjcFS9DFjs+59naVWJv1TB7qdOI=
|
||||
github.com/shirou/gopsutil v3.21.11+incompatible h1:+1+c1VGhc88SSonWP6foOcLhvnKlUeu/erjjvaPEYiI=
|
||||
@@ -1189,10 +1187,10 @@ github.com/testcontainers/testcontainers-go v0.43.0 h1:oEQx5MW2DGd9z3AeEQfB2lPM0
|
||||
github.com/testcontainers/testcontainers-go v0.43.0/go.mod h1:+VxkT2NQnKOZPKi6praMuMKYHYyOGXr0XSBSlSMCzFo=
|
||||
github.com/testcontainers/testcontainers-go/modules/opensearch v0.43.0 h1:a1ipjF7d/VxPX1dgVPIk4F+t6YkgMbE2OtBuRQCHJt8=
|
||||
github.com/testcontainers/testcontainers-go/modules/opensearch v0.43.0/go.mod h1:OWSeUDiGMUy30iMsAltIJIo9uh/CleLv6KyxjYOsgR8=
|
||||
github.com/tetratelabs/wazero v1.12.0 h1:DuWcpNu/FzgEXgGBDp8J1Spc+CWOvvtvVyjKlaZopYU=
|
||||
github.com/tetratelabs/wazero v1.12.0/go.mod h1:LvKtzl2RqO4gyF27BiXU+nKAjcV8f38U+kP/q2vgxh0=
|
||||
github.com/thanhpk/randstr v1.0.6 h1:psAOktJFD4vV9NEVb3qkhRSMvYh4ORRaj1+w/hn4B+o=
|
||||
github.com/thanhpk/randstr v1.0.6/go.mod h1:M/H2P1eNLZzlDwAzpkkkUvoyNNMbzRGhESZuEQk3r0U=
|
||||
github.com/theckman/yacspin v0.13.12 h1:CdZ57+n0U6JMuh2xqjnjRq5Haj6v1ner2djtLQRzJr4=
|
||||
github.com/theckman/yacspin v0.13.12/go.mod h1:Rd2+oG2LmQi5f3zC3yeZAOl245z8QOvrH4OPOJNZxLg=
|
||||
github.com/thejerf/suture/v4 v4.0.6 h1:QsuCEsCqb03xF9tPAsWAj8QOAJBgQI1c0VqJNaingg8=
|
||||
github.com/thejerf/suture/v4 v4.0.6/go.mod h1:gu9Y4dXNUWFrByqRt30Rm9/UZ0wzRSt9AJS6xu/ZGxU=
|
||||
github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
||||
@@ -1229,8 +1227,8 @@ github.com/valyala/fastjson v1.6.10 h1:/yjJg8jaVQdYR3arGxPE2X5z89xrlhS0eGXdv+ADT
|
||||
github.com/valyala/fastjson v1.6.10/go.mod h1:e6FubmQouUNP73jtMLmcbxS6ydWIpOfhz34TSfO3JaE=
|
||||
github.com/valyala/fasttemplate v1.0.1/go.mod h1:UQGH1tvbgY+Nz5t2n7tXsz52dQxojPUpymEIMZ47gx8=
|
||||
github.com/valyala/fasttemplate v1.1.0/go.mod h1:UQGH1tvbgY+Nz5t2n7tXsz52dQxojPUpymEIMZ47gx8=
|
||||
github.com/vektah/gqlparser/v2 v2.5.34 h1:MEea5P0qhdcqfBL45ghKE+qr9laidVHTMHjav5h7ckk=
|
||||
github.com/vektah/gqlparser/v2 v2.5.34/go.mod h1:mFdHLGCio7OGX1fby9ZjTW6FN+qxgmbnBcRIeeScE5s=
|
||||
github.com/vektah/gqlparser/v2 v2.5.36 h1:CN9mKVHgMkc+XftdOWIhb4HEL8wKSYkFAqhf8booa7s=
|
||||
github.com/vektah/gqlparser/v2 v2.5.36/go.mod h1:cAJ9qwVgPaUkWv6Gn8vn0mqOE0Ui5Pn56wNy5396XWo=
|
||||
github.com/vinyldns/go-vinyldns v0.0.0-20200917153823-148a5f6b8f14/go.mod h1:RWc47jtnVuQv6+lY3c768WtXCas/Xi+U5UFc5xULmYg=
|
||||
github.com/vmihailenco/msgpack/v5 v5.4.1 h1:cQriyiUvjTwOHg8QZaPihLWeRAAVoCpE00IUPn0Bjt8=
|
||||
github.com/vmihailenco/msgpack/v5 v5.4.1/go.mod h1:GaZTsDaehaPpQVyxrf5mtQlH+pc21PIudVV/E3rRQok=
|
||||
@@ -1356,8 +1354,8 @@ golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0
|
||||
golang.org/x/crypto v0.14.0/go.mod h1:MVFd36DqK4CsrnJYDkBA3VC4m2GkXAM0PvzMCn4JQf4=
|
||||
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
|
||||
golang.org/x/crypto v0.21.0/go.mod h1:0BP7YvVV9gBbVKyeTG0Gyn+gZm94bibOW5BjDEYAOMs=
|
||||
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
|
||||
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
|
||||
golang.org/x/crypto v0.54.0 h1:YLIA59K4fiNzHzjnZt2tUJQjQtUWfWbeHBqKtk3eScw=
|
||||
golang.org/x/crypto v0.54.0/go.mod h1:KWL8ny2AZdGR2cWmzeHrp2azQPGogOv+HeQaVEXC2dk=
|
||||
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
|
||||
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
|
||||
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
|
||||
@@ -1447,8 +1445,8 @@ golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
|
||||
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
|
||||
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
|
||||
golang.org/x/net v0.23.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
|
||||
golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o=
|
||||
golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec=
|
||||
golang.org/x/net v0.57.0 h1:K5+3DljvIuDG9/Jv9rvyMywYNFCQ9RSUY6OOTTkT+tE=
|
||||
golang.org/x/net v0.57.0/go.mod h1:KpXc8iv+r3XplLAG/f7Jsf9RPszJzdR0f58q9vGOuEU=
|
||||
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
|
||||
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
|
||||
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
|
||||
@@ -1705,8 +1703,8 @@ google.golang.org/genproto v0.0.0-20260319201613-d00831a3d3e7 h1:XzmzkmB14QhVhgn
|
||||
google.golang.org/genproto v0.0.0-20260319201613-d00831a3d3e7/go.mod h1:L43LFes82YgSonw6iTXTxXUX1OlULt4AQtkik4ULL/I=
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa h1:Kjn0N0tCrDgiAFW+lGO4JZ3ck44CehvJQMAwj9QF0G8=
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa/go.mod h1:q4lMZS6kskjT5HvCPrnnypcDPVJqT/f4nfxmkE7gryY=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa h1:mZHHdPZl0dbGHCflZgAq/Q468DWVFcU2whhB2KAo8fk=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260729162451-8efbd57d26e0 h1:mJiOtnGp0k/BcSgdu03G2NwnscCfCH+h2QKUBZr18KI=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260729162451-8efbd57d26e0/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
|
||||
google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs=
|
||||
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
|
||||
google.golang.org/grpc v1.19.1/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
|
||||
@@ -1722,8 +1720,8 @@ google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3Iji
|
||||
google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
|
||||
google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
|
||||
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
|
||||
google.golang.org/grpc v1.82.0 h1:vguDnZUPjE26w09A63VoxZPnvPjB5Riyc0mkXPFmAIU=
|
||||
google.golang.org/grpc v1.82.0/go.mod h1:yzTZ1TB1Z3SG+LIYaI+WiE8D5+PZ3ArnrSp8zF3+/ZA=
|
||||
google.golang.org/grpc v1.83.0 h1:JeNZEKJFbQxArAMl+hiytHauacDNqJUllNfmIMmpqnQ=
|
||||
google.golang.org/grpc v1.83.0/go.mod h1:kDyl6SKsiHKt0uylY5gtn5cEjkrIOhQOGDgIc4JGwzQ=
|
||||
google.golang.org/grpc/examples v0.0.0-20211102180624-670c133e568e h1:m7aQHHqd0q89mRwhwS9Bx2rjyl/hsFAeta+uGrHsQaU=
|
||||
google.golang.org/grpc/examples v0.0.0-20211102180624-670c133e568e/go.mod h1:gID3PKrg7pWKntu9Ss6zTLJ0ttC0X9IHgREOCZwbCVU=
|
||||
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
|
||||
|
||||
+128
-413
@@ -1,17 +1,16 @@
|
||||
package command
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/opencloud-eu/opencloud/opencloud/pkg/register"
|
||||
"github.com/opencloud-eu/opencloud/pkg/config"
|
||||
"github.com/opencloud-eu/opencloud/pkg/config/configlog"
|
||||
"github.com/opencloud-eu/opencloud/pkg/config/parser"
|
||||
"github.com/opencloud-eu/opencloud/pkg/x/path/filepathx"
|
||||
storageUsersParser "github.com/opencloud-eu/opencloud/services/storage-users/pkg/config/parser"
|
||||
"github.com/opencloud-eu/opencloud/services/storage-users/pkg/event"
|
||||
"github.com/opencloud-eu/opencloud/services/storage-users/pkg/revaconfig"
|
||||
@@ -19,27 +18,10 @@ import (
|
||||
"github.com/opencloud-eu/reva/v2/pkg/storage/fs/posix/ignore"
|
||||
"github.com/opencloud-eu/reva/v2/pkg/storage/fs/posix/options"
|
||||
"github.com/opencloud-eu/reva/v2/pkg/storage/fs/registry"
|
||||
"github.com/opencloud-eu/reva/v2/pkg/storage/pkg/decomposedfs/metadata/prefixes"
|
||||
|
||||
"github.com/pkg/xattr"
|
||||
"github.com/rs/zerolog"
|
||||
"github.com/spf13/cobra"
|
||||
"github.com/theckman/yacspin"
|
||||
"github.com/vmihailenco/msgpack/v5"
|
||||
)
|
||||
|
||||
// Define the names of the extended attributes we are working with.
|
||||
const (
|
||||
parentIDAttrName = "user.oc.parentid"
|
||||
idAttrName = "user.oc.id"
|
||||
nameAttrName = "user.oc.name"
|
||||
spaceIDAttrName = "user.oc.space.id"
|
||||
ownerIDAttrName = "user.oc.owner.id"
|
||||
)
|
||||
|
||||
var (
|
||||
spinner *yacspin.Spinner
|
||||
restartRequired = false
|
||||
ignorer *ignore.Ignorer
|
||||
)
|
||||
|
||||
type IDCacher interface {
|
||||
@@ -94,6 +76,39 @@ func scanCmd(ocCfg *config.Config) *cobra.Command {
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
storageRoot := cfg.Drivers.Posix.Root
|
||||
root := storageRoot
|
||||
defaultRoot := true
|
||||
if v, err := cmd.Flags().GetString("basepath"); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Failed to parse command-line parameter '--basepath': %v\n", err)
|
||||
os.Exit(1)
|
||||
} else if v != "" {
|
||||
root = v
|
||||
if !filepath.IsAbs(v) {
|
||||
if v, err = filepath.Abs(v); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Failed to make the basepath mentioned using '--basepath' absolute: %v\n", err)
|
||||
os.Exit(1)
|
||||
} else {
|
||||
root = v
|
||||
}
|
||||
} else {
|
||||
root = v
|
||||
}
|
||||
root = filepath.Clean(root)
|
||||
defaultRoot = false
|
||||
}
|
||||
|
||||
// ensure that, if a basepath has been indicated, it is under the storage root
|
||||
if !defaultRoot {
|
||||
if contained, err := filepathx.IsSameOrContainedBy(storageRoot, root); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Failed to determine whether the specified basepath %q is contained by the storage root %q: %v\n", root, storageRoot, err)
|
||||
os.Exit(1)
|
||||
} else if !contained {
|
||||
fmt.Fprintf(os.Stderr, "The specified basepath %q is neither the storage root %q, nor a subdirectory thereof, nor a file underneath it\n", root, storageRoot)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
// We want to initialize the driver but disable scanfs on boot, so we can trigger it manually afterwards
|
||||
drivers := revaconfig.StorageProviderDrivers(cfg)
|
||||
drivers["posix"] = revaconfig.Posix(cfg, false, false)
|
||||
@@ -105,6 +120,11 @@ func scanCmd(ocCfg *config.Config) *cobra.Command {
|
||||
fmt.Fprintf(os.Stderr, "Failed to create event stream for posix driver: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
log := logger("posixfs")
|
||||
|
||||
if !defaultRoot {
|
||||
log = log.With().Str("basepath", root).Logger()
|
||||
}
|
||||
|
||||
f, ok := registry.NewFuncs["posix"]
|
||||
if !ok {
|
||||
@@ -112,7 +132,7 @@ func scanCmd(ocCfg *config.Config) *cobra.Command {
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
fs, err := f(drivers["posix"].(map[string]any), fsStream, nil)
|
||||
fs, err := f(drivers["posix"].(map[string]any), fsStream, &log)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Failed to initialize filesystem driver '%s': %v\n", cfg.Driver, err)
|
||||
return err
|
||||
@@ -124,8 +144,12 @@ func scanCmd(ocCfg *config.Config) *cobra.Command {
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
fmt.Println("Starting posixfs scan...")
|
||||
err = cacher.WarmupIDCache(cfg.Drivers.Posix.Root, true, false)
|
||||
if defaultRoot {
|
||||
fmt.Println("Starting posixfs scan...")
|
||||
} else {
|
||||
fmt.Printf("Starting posixfs scan at '%s'...\n", root)
|
||||
}
|
||||
err = cacher.WarmupIDCache(root, true, false)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Scan failed: %v\n", err)
|
||||
return err
|
||||
@@ -135,407 +159,98 @@ func scanCmd(ocCfg *config.Config) *cobra.Command {
|
||||
return nil
|
||||
},
|
||||
}
|
||||
cmd.Flags().StringP("basepath", "p", "", "the root under which to scan files, which may be a directory or a file (when omitted, detaults to using the storage root)")
|
||||
return cmd
|
||||
}
|
||||
|
||||
// consistencyCmd returns a command to check the consistency of the posixfs storage.
|
||||
func consistencyCmd(cfg *config.Config) *cobra.Command {
|
||||
func consistencyCmd(ocCfg *config.Config) *cobra.Command {
|
||||
consCmd := &cobra.Command{
|
||||
Use: "consistency",
|
||||
Short: "check the consistency of the posixfs storage",
|
||||
Use: "consistency [path ...]",
|
||||
Short: "Check the consistency of the posixfs storage",
|
||||
Long: `Check the consistency of the posixfs storage.
|
||||
|
||||
You can specify one or more paths to limit the scope of the check.
|
||||
If no path is provided, the whole storage is checked.
|
||||
|
||||
The provided arguments determines the scope of the check:
|
||||
- a storage root: the whole storage (all personal and project spaces) is checked
|
||||
- a space root: only that space is checked
|
||||
- a file or directory: only that single entity is checked (and its children, if it is a directory)`,
|
||||
Args: cobra.ArbitraryArgs,
|
||||
PreRunE: func(cmd *cobra.Command, args []string) error {
|
||||
|
||||
if err := parser.ParseConfig(ocCfg, true); err != nil {
|
||||
return configlog.ReturnError(err)
|
||||
}
|
||||
|
||||
// Parse storage users config
|
||||
ocCfg.StorageUsers.Commons = ocCfg.Commons
|
||||
|
||||
return configlog.ReturnFatal(storageUsersParser.ParseConfig(ocCfg.StorageUsers))
|
||||
},
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
return checkPosixfsConsistency(cmd, cfg)
|
||||
cfg := ocCfg.StorageUsers
|
||||
if len(args) == 0 {
|
||||
args = []string{cfg.Drivers.Posix.Root}
|
||||
}
|
||||
log := logger("posixfs")
|
||||
recalculateChecksums, _ := cmd.Flags().GetBool("fix-checksums")
|
||||
|
||||
drivers := revaconfig.StorageProviderDrivers(cfg)
|
||||
drivers["posix"] = revaconfig.Posix(cfg, false, false)
|
||||
opts, err := options.New(drivers["posix"].(map[string]any))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
ignorer := ignore.NewIgnorer(opts, &log)
|
||||
|
||||
checker := &consistencyChecker{
|
||||
cfg: cfg,
|
||||
ignorer: ignorer,
|
||||
recalculateChecksums: recalculateChecksums,
|
||||
}
|
||||
|
||||
return checker.Check(args)
|
||||
},
|
||||
}
|
||||
consCmd.Flags().StringP("root", "r", "", "Path to the root directory of the posixfs storage")
|
||||
_ = consCmd.MarkFlagRequired("root")
|
||||
consCmd.Flags().Bool("fix-checksums", false, "Recalculate and fix the file checksums. This reads every file and can be slow on large storages.")
|
||||
|
||||
return consCmd
|
||||
}
|
||||
|
||||
// checkPosixfsConsistency checks the consistency of the posixfs storage.
|
||||
func checkPosixfsConsistency(cmd *cobra.Command, cfg *config.Config) error {
|
||||
rootPath, _ := cmd.Flags().GetString("root")
|
||||
indexesPath := filepath.Join(rootPath, "indexes")
|
||||
|
||||
opt, _ := options.New(map[string]interface{}{
|
||||
"root": rootPath,
|
||||
})
|
||||
log := zerolog.Nop()
|
||||
ignorer = ignore.NewIgnorer(opt, &log)
|
||||
|
||||
_, err := os.Stat(indexesPath)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return fmt.Errorf("consistency check failed: '%s' is not a posixfs root", rootPath)
|
||||
}
|
||||
return fmt.Errorf("error accessing '%s': %w", indexesPath, err)
|
||||
}
|
||||
|
||||
spinnerCfg := yacspin.Config{
|
||||
Frequency: 100 * time.Millisecond,
|
||||
CharSet: yacspin.CharSets[11],
|
||||
StopCharacter: "✓",
|
||||
StopColors: []string{"fgGreen"},
|
||||
StopFailCharacter: "✗",
|
||||
StopFailColors: []string{"fgRed"},
|
||||
}
|
||||
|
||||
spinner, err = yacspin.New(spinnerCfg)
|
||||
err = spinner.Start()
|
||||
if err != nil {
|
||||
return fmt.Errorf("error creating spinner: %w", err)
|
||||
}
|
||||
|
||||
checkSpaces(filepath.Join(rootPath, "users"))
|
||||
spinner.Suffix(" Personal spaces check ")
|
||||
spinner.StopMessage("completed\n")
|
||||
spinner.Stop()
|
||||
|
||||
checkSpaces(filepath.Join(rootPath, "projects"))
|
||||
spinner.Suffix(" Project spaces check ")
|
||||
spinner.StopMessage("completed")
|
||||
spinner.Stop()
|
||||
|
||||
if restartRequired {
|
||||
fmt.Println("\n\n ⚠️ Please restart your openCloud instance to apply changes.")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func checkSpaces(basePath string) {
|
||||
dirEntries, err := os.ReadDir(basePath)
|
||||
if err != nil {
|
||||
spinner.Message(fmt.Sprintf("Error reading spaces directory '%s'\n", basePath))
|
||||
spinner.StopFail()
|
||||
return
|
||||
}
|
||||
|
||||
for _, entry := range dirEntries {
|
||||
if entry.IsDir() {
|
||||
fullPath := filepath.Join(basePath, entry.Name())
|
||||
checkSpace(fullPath)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func checkSpace(spacePath string) {
|
||||
spinner.Message("")
|
||||
spinner.Suffix(fmt.Sprintf(" Checking space '%s'", spacePath))
|
||||
|
||||
info, err := os.Stat(spacePath)
|
||||
if err != nil {
|
||||
logFailure("Error accessing path '%s': %v", spacePath, err)
|
||||
return
|
||||
}
|
||||
if !info.IsDir() {
|
||||
logFailure("Error: The provided path '%s' is not a directory\n", spacePath)
|
||||
return
|
||||
}
|
||||
|
||||
spaceID, err := xattr.Get(spacePath, spaceIDAttrName)
|
||||
if err != nil || len(spaceID) == 0 {
|
||||
logFailure("Error: The directory '%s' does not seem to be a space root, it's missing the '%s' attribute\n", spacePath, spaceIDAttrName)
|
||||
return
|
||||
}
|
||||
|
||||
checkSpaceID(spacePath)
|
||||
checkNodeIDs(spacePath)
|
||||
}
|
||||
|
||||
func checkSpaceID(spacePath string) {
|
||||
spinner.Message(" - checking space ID uniqueness")
|
||||
|
||||
entries, uniqueIDs, oldestEntry, err := gatherAttributes(spacePath)
|
||||
if err != nil {
|
||||
logFailure("Failed to gather attributes: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
if len(entries) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
if len(uniqueIDs) > 1 {
|
||||
spinner.Pause()
|
||||
fmt.Println("\n ⚠ Multiple space IDs found:")
|
||||
for id := range uniqueIDs {
|
||||
fmt.Printf(" - %s\n", id)
|
||||
}
|
||||
|
||||
fmt.Printf("\n ⏳ Oldest entry is '%s' (modified on %s).\n",
|
||||
filepath.Base(oldestEntry.Path), oldestEntry.ModTime.Format(time.RFC1123))
|
||||
|
||||
targetID := oldestEntry.ParentID
|
||||
fmt.Printf(" ✅ Proposed target Parent ID: %s\n", targetID)
|
||||
|
||||
fmt.Printf("\n Do you want to unify all parent IDs to '%s'? This will modify %d entries, the directory, and the user index. (y/N): ", targetID, len(entries))
|
||||
|
||||
reader := bufio.NewReader(os.Stdin)
|
||||
input, _ := reader.ReadString('\n')
|
||||
input = strings.TrimSpace(strings.ToLower(input))
|
||||
|
||||
if input != "y" {
|
||||
spinner.Unpause()
|
||||
logFailure("Operation cancelled by user.")
|
||||
return
|
||||
}
|
||||
restartRequired = true
|
||||
|
||||
obsoleteIDs := []string{}
|
||||
for id := range uniqueIDs {
|
||||
if id != targetID {
|
||||
obsoleteIDs = append(obsoleteIDs, id)
|
||||
}
|
||||
}
|
||||
fixSpaceID(spacePath, obsoleteIDs, targetID, entries)
|
||||
spinner.Unpause()
|
||||
}
|
||||
}
|
||||
|
||||
func walkNodes(dir string, parentID string) int {
|
||||
fixes := 0
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
logFailure("Error reading directory '%s': %v", dir, err)
|
||||
return 0
|
||||
}
|
||||
|
||||
for _, entry := range entries {
|
||||
fullPath := filepath.Join(dir, entry.Name())
|
||||
|
||||
if ignorer.IsIgnored(fullPath) {
|
||||
continue
|
||||
}
|
||||
|
||||
// Check if the parent ID attribute matches the expected parent ID, if not, fix it.
|
||||
actualParentID, err := xattr.Get(fullPath, parentIDAttrName)
|
||||
if err != nil || string(actualParentID) != parentID {
|
||||
err = xattr.Set(fullPath, parentIDAttrName, []byte(parentID))
|
||||
if err != nil {
|
||||
logFailure("Failed to fix parent ID for '%s': %v", fullPath, err)
|
||||
} else {
|
||||
spinner.Pause()
|
||||
fmt.Printf(" + Fixed parent ID for '%s'", fullPath)
|
||||
spinner.Unpause()
|
||||
fixes++
|
||||
restartRequired = true
|
||||
}
|
||||
}
|
||||
|
||||
// Check that the name attribute matches the actual name of the file/directory, if not, fix it.
|
||||
nameAttr, err := xattr.Get(fullPath, nameAttrName)
|
||||
if err != nil || string(nameAttr) != entry.Name() {
|
||||
err = xattr.Set(fullPath, nameAttrName, []byte(entry.Name()))
|
||||
if err != nil {
|
||||
logFailure("Failed to fix name attribute for '%s': %v", fullPath, err)
|
||||
} else {
|
||||
spinner.Pause()
|
||||
fmt.Printf(" + Fixed name attribute for '%s'", fullPath)
|
||||
spinner.Unpause()
|
||||
fixes++
|
||||
restartRequired = true
|
||||
}
|
||||
}
|
||||
|
||||
if entry.IsDir() {
|
||||
nodeID, err := xattr.Get(fullPath, idAttrName)
|
||||
if err != nil || len(nodeID) == 0 {
|
||||
logFailure("Directory '%s' missing '%s', skipping its children", fullPath, idAttrName)
|
||||
continue
|
||||
}
|
||||
walkNodes(fullPath, string(nodeID))
|
||||
}
|
||||
}
|
||||
return fixes
|
||||
}
|
||||
|
||||
func checkNodeIDs(spacePath string) {
|
||||
spinner.Message(" - checking nodes")
|
||||
|
||||
rootID, err := xattr.Get(spacePath, idAttrName)
|
||||
if err != nil || len(rootID) == 0 {
|
||||
logFailure("Space root '%s' missing '%s' attribute", spacePath, idAttrName)
|
||||
return
|
||||
}
|
||||
|
||||
fixes := walkNodes(spacePath, string(rootID))
|
||||
|
||||
if fixes > 0 {
|
||||
spinner.Pause()
|
||||
fmt.Printf("\n ✓ Fixed %d incorrect node attributes in %s\n", fixes, filepath.Base(spacePath))
|
||||
spinner.Unpause()
|
||||
}
|
||||
}
|
||||
|
||||
func fixSpaceID(spacePath string, obsoleteIDs []string, targetID string, entries []EntryInfo) {
|
||||
// Set all parentid attributes to the proper space ID
|
||||
err := setAllParentIDAttributes(entries, targetID)
|
||||
if err != nil {
|
||||
logFailure("an error occurred during file attribute update: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
// Update space ID itself
|
||||
fmt.Printf(" Updating directory '%s' with attribute '%s' -> %s\n", filepath.Base(spacePath), idAttrName, targetID)
|
||||
err = xattr.Set(spacePath, idAttrName, []byte(targetID))
|
||||
if err != nil {
|
||||
logFailure("Failed to set attribute on directory '%s': %v", spacePath, err)
|
||||
return
|
||||
}
|
||||
err = xattr.Set(spacePath, spaceIDAttrName, []byte(targetID))
|
||||
if err != nil {
|
||||
logFailure("Failed to set attribute on directory '%s': %v", spacePath, err)
|
||||
return
|
||||
}
|
||||
|
||||
// update the index
|
||||
err = updateOwnerIndexFile(spacePath, obsoleteIDs)
|
||||
if err != nil {
|
||||
logFailure("Could not update the owner index file: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func gatherAttributes(path string) ([]EntryInfo, map[string]struct{}, EntryInfo, error) {
|
||||
dirEntries, err := os.ReadDir(path)
|
||||
if err != nil {
|
||||
return nil, nil, EntryInfo{}, fmt.Errorf("failed to read directory: %w", err)
|
||||
}
|
||||
|
||||
var allEntries []EntryInfo
|
||||
uniqueIDs := make(map[string]struct{})
|
||||
var oldestEntry EntryInfo
|
||||
oldestTime := time.Now().Add(100 * 365 * 24 * time.Hour) // Set to a future date to find the oldest entry
|
||||
|
||||
for _, entry := range dirEntries {
|
||||
fullPath := filepath.Join(path, entry.Name())
|
||||
if ignorer.IsIgnored(fullPath) {
|
||||
continue
|
||||
}
|
||||
info, err := os.Stat(fullPath)
|
||||
if err != nil {
|
||||
fmt.Printf(" - Warning: could not stat %s: %v\n", entry.Name(), err)
|
||||
continue
|
||||
}
|
||||
|
||||
parentID, err := xattr.Get(fullPath, parentIDAttrName)
|
||||
if err != nil {
|
||||
continue // Skip if attribute doesn't exist or can't be read
|
||||
}
|
||||
|
||||
entryInfo := EntryInfo{
|
||||
Path: fullPath,
|
||||
ModTime: info.ModTime(),
|
||||
ParentID: string(parentID),
|
||||
}
|
||||
|
||||
allEntries = append(allEntries, entryInfo)
|
||||
uniqueIDs[string(parentID)] = struct{}{}
|
||||
|
||||
if entryInfo.ModTime.Before(oldestTime) {
|
||||
oldestTime = entryInfo.ModTime
|
||||
oldestEntry = entryInfo
|
||||
}
|
||||
}
|
||||
|
||||
return allEntries, uniqueIDs, oldestEntry, nil
|
||||
}
|
||||
|
||||
func setAllParentIDAttributes(entries []EntryInfo, targetID string) error {
|
||||
fmt.Printf(" Setting all parent IDs to '%s':\n", targetID)
|
||||
|
||||
for _, entry := range entries {
|
||||
if entry.ParentID == targetID {
|
||||
fmt.Printf(" - Skipping '%s' (already has target ID).\n", filepath.Base(entry.Path))
|
||||
continue
|
||||
}
|
||||
|
||||
fmt.Printf(" - Removing all attributes from '%s'. It will be re-assimilated\n", filepath.Base(entry.Path))
|
||||
filepath.WalkDir(entry.Path, func(path string, d os.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return fmt.Errorf("error walking path '%s': %w", path, err)
|
||||
}
|
||||
|
||||
// Remove all attributes from the file.
|
||||
if err := removeAttributes(path); err != nil {
|
||||
fmt.Printf("failed to remove attributes from '%s': %v", path, err)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// updateOwnerIndexFile handles the logic of reading, modifying, and writing the MessagePack index file.
|
||||
func updateOwnerIndexFile(basePath string, obsoleteIDs []string) error {
|
||||
fmt.Printf(" Rewriting index file '%s'\n", basePath)
|
||||
|
||||
ownerID, err := xattr.Get(basePath, ownerIDAttrName)
|
||||
if err != nil {
|
||||
return fmt.Errorf("could not get owner ID from oldest entry '%s' to find index: %w", basePath, err)
|
||||
}
|
||||
|
||||
indexPath := filepath.Join(basePath, "../../indexes/by-user-id", string(ownerID)+".mpk")
|
||||
indexPath = filepath.Clean(indexPath)
|
||||
|
||||
// Read the MessagePack file
|
||||
fileData, err := os.ReadFile(indexPath)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return fmt.Errorf("index file does not exist, skipping update")
|
||||
}
|
||||
return fmt.Errorf("could not read index file: %w", err)
|
||||
}
|
||||
var indexMap map[string]string
|
||||
if err := msgpack.Unmarshal(fileData, &indexMap); err != nil {
|
||||
return fmt.Errorf("failed to parse MessagePack index file (is it corrupt?): %w", err)
|
||||
}
|
||||
|
||||
// Remove obsolete IDs from the map
|
||||
itemsRemoved := 0
|
||||
for _, id := range obsoleteIDs {
|
||||
if _, exists := indexMap[id]; exists {
|
||||
fmt.Printf(" - Removing obsolete ID '%s' from index.\n", id)
|
||||
delete(indexMap, id)
|
||||
itemsRemoved++
|
||||
} else {
|
||||
fmt.Printf(" - Obsolete ID '%s' not found in index\n", id)
|
||||
}
|
||||
}
|
||||
|
||||
if itemsRemoved == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Write the data back to the file
|
||||
updatedData, err := msgpack.Marshal(&indexMap)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to marshal updated index map: %w", err)
|
||||
}
|
||||
if err := os.WriteFile(indexPath, updatedData, 0644); err != nil {
|
||||
return fmt.Errorf("failed to write updated index file: %w", err)
|
||||
}
|
||||
|
||||
fmt.Printf(" ✓ Successfully removed %d item(s) and saved index file.\n", itemsRemoved)
|
||||
return nil
|
||||
}
|
||||
|
||||
func removeAttributes(path string) error {
|
||||
attrNames, err := xattr.List(path)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to list attributes for '%s': %w", path, err)
|
||||
}
|
||||
|
||||
for _, attrName := range attrNames {
|
||||
if err := xattr.Remove(path, attrName); err != nil {
|
||||
return fmt.Errorf("failed to remove attribute '%s' from '%s': %w", attrName, path, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func logFailure(message string, args ...any) {
|
||||
spinner.StopFailMessage(fmt.Sprintf("\n"+message, args...))
|
||||
spinner.StopFail()
|
||||
spinner.Start()
|
||||
fmt.Fprintf(os.Stderr, message+"\n", args...)
|
||||
}
|
||||
|
||||
// findStorageRoot walks up the directory tree starting at path until it finds a
|
||||
// directory that contains an "indexes" subdirectory which marks the root of a
|
||||
// posixfs storage. A user directory inside a space might also be named "indexes",
|
||||
// so to disambiguate we require that the "indexes" directory is an internal
|
||||
// directory: the storage's own indexes directory is skipped during assimilation
|
||||
// and therefore never receives a node ID attribute, whereas a regular user
|
||||
// directory would have one.
|
||||
func findStorageRoot(path string) (string, error) {
|
||||
current := path
|
||||
for {
|
||||
indexesPath := filepath.Join(current, "indexes")
|
||||
if info, err := os.Stat(indexesPath); err == nil && info.IsDir() {
|
||||
if id, err := xattr.Get(indexesPath, prefixes.IDAttr); err != nil || len(id) == 0 {
|
||||
return current, nil
|
||||
}
|
||||
}
|
||||
|
||||
parent := filepath.Dir(current)
|
||||
if parent == current {
|
||||
return "", fmt.Errorf("'%s' does not appear to be inside a posixfs storage (no 'indexes' directory found)", path)
|
||||
}
|
||||
current = parent
|
||||
}
|
||||
}
|
||||
|
||||
// isSpaceRoot reports whether the given path is a space root, which is
|
||||
// identified by the presence of the space ID attribute.
|
||||
func isSpaceRoot(path string) bool {
|
||||
spaceID, err := xattr.Get(path, prefixes.SpaceIDAttr)
|
||||
return err == nil && len(spaceID) > 0
|
||||
}
|
||||
@@ -0,0 +1,497 @@
|
||||
// Copyright 2026 OpenCloud GmbH <mail@opencloud.eu>
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
package command
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/opencloud-eu/opencloud/pkg/x/path/filepathx"
|
||||
storageUsersConfig "github.com/opencloud-eu/opencloud/services/storage-users/pkg/config"
|
||||
"github.com/opencloud-eu/reva/v2/pkg/storage/fs/posix/ignore"
|
||||
"github.com/opencloud-eu/reva/v2/pkg/storage/pkg/decomposedfs/metadata/prefixes"
|
||||
"github.com/opencloud-eu/reva/v2/pkg/storage/pkg/decomposedfs/node"
|
||||
"github.com/pkg/xattr"
|
||||
"github.com/shamaton/msgpack/v2"
|
||||
)
|
||||
|
||||
type consistencyChecker struct {
|
||||
cfg *storageUsersConfig.Config
|
||||
ignorer *ignore.Ignorer
|
||||
recalculateChecksums bool
|
||||
|
||||
restartRequired bool
|
||||
}
|
||||
|
||||
// checkPosixfsConsistency checks the consistency of the posixfs storage. The
|
||||
// given path determines the scope of the check: the whole storage, a single
|
||||
// space or a single entity within a space.
|
||||
func (c *consistencyChecker) Check(paths []string) error {
|
||||
for _, path := range paths {
|
||||
rootPath, err := findStorageRoot(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
path = filepath.Clean(path)
|
||||
if _, err := os.Stat(path); err != nil {
|
||||
return fmt.Errorf("error accessing '%s': %w", path, err)
|
||||
}
|
||||
contained, _ := filepathx.IsSameOrContainedBy(rootPath, path)
|
||||
|
||||
switch {
|
||||
case path == rootPath:
|
||||
fmt.Println("Checking personal spaces...")
|
||||
c.checkSpaces(filepath.Join(path, "users"))
|
||||
|
||||
fmt.Println("Checking project spaces...")
|
||||
c.checkSpaces(filepath.Join(path, "projects"))
|
||||
case isSpaceRoot(path):
|
||||
fmt.Printf("Checking space '%s'...\n", path)
|
||||
c.checkSpace(path)
|
||||
case contained:
|
||||
if c.ignorer.IsIgnored(path) {
|
||||
continue
|
||||
}
|
||||
fmt.Printf("Checking '%s'...\n", path)
|
||||
c.checkEntity(path)
|
||||
default:
|
||||
return fmt.Errorf("the provided path '%s' is neither a space root nor contained by the storage root '%s'", path, rootPath)
|
||||
}
|
||||
}
|
||||
|
||||
if c.restartRequired {
|
||||
fmt.Println("\n\n ⚠️ Please restart your openCloud instance to apply changes.")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *consistencyChecker) checkSpaces(basePath string) {
|
||||
dirEntries, err := os.ReadDir(basePath)
|
||||
if err != nil {
|
||||
logFailure("Error reading spaces directory '%s': %v", basePath, err)
|
||||
return
|
||||
}
|
||||
|
||||
for _, entry := range dirEntries {
|
||||
if entry.IsDir() {
|
||||
fullPath := filepath.Join(basePath, entry.Name())
|
||||
c.checkSpace(fullPath)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (c *consistencyChecker) checkSpace(spacePath string) {
|
||||
info, err := os.Stat(spacePath)
|
||||
if err != nil {
|
||||
logFailure("Error accessing path '%s': %v", spacePath, err)
|
||||
return
|
||||
}
|
||||
if !info.IsDir() {
|
||||
logFailure("Error: The provided path '%s' is not a directory\n", spacePath)
|
||||
return
|
||||
}
|
||||
|
||||
spaceID, err := xattr.Get(spacePath, prefixes.SpaceIDAttr)
|
||||
if err != nil || len(spaceID) == 0 {
|
||||
logFailure("Error: The directory '%s' does not seem to be a space root, it's missing the '%s' attribute\n", spacePath, prefixes.SpaceIDAttr)
|
||||
return
|
||||
}
|
||||
|
||||
c.checkSpaceID(spacePath)
|
||||
c.checkNodes(spacePath)
|
||||
}
|
||||
|
||||
func (c *consistencyChecker) checkSpaceID(spacePath string) {
|
||||
entries, uniqueIDs, oldestEntry, err := c.gatherAttributes(spacePath)
|
||||
if err != nil {
|
||||
logFailure("Failed to gather attributes: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
if len(entries) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
if len(uniqueIDs) > 1 {
|
||||
fmt.Println("\n ⚠ Multiple space IDs found:")
|
||||
for id := range uniqueIDs {
|
||||
fmt.Printf(" - %s\n", id)
|
||||
}
|
||||
|
||||
fmt.Printf("\n ⏳ Oldest entry is '%s' (modified on %s).\n",
|
||||
filepath.Base(oldestEntry.Path), oldestEntry.ModTime.Format(time.RFC1123))
|
||||
|
||||
targetID := oldestEntry.ParentID
|
||||
fmt.Printf(" ✅ Proposed target Parent ID: %s\n", targetID)
|
||||
|
||||
fmt.Printf("\n Do you want to unify all parent IDs to '%s'? This will modify %d entries, the directory, and the user index. (y/N): ", targetID, len(entries))
|
||||
|
||||
reader := bufio.NewReader(os.Stdin)
|
||||
input, _ := reader.ReadString('\n')
|
||||
input = strings.TrimSpace(strings.ToLower(input))
|
||||
|
||||
if input != "y" {
|
||||
logFailure("Operation cancelled by user.")
|
||||
return
|
||||
}
|
||||
c.restartRequired = true
|
||||
|
||||
obsoleteIDs := []string{}
|
||||
for id := range uniqueIDs {
|
||||
if id != targetID {
|
||||
obsoleteIDs = append(obsoleteIDs, id)
|
||||
}
|
||||
}
|
||||
c.fixSpaceID(spacePath, obsoleteIDs, targetID, entries)
|
||||
}
|
||||
}
|
||||
|
||||
func (c *consistencyChecker) walkNodes(dir string, parentID string) int {
|
||||
fixes := 0
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
logFailure("Error reading directory '%s': %v", dir, err)
|
||||
return 0
|
||||
}
|
||||
|
||||
for _, entry := range entries {
|
||||
fullPath := filepath.Join(dir, entry.Name())
|
||||
|
||||
if c.ignorer.IsIgnored(fullPath) {
|
||||
continue
|
||||
}
|
||||
|
||||
fixes += c.checkNodeAttributes(fullPath, entry.Name(), parentID, entry.IsDir())
|
||||
|
||||
if entry.IsDir() {
|
||||
nodeID, err := xattr.Get(fullPath, prefixes.IDAttr)
|
||||
if err != nil || len(nodeID) == 0 {
|
||||
logFailure("Directory '%s' missing '%s', skipping its children", fullPath, prefixes.IDAttr)
|
||||
continue
|
||||
}
|
||||
fixes += c.walkNodes(fullPath, string(nodeID))
|
||||
}
|
||||
}
|
||||
return fixes
|
||||
}
|
||||
|
||||
// checkNodeAttributes checks and fixes the parent ID and name attributes of a
|
||||
// single node. For files it additionally checks the blobsize and, when
|
||||
// requested, the checksums. It returns the number of fixes applied.
|
||||
func (c *consistencyChecker) checkNodeAttributes(path, name, parentID string, isDir bool) int {
|
||||
fixes := 0
|
||||
|
||||
// Check if the parent ID attribute matches the expected parent ID, if not, fix it.
|
||||
actualParentID, err := xattr.Get(path, prefixes.ParentidAttr)
|
||||
if err != nil || string(actualParentID) != parentID {
|
||||
if err := xattr.Set(path, prefixes.ParentidAttr, []byte(parentID)); err != nil {
|
||||
logFailure("Failed to fix parent ID for '%s': %v", path, err)
|
||||
} else {
|
||||
fmt.Printf(" + Fixed parent ID for '%s'\n", path)
|
||||
fixes++
|
||||
c.restartRequired = true
|
||||
}
|
||||
}
|
||||
|
||||
// Check that the name attribute matches the actual name of the file/directory, if not, fix it.
|
||||
nameAttr, err := xattr.Get(path, prefixes.NameAttr)
|
||||
if err != nil || string(nameAttr) != name {
|
||||
if err := xattr.Set(path, prefixes.NameAttr, []byte(name)); err != nil {
|
||||
logFailure("Failed to fix name attribute for '%s': %v", path, err)
|
||||
} else {
|
||||
fmt.Printf(" + Fixed name attribute for '%s'\n", path)
|
||||
fixes++
|
||||
c.restartRequired = true
|
||||
}
|
||||
}
|
||||
|
||||
if !isDir {
|
||||
fixes += c.checkBlobsize(path)
|
||||
if c.recalculateChecksums {
|
||||
fixes += c.fixChecksums(path)
|
||||
}
|
||||
}
|
||||
|
||||
return fixes
|
||||
}
|
||||
|
||||
// checkEntity checks a single file or directory within a space, including its own
|
||||
// parent ID, name and (for files) blobsize/checksums. If the entity is a directory
|
||||
// its children are checked recursively.
|
||||
func (c *consistencyChecker) checkEntity(path string) {
|
||||
info, err := os.Stat(path)
|
||||
if err != nil {
|
||||
logFailure("Error accessing path '%s': %v", path, err)
|
||||
return
|
||||
}
|
||||
|
||||
// The expected parent ID is the ID attribute of the containing directory.
|
||||
parentDir := filepath.Dir(path)
|
||||
parentID, err := xattr.Get(parentDir, prefixes.IDAttr)
|
||||
if err != nil || len(parentID) == 0 {
|
||||
logFailure("Parent directory '%s' is missing the '%s' attribute", parentDir, prefixes.IDAttr)
|
||||
return
|
||||
}
|
||||
|
||||
fixes := c.checkNodeAttributes(path, info.Name(), string(parentID), info.IsDir())
|
||||
|
||||
if info.IsDir() {
|
||||
nodeID, err := xattr.Get(path, prefixes.IDAttr)
|
||||
if err != nil || len(nodeID) == 0 {
|
||||
logFailure("Directory '%s' missing '%s' attribute", path, prefixes.IDAttr)
|
||||
} else {
|
||||
fixes += c.walkNodes(path, string(nodeID))
|
||||
}
|
||||
}
|
||||
|
||||
if fixes > 0 {
|
||||
fmt.Printf(" ✓ Fixed %d incorrect node attributes for %s\n", fixes, filepath.Base(path))
|
||||
}
|
||||
}
|
||||
|
||||
// checkBlobsize verifies that the stored blobsize attribute matches the actual
|
||||
// file size and fixes it if it doesn't. It returns the number of fixes applied.
|
||||
func (c *consistencyChecker) checkBlobsize(path string) int {
|
||||
info, err := os.Stat(path)
|
||||
if err != nil {
|
||||
logFailure("Error accessing file '%s': %v", path, err)
|
||||
return 0
|
||||
}
|
||||
|
||||
expectedSize := strconv.FormatInt(info.Size(), 10)
|
||||
blobsize, err := xattr.Get(path, prefixes.BlobsizeAttr)
|
||||
if err == nil && string(blobsize) == expectedSize {
|
||||
return 0
|
||||
}
|
||||
|
||||
if err := xattr.Set(path, prefixes.BlobsizeAttr, []byte(expectedSize)); err != nil {
|
||||
logFailure("Failed to fix blobsize for '%s': %v", path, err)
|
||||
return 0
|
||||
}
|
||||
|
||||
fmt.Printf(" + Fixed blobsize for '%s'\n", path)
|
||||
c.restartRequired = true
|
||||
return 1
|
||||
}
|
||||
|
||||
// fixChecksums recalculates the sha1, md5 and adler32 checksums of the file and
|
||||
// updates the stored attributes if they differ. It returns the number of fixes applied.
|
||||
func (c *consistencyChecker) fixChecksums(path string) int {
|
||||
sha1h, md5h, adler32h, err := node.CalculateChecksums(context.Background(), path)
|
||||
if err != nil {
|
||||
logFailure("Failed to calculate checksums for '%s': %v", path, err)
|
||||
return 0
|
||||
}
|
||||
|
||||
checksums := map[string][]byte{
|
||||
prefixes.ChecksumPrefix + "sha1": sha1h.Sum(nil),
|
||||
prefixes.ChecksumPrefix + "md5": md5h.Sum(nil),
|
||||
prefixes.ChecksumPrefix + "adler32": adler32h.Sum(nil),
|
||||
}
|
||||
|
||||
fixes := 0
|
||||
for attrName, sum := range checksums {
|
||||
current, err := xattr.Get(path, attrName)
|
||||
if err == nil && bytes.Equal(current, sum) {
|
||||
continue
|
||||
}
|
||||
if err := xattr.Set(path, attrName, sum); err != nil {
|
||||
logFailure("Failed to fix checksum '%s' for '%s': %v", attrName, path, err)
|
||||
continue
|
||||
}
|
||||
fmt.Printf(" + Fixed checksum '%s' for '%s'\n", attrName, path)
|
||||
c.restartRequired = true
|
||||
fixes++
|
||||
}
|
||||
return fixes
|
||||
}
|
||||
|
||||
func (c *consistencyChecker) checkNodes(spacePath string) {
|
||||
rootID, err := xattr.Get(spacePath, prefixes.IDAttr)
|
||||
if err != nil || len(rootID) == 0 {
|
||||
logFailure("Space root '%s' missing '%s' attribute", spacePath, prefixes.IDAttr)
|
||||
return
|
||||
}
|
||||
|
||||
fixes := c.walkNodes(spacePath, string(rootID))
|
||||
|
||||
if fixes > 0 {
|
||||
fmt.Printf(" ✓ Fixed %d incorrect node attributes in %s\n", fixes, filepath.Base(spacePath))
|
||||
}
|
||||
}
|
||||
|
||||
// fixSpaceID updates the parentid attributes of all entries in a space to a new target ID,
|
||||
// updates the space's own ID attributes, and removes obsolete IDs from the user index file.
|
||||
func (c *consistencyChecker) fixSpaceID(spacePath string, obsoleteIDs []string, targetID string, entries []EntryInfo) {
|
||||
// Set all parentid attributes to the proper space ID
|
||||
err := setAllParentIDAttributes(entries, targetID)
|
||||
if err != nil {
|
||||
logFailure("an error occurred during file attribute update: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
// Update space ID itself
|
||||
fmt.Printf(" Updating directory '%s' with attribute '%s' -> %s\n", filepath.Base(spacePath), prefixes.IDAttr, targetID)
|
||||
err = xattr.Set(spacePath, prefixes.IDAttr, []byte(targetID))
|
||||
if err != nil {
|
||||
logFailure("Failed to set attribute on directory '%s': %v", spacePath, err)
|
||||
return
|
||||
}
|
||||
err = xattr.Set(spacePath, prefixes.SpaceIDAttr, []byte(targetID))
|
||||
if err != nil {
|
||||
logFailure("Failed to set attribute on directory '%s': %v", spacePath, err)
|
||||
return
|
||||
}
|
||||
|
||||
// update the index
|
||||
err = c.updateOwnerIndexFile(spacePath, obsoleteIDs)
|
||||
if err != nil {
|
||||
logFailure("Could not update the owner index file: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func (c *consistencyChecker) gatherAttributes(path string) ([]EntryInfo, map[string]struct{}, EntryInfo, error) {
|
||||
dirEntries, err := os.ReadDir(path)
|
||||
if err != nil {
|
||||
return nil, nil, EntryInfo{}, fmt.Errorf("failed to read directory: %w", err)
|
||||
}
|
||||
|
||||
var allEntries []EntryInfo
|
||||
uniqueIDs := make(map[string]struct{})
|
||||
var oldestEntry EntryInfo
|
||||
oldestTime := time.Now().Add(100 * 365 * 24 * time.Hour) // Set to a future date to find the oldest entry
|
||||
|
||||
for _, entry := range dirEntries {
|
||||
fullPath := filepath.Join(path, entry.Name())
|
||||
if c.ignorer.IsIgnored(fullPath) {
|
||||
continue
|
||||
}
|
||||
info, err := os.Stat(fullPath)
|
||||
if err != nil {
|
||||
fmt.Printf(" - Warning: could not stat %s: %v\n", entry.Name(), err)
|
||||
continue
|
||||
}
|
||||
|
||||
parentID, err := xattr.Get(fullPath, prefixes.ParentidAttr)
|
||||
if err != nil {
|
||||
continue // Skip if attribute doesn't exist or can't be read
|
||||
}
|
||||
|
||||
entryInfo := EntryInfo{
|
||||
Path: fullPath,
|
||||
ModTime: info.ModTime(),
|
||||
ParentID: string(parentID),
|
||||
}
|
||||
|
||||
allEntries = append(allEntries, entryInfo)
|
||||
uniqueIDs[string(parentID)] = struct{}{}
|
||||
|
||||
if entryInfo.ModTime.Before(oldestTime) {
|
||||
oldestTime = entryInfo.ModTime
|
||||
oldestEntry = entryInfo
|
||||
}
|
||||
}
|
||||
|
||||
return allEntries, uniqueIDs, oldestEntry, nil
|
||||
}
|
||||
|
||||
func setAllParentIDAttributes(entries []EntryInfo, targetID string) error {
|
||||
fmt.Printf(" Setting all parent IDs to '%s':\n", targetID)
|
||||
|
||||
for _, entry := range entries {
|
||||
if entry.ParentID == targetID {
|
||||
fmt.Printf(" - Skipping '%s' (already has target ID).\n", filepath.Base(entry.Path))
|
||||
continue
|
||||
}
|
||||
|
||||
fmt.Printf(" - Removing all attributes from '%s'. It will be re-assimilated\n", filepath.Base(entry.Path))
|
||||
filepath.WalkDir(entry.Path, func(path string, d os.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return fmt.Errorf("error walking path '%s': %w", path, err)
|
||||
}
|
||||
|
||||
// Remove all attributes from the file.
|
||||
if err := removeAttributes(path); err != nil {
|
||||
fmt.Printf("failed to remove attributes from '%s': %v", path, err)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// updateOwnerIndexFile handles the logic of reading, modifying, and writing the MessagePack index file.
|
||||
func (c *consistencyChecker) updateOwnerIndexFile(basePath string, obsoleteIDs []string) error {
|
||||
fmt.Printf(" Rewriting index file '%s'\n", basePath)
|
||||
|
||||
ownerID, err := xattr.Get(basePath, prefixes.OwnerIDAttr)
|
||||
if err != nil {
|
||||
return fmt.Errorf("could not get owner ID from oldest entry '%s' to find index: %w", basePath, err)
|
||||
}
|
||||
|
||||
indexPath := filepath.Join(c.cfg.Drivers.Posix.Root, "indexes", "by-user-id", string(ownerID)+".mpk")
|
||||
indexPath = filepath.Clean(indexPath)
|
||||
|
||||
// Read the MessagePack file
|
||||
fileData, err := os.ReadFile(indexPath)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return fmt.Errorf("index file does not exist, skipping update")
|
||||
}
|
||||
return fmt.Errorf("could not read index file: %w", err)
|
||||
}
|
||||
var indexMap map[string]string
|
||||
if err := msgpack.Unmarshal(fileData, &indexMap); err != nil {
|
||||
return fmt.Errorf("failed to parse MessagePack index file (is it corrupt?): %w", err)
|
||||
}
|
||||
|
||||
// Remove obsolete IDs from the map
|
||||
itemsRemoved := 0
|
||||
for _, id := range obsoleteIDs {
|
||||
if _, exists := indexMap[id]; exists {
|
||||
fmt.Printf(" - Removing obsolete ID '%s' from index.\n", id)
|
||||
delete(indexMap, id)
|
||||
itemsRemoved++
|
||||
} else {
|
||||
fmt.Printf(" - Obsolete ID '%s' not found in index\n", id)
|
||||
}
|
||||
}
|
||||
|
||||
if itemsRemoved == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Write the data back to the file
|
||||
updatedData, err := msgpack.Marshal(&indexMap)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to marshal updated index map: %w", err)
|
||||
}
|
||||
if err := os.WriteFile(indexPath, updatedData, 0644); err != nil {
|
||||
return fmt.Errorf("failed to write updated index file: %w", err)
|
||||
}
|
||||
|
||||
fmt.Printf(" ✓ Successfully removed %d item(s) and saved index file.\n", itemsRemoved)
|
||||
return nil
|
||||
}
|
||||
|
||||
func removeAttributes(path string) error {
|
||||
attrNames, err := xattr.List(path)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to list attributes for '%s': %w", path, err)
|
||||
}
|
||||
|
||||
for _, attrName := range attrNames {
|
||||
if err := xattr.Remove(path, attrName); err != nil {
|
||||
return fmt.Errorf("failed to remove attribute '%s' from '%s': %w", attrName, path, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -6,11 +6,13 @@ import (
|
||||
"os/signal"
|
||||
"syscall"
|
||||
|
||||
"github.com/rs/zerolog"
|
||||
"github.com/spf13/cobra"
|
||||
|
||||
"github.com/opencloud-eu/opencloud/opencloud/pkg/register"
|
||||
"github.com/opencloud-eu/opencloud/pkg/clihelper"
|
||||
"github.com/opencloud-eu/opencloud/pkg/config"
|
||||
|
||||
"github.com/spf13/cobra"
|
||||
oclog "github.com/opencloud-eu/opencloud/pkg/log"
|
||||
)
|
||||
|
||||
// Execute is the entry point for the opencloud command.
|
||||
@@ -38,3 +40,11 @@ func Execute() error {
|
||||
ctx, _ := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM, syscall.SIGQUIT, syscall.SIGHUP)
|
||||
return app.ExecuteContext(ctx)
|
||||
}
|
||||
|
||||
func logger(name string) zerolog.Logger {
|
||||
return oclog.NewLogger(
|
||||
oclog.Name(name),
|
||||
oclog.Level("info"),
|
||||
oclog.Pretty(true),
|
||||
oclog.Color(true)).Logger
|
||||
}
|
||||
@@ -9,7 +9,6 @@ import (
|
||||
"github.com/opencloud-eu/opencloud/pkg/config"
|
||||
"github.com/opencloud-eu/opencloud/pkg/config/configlog"
|
||||
"github.com/opencloud-eu/opencloud/pkg/config/parser"
|
||||
oclog "github.com/opencloud-eu/opencloud/pkg/log"
|
||||
mregistry "github.com/opencloud-eu/opencloud/pkg/registry"
|
||||
sharing "github.com/opencloud-eu/opencloud/services/sharing/pkg/config"
|
||||
sharingparser "github.com/opencloud-eu/opencloud/services/sharing/pkg/config/parser"
|
||||
@@ -85,7 +84,7 @@ func cleanup(_ *cobra.Command, cfg *config.Config) error {
|
||||
return configlog.ReturnError(errors.New("cleanup is only implemented for the jsoncs3 share manager"))
|
||||
}
|
||||
|
||||
l := logger()
|
||||
l := logger("migrate")
|
||||
|
||||
zerolog.SetGlobalLevel(zerolog.InfoLevel)
|
||||
|
||||
@@ -94,7 +93,7 @@ func cleanup(_ *cobra.Command, cfg *config.Config) error {
|
||||
if !ok {
|
||||
return configlog.ReturnError(errors.New("Unknown share manager type '" + driver + "'"))
|
||||
}
|
||||
mgr, err := f(rcfg[driver].(map[string]any), l)
|
||||
mgr, err := f(rcfg[driver].(map[string]any), &l)
|
||||
if err != nil {
|
||||
return configlog.ReturnError(err)
|
||||
}
|
||||
@@ -167,12 +166,3 @@ func revaShareConfig(cfg *sharing.Config) map[string]any {
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func logger() *zerolog.Logger {
|
||||
log := oclog.NewLogger(
|
||||
oclog.Name("migrate"),
|
||||
oclog.Level("info"),
|
||||
oclog.Pretty(true),
|
||||
oclog.Color(true)).Logger
|
||||
return &log
|
||||
}
|
||||
@@ -389,9 +389,16 @@ func Start(ctx context.Context, o ...Option) error {
|
||||
if ev.Restarting {
|
||||
l = s.Log.Error()
|
||||
}
|
||||
l.Str("event", e.String()).Str("service", ev.ServiceName).Str("supervisor", ev.SupervisorName).
|
||||
Bool("restarting", ev.Restarting).Float64("failures", ev.CurrentFailures).Float64("threshold", ev.FailureThreshold).
|
||||
Interface("error", ev.Err).Msg("service terminated")
|
||||
l = l.Str("event", e.String()).Str("service", ev.ServiceName).Str("supervisor", ev.SupervisorName).
|
||||
Bool("restarting", ev.Restarting).Float64("failures", ev.CurrentFailures).Float64("threshold", ev.FailureThreshold)
|
||||
// ev.Err is an interface{}: marshaling an error yields {} because
|
||||
// its fields are unexported, so the message has to go through Err
|
||||
if err, ok := ev.Err.(error); ok {
|
||||
l = l.Err(err)
|
||||
} else {
|
||||
l = l.Interface("error", ev.Err)
|
||||
}
|
||||
l.Msg("service terminated")
|
||||
case suture.EventBackoff:
|
||||
s.Log.Warn().Str("event", e.String()).Str("supervisor", ev.SupervisorName).Msg("service backoff")
|
||||
case suture.EventResume:
|
||||
|
||||
@@ -14,5 +14,17 @@ func DefaultApp(app *cobra.Command) *cobra.Command {
|
||||
// version info
|
||||
app.Version = fmt.Sprintf("%s (%s <%s>) (%s)", version.String, "OpenCloud GmbH", "support@opencloud.eu", version.Compiled())
|
||||
|
||||
// cobra would print the error on top of what main() already prints
|
||||
app.SilenceErrors = true
|
||||
|
||||
// keep the usage block for flag parse errors, drop it once RunE runs.
|
||||
// Traversing runs the hook even below a subcommand that brings its own,
|
||||
// e.g. every service below ServiceCommand.
|
||||
cobra.EnableTraverseRunHooks = true
|
||||
app.PersistentPreRunE = func(cmd *cobra.Command, _ []string) error {
|
||||
cmd.SilenceUsage = true
|
||||
return nil
|
||||
}
|
||||
|
||||
return app
|
||||
}
|
||||
@@ -34,7 +34,7 @@ var (
|
||||
// LatestTag is the latest released version plus the dev meta version.
|
||||
// Will be overwritten by the release pipeline
|
||||
// Needs a manual change for every tagged release
|
||||
LatestTag = "7.3.0+dev"
|
||||
LatestTag = "7.4.0+dev"
|
||||
|
||||
// Date indicates the build date.
|
||||
// This has been removed, it looks like you can only replace static strings with recent go versions
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
package filepathx
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// JailJoin joins any number of path elements into a single path,
|
||||
@@ -10,3 +12,27 @@ import (
|
||||
func JailJoin(jail string, elem ...string) string {
|
||||
return filepath.Join(jail, filepath.Join(append([]string{"/"}, elem...)...))
|
||||
}
|
||||
|
||||
// Determines whether the file or directory 'child' is same as or underneath the directory 'parent'.
|
||||
//
|
||||
// Note that 'parent' is expected to be a directory.
|
||||
func IsSameOrContainedBy(parent string, child string) (bool, error) {
|
||||
absParent, err := filepath.Abs(parent)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("failed to make parent directory absolute: %q: %w", parent, err)
|
||||
}
|
||||
|
||||
absChild, err := filepath.Abs(child)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("failed to make child file/directory absolute: %q: %w", child, err)
|
||||
}
|
||||
|
||||
rel, err := filepath.Rel(absParent, absChild)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("failed to determine the relative path between the parent directory %q and the child file/directory: %q: %w", absParent, absChild, err)
|
||||
}
|
||||
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||
return false, nil
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
@@ -1,9 +1,12 @@
|
||||
package filepathx_test
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/opencloud-eu/opencloud/pkg/x/path/filepathx"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestJailJoin(t *testing.T) {
|
||||
@@ -61,3 +64,23 @@ func TestJailJoin(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsSameOrContainedBy(t *testing.T) {
|
||||
for _, tt := range []struct {
|
||||
parent string
|
||||
child string
|
||||
expected bool
|
||||
}{
|
||||
{"foo", "foo", true},
|
||||
{"/foo", "/foo", true},
|
||||
{"foo", "foo/bar", true},
|
||||
{"foo", "bar", false},
|
||||
} {
|
||||
t.Run(fmt.Sprintf("%s: %s vs %s", t.Name(), strings.ReplaceAll(tt.parent, "/", "."), strings.ReplaceAll(tt.child, "/", ".")), func(t *testing.T) {
|
||||
require := require.New(t)
|
||||
b, err := filepathx.IsSameOrContainedBy(tt.parent, tt.child)
|
||||
require.NoError(err)
|
||||
require.Equal(tt.expected, b)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -627,13 +627,13 @@ type Bundle struct {
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"` // @gotags: yaml:"id"
|
||||
Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"` // @gotags: yaml:"name"
|
||||
Type Bundle_Type `protobuf:"varint,3,opt,name=type,proto3,enum=opencloud.messages.settings.v0.Bundle_Type" json:"type,omitempty"` // @gotags: yaml:"type"
|
||||
Extension string `protobuf:"bytes,4,opt,name=extension,proto3" json:"extension,omitempty"` // @gotags: yaml:"extension"
|
||||
DisplayName string `protobuf:"bytes,5,opt,name=display_name,json=displayName,proto3" json:"display_name,omitempty"` // @gotags: yaml:"display_name"
|
||||
Settings []*Setting `protobuf:"bytes,6,rep,name=settings,proto3" json:"settings,omitempty"` // @gotags: yaml:"settings"
|
||||
Resource *Resource `protobuf:"bytes,7,opt,name=resource,proto3" json:"resource,omitempty"` // @gotags: yaml:"resource"
|
||||
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty" yaml:"id"` // @gotags: yaml:"id"
|
||||
Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty" yaml:"name"` // @gotags: yaml:"name"
|
||||
Type Bundle_Type `protobuf:"varint,3,opt,name=type,proto3,enum=opencloud.messages.settings.v0.Bundle_Type" json:"type,omitempty" yaml:"type"` // @gotags: yaml:"type"
|
||||
Extension string `protobuf:"bytes,4,opt,name=extension,proto3" json:"extension,omitempty" yaml:"extension"` // @gotags: yaml:"extension"
|
||||
DisplayName string `protobuf:"bytes,5,opt,name=display_name,json=displayName,proto3" json:"display_name,omitempty" yaml:"display_name"` // @gotags: yaml:"display_name"
|
||||
Settings []*Setting `protobuf:"bytes,6,rep,name=settings,proto3" json:"settings,omitempty" yaml:"settings"` // @gotags: yaml:"settings"
|
||||
Resource *Resource `protobuf:"bytes,7,opt,name=resource,proto3" json:"resource,omitempty" yaml:"resource"` // @gotags: yaml:"resource"
|
||||
}
|
||||
|
||||
func (x *Bundle) Reset() {
|
||||
@@ -722,10 +722,10 @@ type Setting struct {
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"` // @gotags: yaml:"id"
|
||||
Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"` // @gotags: yaml:"name"
|
||||
DisplayName string `protobuf:"bytes,3,opt,name=display_name,json=displayName,proto3" json:"display_name,omitempty"` // @gotags: yaml:"display_name"
|
||||
Description string `protobuf:"bytes,4,opt,name=description,proto3" json:"description,omitempty"` // @gotags: yaml:"description"
|
||||
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty" yaml:"id"` // @gotags: yaml:"id"
|
||||
Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty" yaml:"name"` // @gotags: yaml:"name"
|
||||
DisplayName string `protobuf:"bytes,3,opt,name=display_name,json=displayName,proto3" json:"display_name,omitempty" yaml:"display_name"` // @gotags: yaml:"display_name"
|
||||
Description string `protobuf:"bytes,4,opt,name=description,proto3" json:"description,omitempty" yaml:"description"` // @gotags: yaml:"description"
|
||||
// Types that are assignable to Value:
|
||||
//
|
||||
// *Setting_IntValue
|
||||
@@ -736,7 +736,7 @@ type Setting struct {
|
||||
// *Setting_PermissionValue
|
||||
// *Setting_MultiChoiceCollectionValue
|
||||
Value isSetting_Value `protobuf_oneof:"value"`
|
||||
Resource *Resource `protobuf:"bytes,11,opt,name=resource,proto3" json:"resource,omitempty"` // @gotags: yaml:"resource"
|
||||
Resource *Resource `protobuf:"bytes,11,opt,name=resource,proto3" json:"resource,omitempty" yaml:"resource"` // @gotags: yaml:"resource"
|
||||
}
|
||||
|
||||
func (x *Setting) Reset() {
|
||||
@@ -867,31 +867,31 @@ type isSetting_Value interface {
|
||||
}
|
||||
|
||||
type Setting_IntValue struct {
|
||||
IntValue *Int `protobuf:"bytes,5,opt,name=int_value,json=intValue,proto3,oneof"` // @gotags: yaml:"int_value"
|
||||
IntValue *Int `protobuf:"bytes,5,opt,name=int_value,json=intValue,proto3,oneof" yaml:"int_value"` // @gotags: yaml:"int_value"
|
||||
}
|
||||
|
||||
type Setting_StringValue struct {
|
||||
StringValue *String `protobuf:"bytes,6,opt,name=string_value,json=stringValue,proto3,oneof"` // @gotags: yaml:"string_value"
|
||||
StringValue *String `protobuf:"bytes,6,opt,name=string_value,json=stringValue,proto3,oneof" yaml:"string_value"` // @gotags: yaml:"string_value"
|
||||
}
|
||||
|
||||
type Setting_BoolValue struct {
|
||||
BoolValue *Bool `protobuf:"bytes,7,opt,name=bool_value,json=boolValue,proto3,oneof"` // @gotags: yaml:"bool_value"
|
||||
BoolValue *Bool `protobuf:"bytes,7,opt,name=bool_value,json=boolValue,proto3,oneof" yaml:"bool_value"` // @gotags: yaml:"bool_value"
|
||||
}
|
||||
|
||||
type Setting_SingleChoiceValue struct {
|
||||
SingleChoiceValue *SingleChoiceList `protobuf:"bytes,8,opt,name=single_choice_value,json=singleChoiceValue,proto3,oneof"` // @gotags: yaml:"single_choice_value"
|
||||
SingleChoiceValue *SingleChoiceList `protobuf:"bytes,8,opt,name=single_choice_value,json=singleChoiceValue,proto3,oneof" yaml:"single_choice_value"` // @gotags: yaml:"single_choice_value"
|
||||
}
|
||||
|
||||
type Setting_MultiChoiceValue struct {
|
||||
MultiChoiceValue *MultiChoiceList `protobuf:"bytes,9,opt,name=multi_choice_value,json=multiChoiceValue,proto3,oneof"` // @gotags: yaml:"multi_choice_value"
|
||||
MultiChoiceValue *MultiChoiceList `protobuf:"bytes,9,opt,name=multi_choice_value,json=multiChoiceValue,proto3,oneof" yaml:"multi_choice_value"` // @gotags: yaml:"multi_choice_value"
|
||||
}
|
||||
|
||||
type Setting_PermissionValue struct {
|
||||
PermissionValue *Permission `protobuf:"bytes,10,opt,name=permission_value,json=permissionValue,proto3,oneof"` // @gotags: yaml:"permission_value"
|
||||
PermissionValue *Permission `protobuf:"bytes,10,opt,name=permission_value,json=permissionValue,proto3,oneof" yaml:"permission_value"` // @gotags: yaml:"permission_value"
|
||||
}
|
||||
|
||||
type Setting_MultiChoiceCollectionValue struct {
|
||||
MultiChoiceCollectionValue *MultiChoiceCollection `protobuf:"bytes,12,opt,name=multi_choice_collection_value,json=multiChoiceCollectionValue,proto3,oneof"` // @gotags: yaml:"multi_choice_collection_value"
|
||||
MultiChoiceCollectionValue *MultiChoiceCollection `protobuf:"bytes,12,opt,name=multi_choice_collection_value,json=multiChoiceCollectionValue,proto3,oneof" yaml:"multi_choice_collection_value"` // @gotags: yaml:"multi_choice_collection_value"
|
||||
}
|
||||
|
||||
func (*Setting_IntValue) isSetting_Value() {}
|
||||
@@ -913,11 +913,11 @@ type Int struct {
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Default int64 `protobuf:"varint,1,opt,name=default,proto3" json:"default,omitempty"` // @gotags: yaml:"default"
|
||||
Min int64 `protobuf:"varint,2,opt,name=min,proto3" json:"min,omitempty"` // @gotags: yaml:"min"
|
||||
Max int64 `protobuf:"varint,3,opt,name=max,proto3" json:"max,omitempty"` // @gotags: yaml:"max"
|
||||
Step int64 `protobuf:"varint,4,opt,name=step,proto3" json:"step,omitempty"` // @gotags: yaml:"step"
|
||||
Placeholder string `protobuf:"bytes,5,opt,name=placeholder,proto3" json:"placeholder,omitempty"` // @gotags: yaml:"placeholder"
|
||||
Default int64 `protobuf:"varint,1,opt,name=default,proto3" json:"default,omitempty" yaml:"default"` // @gotags: yaml:"default"
|
||||
Min int64 `protobuf:"varint,2,opt,name=min,proto3" json:"min,omitempty" yaml:"min"` // @gotags: yaml:"min"
|
||||
Max int64 `protobuf:"varint,3,opt,name=max,proto3" json:"max,omitempty" yaml:"max"` // @gotags: yaml:"max"
|
||||
Step int64 `protobuf:"varint,4,opt,name=step,proto3" json:"step,omitempty" yaml:"step"` // @gotags: yaml:"step"
|
||||
Placeholder string `protobuf:"bytes,5,opt,name=placeholder,proto3" json:"placeholder,omitempty" yaml:"placeholder"` // @gotags: yaml:"placeholder"
|
||||
}
|
||||
|
||||
func (x *Int) Reset() {
|
||||
@@ -992,11 +992,11 @@ type String struct {
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Default string `protobuf:"bytes,1,opt,name=default,proto3" json:"default,omitempty"` // @gotags: yaml:"default"
|
||||
Required bool `protobuf:"varint,2,opt,name=required,proto3" json:"required,omitempty"` // @gotags: yaml:"required"
|
||||
MinLength int32 `protobuf:"varint,3,opt,name=min_length,json=minLength,proto3" json:"min_length,omitempty"` // @gotags: yaml:"min_length"
|
||||
MaxLength int32 `protobuf:"varint,4,opt,name=max_length,json=maxLength,proto3" json:"max_length,omitempty"` // @gotags: yaml:"max_length"
|
||||
Placeholder string `protobuf:"bytes,5,opt,name=placeholder,proto3" json:"placeholder,omitempty"` // @gotags: yaml:"placeholder"
|
||||
Default string `protobuf:"bytes,1,opt,name=default,proto3" json:"default,omitempty" yaml:"default"` // @gotags: yaml:"default"
|
||||
Required bool `protobuf:"varint,2,opt,name=required,proto3" json:"required,omitempty" yaml:"required"` // @gotags: yaml:"required"
|
||||
MinLength int32 `protobuf:"varint,3,opt,name=min_length,json=minLength,proto3" json:"min_length,omitempty" yaml:"min_length"` // @gotags: yaml:"min_length"
|
||||
MaxLength int32 `protobuf:"varint,4,opt,name=max_length,json=maxLength,proto3" json:"max_length,omitempty" yaml:"max_length"` // @gotags: yaml:"max_length"
|
||||
Placeholder string `protobuf:"bytes,5,opt,name=placeholder,proto3" json:"placeholder,omitempty" yaml:"placeholder"` // @gotags: yaml:"placeholder"
|
||||
}
|
||||
|
||||
func (x *String) Reset() {
|
||||
@@ -1071,8 +1071,8 @@ type Bool struct {
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Default bool `protobuf:"varint,1,opt,name=default,proto3" json:"default,omitempty"` // @gotags: yaml:"default"
|
||||
Label string `protobuf:"bytes,2,opt,name=label,proto3" json:"label,omitempty"` // @gotags: yaml:"label"
|
||||
Default bool `protobuf:"varint,1,opt,name=default,proto3" json:"default,omitempty" yaml:"default"` // @gotags: yaml:"default"
|
||||
Label string `protobuf:"bytes,2,opt,name=label,proto3" json:"label,omitempty" yaml:"label"` // @gotags: yaml:"label"
|
||||
}
|
||||
|
||||
func (x *Bool) Reset() {
|
||||
@@ -1126,7 +1126,7 @@ type SingleChoiceList struct {
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Options []*ListOption `protobuf:"bytes,1,rep,name=options,proto3" json:"options,omitempty"` // @gotags: yaml:"options"
|
||||
Options []*ListOption `protobuf:"bytes,1,rep,name=options,proto3" json:"options,omitempty" yaml:"options"` // @gotags: yaml:"options"
|
||||
}
|
||||
|
||||
func (x *SingleChoiceList) Reset() {
|
||||
@@ -1173,7 +1173,7 @@ type MultiChoiceList struct {
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Options []*ListOption `protobuf:"bytes,1,rep,name=options,proto3" json:"options,omitempty"` // @gotags: yaml:"options"
|
||||
Options []*ListOption `protobuf:"bytes,1,rep,name=options,proto3" json:"options,omitempty" yaml:"options"` // @gotags: yaml:"options"
|
||||
}
|
||||
|
||||
func (x *MultiChoiceList) Reset() {
|
||||
@@ -1220,9 +1220,9 @@ type ListOption struct {
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Value *ListOptionValue `protobuf:"bytes,1,opt,name=value,proto3" json:"value,omitempty"` // @gotags: yaml:"value"
|
||||
Default bool `protobuf:"varint,2,opt,name=default,proto3" json:"default,omitempty"` // @gotags: yaml:"default"
|
||||
DisplayValue string `protobuf:"bytes,3,opt,name=display_value,json=displayValue,proto3" json:"display_value,omitempty"` // @gotags: yaml:"display_value"
|
||||
Value *ListOptionValue `protobuf:"bytes,1,opt,name=value,proto3" json:"value,omitempty" yaml:"value"` // @gotags: yaml:"value"
|
||||
Default bool `protobuf:"varint,2,opt,name=default,proto3" json:"default,omitempty" yaml:"default"` // @gotags: yaml:"default"
|
||||
DisplayValue string `protobuf:"bytes,3,opt,name=display_value,json=displayValue,proto3" json:"display_value,omitempty" yaml:"display_value"` // @gotags: yaml:"display_value"
|
||||
}
|
||||
|
||||
func (x *ListOption) Reset() {
|
||||
@@ -1283,7 +1283,7 @@ type MultiChoiceCollection struct {
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Options []*MultiChoiceCollectionOption `protobuf:"bytes,1,rep,name=options,proto3" json:"options,omitempty"` // @gotags: yaml:"options"
|
||||
Options []*MultiChoiceCollectionOption `protobuf:"bytes,1,rep,name=options,proto3" json:"options,omitempty" yaml:"options"` // @gotags: yaml:"options"
|
||||
}
|
||||
|
||||
func (x *MultiChoiceCollection) Reset() {
|
||||
@@ -1330,10 +1330,10 @@ type MultiChoiceCollectionOption struct {
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Value *MultiChoiceCollectionOptionValue `protobuf:"bytes,1,opt,name=value,proto3" json:"value,omitempty"` // @gotags: yaml:"value"
|
||||
Key string `protobuf:"bytes,2,opt,name=key,proto3" json:"key,omitempty"` // @gotags: yaml:"key"
|
||||
Attribute string `protobuf:"bytes,3,opt,name=attribute,proto3" json:"attribute,omitempty"` // @gotags: yaml:"attribute"
|
||||
DisplayValue string `protobuf:"bytes,4,opt,name=display_value,json=displayValue,proto3" json:"display_value,omitempty"` // @gotags: yaml:"display_value"
|
||||
Value *MultiChoiceCollectionOptionValue `protobuf:"bytes,1,opt,name=value,proto3" json:"value,omitempty" yaml:"value"` // @gotags: yaml:"value"
|
||||
Key string `protobuf:"bytes,2,opt,name=key,proto3" json:"key,omitempty" yaml:"key"` // @gotags: yaml:"key"
|
||||
Attribute string `protobuf:"bytes,3,opt,name=attribute,proto3" json:"attribute,omitempty" yaml:"attribute"` // @gotags: yaml:"attribute"
|
||||
DisplayValue string `protobuf:"bytes,4,opt,name=display_value,json=displayValue,proto3" json:"display_value,omitempty" yaml:"display_value"` // @gotags: yaml:"display_value"
|
||||
}
|
||||
|
||||
func (x *MultiChoiceCollectionOption) Reset() {
|
||||
@@ -1474,15 +1474,15 @@ type isMultiChoiceCollectionOptionValue_Option interface {
|
||||
}
|
||||
|
||||
type MultiChoiceCollectionOptionValue_IntValue struct {
|
||||
IntValue *Int `protobuf:"bytes,1,opt,name=int_value,json=intValue,proto3,oneof"` // @gotags: yaml:"int_value"
|
||||
IntValue *Int `protobuf:"bytes,1,opt,name=int_value,json=intValue,proto3,oneof" yaml:"int_value"` // @gotags: yaml:"int_value"
|
||||
}
|
||||
|
||||
type MultiChoiceCollectionOptionValue_StringValue struct {
|
||||
StringValue *String `protobuf:"bytes,2,opt,name=string_value,json=stringValue,proto3,oneof"` // @gotags: yaml:"string_value"
|
||||
StringValue *String `protobuf:"bytes,2,opt,name=string_value,json=stringValue,proto3,oneof" yaml:"string_value"` // @gotags: yaml:"string_value"
|
||||
}
|
||||
|
||||
type MultiChoiceCollectionOptionValue_BoolValue struct {
|
||||
BoolValue *Bool `protobuf:"bytes,3,opt,name=bool_value,json=boolValue,proto3,oneof"` // @gotags: yaml:"bool_value"
|
||||
BoolValue *Bool `protobuf:"bytes,3,opt,name=bool_value,json=boolValue,proto3,oneof" yaml:"bool_value"` // @gotags: yaml:"bool_value"
|
||||
}
|
||||
|
||||
func (*MultiChoiceCollectionOptionValue_IntValue) isMultiChoiceCollectionOptionValue_Option() {}
|
||||
@@ -1496,8 +1496,8 @@ type Permission struct {
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Operation Permission_Operation `protobuf:"varint,1,opt,name=operation,proto3,enum=opencloud.messages.settings.v0.Permission_Operation" json:"operation,omitempty"` // @gotags: yaml:"operation"
|
||||
Constraint Permission_Constraint `protobuf:"varint,2,opt,name=constraint,proto3,enum=opencloud.messages.settings.v0.Permission_Constraint" json:"constraint,omitempty"` // @gotags: yaml:"constraint"
|
||||
Operation Permission_Operation `protobuf:"varint,1,opt,name=operation,proto3,enum=opencloud.messages.settings.v0.Permission_Operation" json:"operation,omitempty" yaml:"operation"` // @gotags: yaml:"operation"
|
||||
Constraint Permission_Constraint `protobuf:"varint,2,opt,name=constraint,proto3,enum=opencloud.messages.settings.v0.Permission_Constraint" json:"constraint,omitempty" yaml:"constraint"` // @gotags: yaml:"constraint"
|
||||
}
|
||||
|
||||
func (x *Permission) Reset() {
|
||||
@@ -1552,12 +1552,12 @@ type Value struct {
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
// id is the id of the Value. It is generated on saving it.
|
||||
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"` // @gotags: yaml:"id"
|
||||
BundleId string `protobuf:"bytes,2,opt,name=bundle_id,json=bundleId,proto3" json:"bundle_id,omitempty"` // @gotags: yaml:"bundle_id"
|
||||
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty" yaml:"id"` // @gotags: yaml:"id"
|
||||
BundleId string `protobuf:"bytes,2,opt,name=bundle_id,json=bundleId,proto3" json:"bundle_id,omitempty" yaml:"bundle_id"` // @gotags: yaml:"bundle_id"
|
||||
// setting_id is the id of the setting from within its bundle.
|
||||
SettingId string `protobuf:"bytes,3,opt,name=setting_id,json=settingId,proto3" json:"setting_id,omitempty"` // @gotags: yaml:"setting_id"
|
||||
AccountUuid string `protobuf:"bytes,4,opt,name=account_uuid,json=accountUuid,proto3" json:"account_uuid,omitempty"` // @gotags: yaml:"account_uuid"
|
||||
Resource *Resource `protobuf:"bytes,5,opt,name=resource,proto3" json:"resource,omitempty"` // @gotags: yaml:"resource"
|
||||
SettingId string `protobuf:"bytes,3,opt,name=setting_id,json=settingId,proto3" json:"setting_id,omitempty" yaml:"setting_id"` // @gotags: yaml:"setting_id"
|
||||
AccountUuid string `protobuf:"bytes,4,opt,name=account_uuid,json=accountUuid,proto3" json:"account_uuid,omitempty" yaml:"account_uuid"` // @gotags: yaml:"account_uuid"
|
||||
Resource *Resource `protobuf:"bytes,5,opt,name=resource,proto3" json:"resource,omitempty" yaml:"resource"` // @gotags: yaml:"resource"
|
||||
// Types that are assignable to Value:
|
||||
//
|
||||
// *Value_BoolValue
|
||||
@@ -1682,23 +1682,23 @@ type isValue_Value interface {
|
||||
}
|
||||
|
||||
type Value_BoolValue struct {
|
||||
BoolValue bool `protobuf:"varint,6,opt,name=bool_value,json=boolValue,proto3,oneof"` // @gotags: yaml:"bool_value"
|
||||
BoolValue bool `protobuf:"varint,6,opt,name=bool_value,json=boolValue,proto3,oneof" yaml:"bool_value"` // @gotags: yaml:"bool_value"
|
||||
}
|
||||
|
||||
type Value_IntValue struct {
|
||||
IntValue int64 `protobuf:"varint,7,opt,name=int_value,json=intValue,proto3,oneof"` // @gotags: yaml:"int_value"
|
||||
IntValue int64 `protobuf:"varint,7,opt,name=int_value,json=intValue,proto3,oneof" yaml:"int_value"` // @gotags: yaml:"int_value"
|
||||
}
|
||||
|
||||
type Value_StringValue struct {
|
||||
StringValue string `protobuf:"bytes,8,opt,name=string_value,json=stringValue,proto3,oneof"` // @gotags: yaml:"string_value"
|
||||
StringValue string `protobuf:"bytes,8,opt,name=string_value,json=stringValue,proto3,oneof" yaml:"string_value"` // @gotags: yaml:"string_value"
|
||||
}
|
||||
|
||||
type Value_ListValue struct {
|
||||
ListValue *ListValue `protobuf:"bytes,9,opt,name=list_value,json=listValue,proto3,oneof"` // @gotags: yaml:"list_value"
|
||||
ListValue *ListValue `protobuf:"bytes,9,opt,name=list_value,json=listValue,proto3,oneof" yaml:"list_value"` // @gotags: yaml:"list_value"
|
||||
}
|
||||
|
||||
type Value_CollectionValue struct {
|
||||
CollectionValue *CollectionValue `protobuf:"bytes,10,opt,name=collection_value,json=collectionValue,proto3,oneof"` // @gotags: yaml:"collection_value"
|
||||
CollectionValue *CollectionValue `protobuf:"bytes,10,opt,name=collection_value,json=collectionValue,proto3,oneof" yaml:"collection_value"` // @gotags: yaml:"collection_value"
|
||||
}
|
||||
|
||||
func (*Value_BoolValue) isValue_Value() {}
|
||||
@@ -1716,7 +1716,7 @@ type ListValue struct {
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Values []*ListOptionValue `protobuf:"bytes,1,rep,name=values,proto3" json:"values,omitempty"` // @gotags: yaml:"values"
|
||||
Values []*ListOptionValue `protobuf:"bytes,1,rep,name=values,proto3" json:"values,omitempty" yaml:"values"` // @gotags: yaml:"values"
|
||||
}
|
||||
|
||||
func (x *ListValue) Reset() {
|
||||
@@ -1836,15 +1836,15 @@ type isListOptionValue_Option interface {
|
||||
}
|
||||
|
||||
type ListOptionValue_StringValue struct {
|
||||
StringValue string `protobuf:"bytes,1,opt,name=string_value,json=stringValue,proto3,oneof"` // @gotags: yaml:"string_value"
|
||||
StringValue string `protobuf:"bytes,1,opt,name=string_value,json=stringValue,proto3,oneof" yaml:"string_value"` // @gotags: yaml:"string_value"
|
||||
}
|
||||
|
||||
type ListOptionValue_IntValue struct {
|
||||
IntValue int64 `protobuf:"varint,2,opt,name=int_value,json=intValue,proto3,oneof"` // @gotags: yaml:"int_value"
|
||||
IntValue int64 `protobuf:"varint,2,opt,name=int_value,json=intValue,proto3,oneof" yaml:"int_value"` // @gotags: yaml:"int_value"
|
||||
}
|
||||
|
||||
type ListOptionValue_BoolValue struct {
|
||||
BoolValue bool `protobuf:"varint,3,opt,name=bool_value,json=boolValue,proto3,oneof"` // @gotags: yaml:"bool_value"
|
||||
BoolValue bool `protobuf:"varint,3,opt,name=bool_value,json=boolValue,proto3,oneof" yaml:"bool_value"` // @gotags: yaml:"bool_value"
|
||||
}
|
||||
|
||||
func (*ListOptionValue_StringValue) isListOptionValue_Option() {}
|
||||
@@ -1858,7 +1858,7 @@ type CollectionValue struct {
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Values []*CollectionOption `protobuf:"bytes,1,rep,name=values,proto3" json:"values,omitempty"` // @gotags: yaml:"values"
|
||||
Values []*CollectionOption `protobuf:"bytes,1,rep,name=values,proto3" json:"values,omitempty" yaml:"values"` // @gotags: yaml:"values"
|
||||
}
|
||||
|
||||
func (x *CollectionValue) Reset() {
|
||||
@@ -1906,7 +1906,7 @@ type CollectionOption struct {
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
// required
|
||||
Key string `protobuf:"bytes,1,opt,name=key,proto3" json:"key,omitempty"` // @gotags: yaml:"key"
|
||||
Key string `protobuf:"bytes,1,opt,name=key,proto3" json:"key,omitempty" yaml:"key"` // @gotags: yaml:"key"
|
||||
// Types that are assignable to Option:
|
||||
//
|
||||
// *CollectionOption_IntValue
|
||||
@@ -1987,15 +1987,15 @@ type isCollectionOption_Option interface {
|
||||
}
|
||||
|
||||
type CollectionOption_IntValue struct {
|
||||
IntValue int64 `protobuf:"varint,2,opt,name=int_value,json=intValue,proto3,oneof"` // @gotags: yaml:"int_value"
|
||||
IntValue int64 `protobuf:"varint,2,opt,name=int_value,json=intValue,proto3,oneof" yaml:"int_value"` // @gotags: yaml:"int_value"
|
||||
}
|
||||
|
||||
type CollectionOption_StringValue struct {
|
||||
StringValue string `protobuf:"bytes,3,opt,name=string_value,json=stringValue,proto3,oneof"` // @gotags: yaml:"string_value"
|
||||
StringValue string `protobuf:"bytes,3,opt,name=string_value,json=stringValue,proto3,oneof" yaml:"string_value"` // @gotags: yaml:"string_value"
|
||||
}
|
||||
|
||||
type CollectionOption_BoolValue struct {
|
||||
BoolValue bool `protobuf:"varint,4,opt,name=bool_value,json=boolValue,proto3,oneof"` // @gotags: yaml:"bool_value"
|
||||
BoolValue bool `protobuf:"varint,4,opt,name=bool_value,json=boolValue,proto3,oneof" yaml:"bool_value"` // @gotags: yaml:"bool_value"
|
||||
}
|
||||
|
||||
func (*CollectionOption_IntValue) isCollectionOption_Option() {}
|
||||
|
||||
@@ -40,7 +40,7 @@ func (_m *SearchProviderService) EXPECT() *SearchProviderService_Expecter {
|
||||
}
|
||||
|
||||
// IndexSpace provides a mock function for the type SearchProviderService
|
||||
func (_mock *SearchProviderService) IndexSpace(ctx context.Context, in *v0.IndexSpaceRequest, opts ...client.CallOption) (*v0.IndexSpaceResponse, error) {
|
||||
func (_mock *SearchProviderService) IndexSpace(ctx context.Context, in *v0.IndexSpaceRequest, opts ...client.CallOption) (v0.SearchProvider_IndexSpaceService, error) {
|
||||
var tmpRet mock.Arguments
|
||||
if len(opts) > 0 {
|
||||
tmpRet = _mock.Called(ctx, in, opts)
|
||||
@@ -53,16 +53,16 @@ func (_mock *SearchProviderService) IndexSpace(ctx context.Context, in *v0.Index
|
||||
panic("no return value specified for IndexSpace")
|
||||
}
|
||||
|
||||
var r0 *v0.IndexSpaceResponse
|
||||
var r0 v0.SearchProvider_IndexSpaceService
|
||||
var r1 error
|
||||
if returnFunc, ok := ret.Get(0).(func(context.Context, *v0.IndexSpaceRequest, ...client.CallOption) (*v0.IndexSpaceResponse, error)); ok {
|
||||
if returnFunc, ok := ret.Get(0).(func(context.Context, *v0.IndexSpaceRequest, ...client.CallOption) (v0.SearchProvider_IndexSpaceService, error)); ok {
|
||||
return returnFunc(ctx, in, opts...)
|
||||
}
|
||||
if returnFunc, ok := ret.Get(0).(func(context.Context, *v0.IndexSpaceRequest, ...client.CallOption) *v0.IndexSpaceResponse); ok {
|
||||
if returnFunc, ok := ret.Get(0).(func(context.Context, *v0.IndexSpaceRequest, ...client.CallOption) v0.SearchProvider_IndexSpaceService); ok {
|
||||
r0 = returnFunc(ctx, in, opts...)
|
||||
} else {
|
||||
if ret.Get(0) != nil {
|
||||
r0 = ret.Get(0).(*v0.IndexSpaceResponse)
|
||||
r0 = ret.Get(0).(v0.SearchProvider_IndexSpaceService)
|
||||
}
|
||||
}
|
||||
if returnFunc, ok := ret.Get(1).(func(context.Context, *v0.IndexSpaceRequest, ...client.CallOption) error); ok {
|
||||
@@ -112,12 +112,12 @@ func (_c *SearchProviderService_IndexSpace_Call) Run(run func(ctx context.Contex
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *SearchProviderService_IndexSpace_Call) Return(indexSpaceResponse *v0.IndexSpaceResponse, err error) *SearchProviderService_IndexSpace_Call {
|
||||
_c.Call.Return(indexSpaceResponse, err)
|
||||
func (_c *SearchProviderService_IndexSpace_Call) Return(searchProvider_IndexSpaceService v0.SearchProvider_IndexSpaceService, err error) *SearchProviderService_IndexSpace_Call {
|
||||
_c.Call.Return(searchProvider_IndexSpaceService, err)
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *SearchProviderService_IndexSpace_Call) RunAndReturn(run func(ctx context.Context, in *v0.IndexSpaceRequest, opts ...client.CallOption) (*v0.IndexSpaceResponse, error)) *SearchProviderService_IndexSpace_Call {
|
||||
func (_c *SearchProviderService_IndexSpace_Call) RunAndReturn(run func(ctx context.Context, in *v0.IndexSpaceRequest, opts ...client.CallOption) (v0.SearchProvider_IndexSpaceService, error)) *SearchProviderService_IndexSpace_Call {
|
||||
_c.Call.Return(run)
|
||||
return _c
|
||||
}
|
||||
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
_ "google.golang.org/genproto/googleapis/api/annotations"
|
||||
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
|
||||
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
|
||||
durationpb "google.golang.org/protobuf/types/known/durationpb"
|
||||
_ "google.golang.org/protobuf/types/known/fieldmaskpb"
|
||||
reflect "reflect"
|
||||
sync "sync"
|
||||
@@ -310,6 +311,7 @@ type IndexSpaceRequest struct {
|
||||
SpaceId string `protobuf:"bytes,1,opt,name=space_id,json=spaceId,proto3" json:"space_id,omitempty"`
|
||||
UserId string `protobuf:"bytes,2,opt,name=user_id,json=userId,proto3" json:"user_id,omitempty"`
|
||||
ForceReindex bool `protobuf:"varint,3,opt,name=force_reindex,json=forceReindex,proto3" json:"force_reindex,omitempty"`
|
||||
Concurrency int32 `protobuf:"varint,4,opt,name=concurrency,proto3" json:"concurrency,omitempty"`
|
||||
}
|
||||
|
||||
func (x *IndexSpaceRequest) Reset() {
|
||||
@@ -365,10 +367,28 @@ func (x *IndexSpaceRequest) GetForceReindex() bool {
|
||||
return false
|
||||
}
|
||||
|
||||
func (x *IndexSpaceRequest) GetConcurrency() int32 {
|
||||
if x != nil {
|
||||
return x.Concurrency
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
type IndexSpaceResponse struct {
|
||||
state protoimpl.MessageState
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
// The id of the space that has just been indexed.
|
||||
SpaceId string `protobuf:"bytes,1,opt,name=space_id,json=spaceId,proto3" json:"space_id,omitempty"`
|
||||
// The duration it took to index this space.
|
||||
SpaceDuration *durationpb.Duration `protobuf:"bytes,2,opt,name=space_duration,json=spaceDuration,proto3" json:"space_duration,omitempty"`
|
||||
// The number of spaces that have been indexed so far.
|
||||
IndexedSpaces int64 `protobuf:"varint,3,opt,name=indexed_spaces,json=indexedSpaces,proto3" json:"indexed_spaces,omitempty"`
|
||||
// The total number of spaces that are being indexed.
|
||||
TotalSpaces int64 `protobuf:"varint,4,opt,name=total_spaces,json=totalSpaces,proto3" json:"total_spaces,omitempty"`
|
||||
// Contains an error message in case indexing this particular space failed.
|
||||
Error string `protobuf:"bytes,5,opt,name=error,proto3" json:"error,omitempty"`
|
||||
}
|
||||
|
||||
func (x *IndexSpaceResponse) Reset() {
|
||||
@@ -403,6 +423,41 @@ func (*IndexSpaceResponse) Descriptor() ([]byte, []int) {
|
||||
return file_opencloud_services_search_v0_search_proto_rawDescGZIP(), []int{5}
|
||||
}
|
||||
|
||||
func (x *IndexSpaceResponse) GetSpaceId() string {
|
||||
if x != nil {
|
||||
return x.SpaceId
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *IndexSpaceResponse) GetSpaceDuration() *durationpb.Duration {
|
||||
if x != nil {
|
||||
return x.SpaceDuration
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (x *IndexSpaceResponse) GetIndexedSpaces() int64 {
|
||||
if x != nil {
|
||||
return x.IndexedSpaces
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func (x *IndexSpaceResponse) GetTotalSpaces() int64 {
|
||||
if x != nil {
|
||||
return x.TotalSpaces
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func (x *IndexSpaceResponse) GetError() string {
|
||||
if x != nil {
|
||||
return x.Error
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
var File_opencloud_services_search_v0_search_proto protoreflect.FileDescriptor
|
||||
|
||||
var file_opencloud_services_search_v0_search_proto_rawDesc = []byte{
|
||||
@@ -422,6 +477,8 @@ var file_opencloud_services_search_v0_search_proto_rawDesc = []byte{
|
||||
0x69, 0x2f, 0x61, 0x6e, 0x6e, 0x6f, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x2e, 0x70, 0x72,
|
||||
0x6f, 0x74, 0x6f, 0x1a, 0x20, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74,
|
||||
0x6f, 0x62, 0x75, 0x66, 0x2f, 0x66, 0x69, 0x65, 0x6c, 0x64, 0x5f, 0x6d, 0x61, 0x73, 0x6b, 0x2e,
|
||||
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x1e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2f, 0x70, 0x72,
|
||||
0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x64, 0x75, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x2e,
|
||||
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0xae, 0x01, 0x0a, 0x0d, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68,
|
||||
0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x21, 0x0a, 0x09, 0x70, 0x61, 0x67, 0x65, 0x5f,
|
||||
0x73, 0x69, 0x7a, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x05, 0x42, 0x04, 0xe2, 0x41, 0x01, 0x01,
|
||||
@@ -465,68 +522,83 @@ var file_opencloud_services_search_v0_search_proto_rawDesc = []byte{
|
||||
0x74, 0x50, 0x61, 0x67, 0x65, 0x54, 0x6f, 0x6b, 0x65, 0x6e, 0x12, 0x23, 0x0a, 0x0d, 0x74, 0x6f,
|
||||
0x74, 0x61, 0x6c, 0x5f, 0x6d, 0x61, 0x74, 0x63, 0x68, 0x65, 0x73, 0x18, 0x03, 0x20, 0x01, 0x28,
|
||||
0x05, 0x52, 0x0c, 0x74, 0x6f, 0x74, 0x61, 0x6c, 0x4d, 0x61, 0x74, 0x63, 0x68, 0x65, 0x73, 0x22,
|
||||
0x6c, 0x0a, 0x11, 0x49, 0x6e, 0x64, 0x65, 0x78, 0x53, 0x70, 0x61, 0x63, 0x65, 0x52, 0x65, 0x71,
|
||||
0x75, 0x65, 0x73, 0x74, 0x12, 0x19, 0x0a, 0x08, 0x73, 0x70, 0x61, 0x63, 0x65, 0x5f, 0x69, 0x64,
|
||||
0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x73, 0x70, 0x61, 0x63, 0x65, 0x49, 0x64, 0x12,
|
||||
0x17, 0x0a, 0x07, 0x75, 0x73, 0x65, 0x72, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09,
|
||||
0x52, 0x06, 0x75, 0x73, 0x65, 0x72, 0x49, 0x64, 0x12, 0x23, 0x0a, 0x0d, 0x66, 0x6f, 0x72, 0x63,
|
||||
0x65, 0x5f, 0x72, 0x65, 0x69, 0x6e, 0x64, 0x65, 0x78, 0x18, 0x03, 0x20, 0x01, 0x28, 0x08, 0x52,
|
||||
0x0c, 0x66, 0x6f, 0x72, 0x63, 0x65, 0x52, 0x65, 0x69, 0x6e, 0x64, 0x65, 0x78, 0x22, 0x14, 0x0a,
|
||||
0x12, 0x49, 0x6e, 0x64, 0x65, 0x78, 0x53, 0x70, 0x61, 0x63, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f,
|
||||
0x6e, 0x73, 0x65, 0x32, 0xb1, 0x02, 0x0a, 0x0e, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x50, 0x72,
|
||||
0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x12, 0x85, 0x01, 0x0a, 0x06, 0x53, 0x65, 0x61, 0x72, 0x63,
|
||||
0x68, 0x12, 0x2b, 0x2e, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2e, 0x73, 0x65,
|
||||
0x72, 0x76, 0x69, 0x63, 0x65, 0x73, 0x2e, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2e, 0x76, 0x30,
|
||||
0x2e, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2c,
|
||||
0x2e, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2e, 0x73, 0x65, 0x72, 0x76, 0x69,
|
||||
0x63, 0x65, 0x73, 0x2e, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2e, 0x76, 0x30, 0x2e, 0x53, 0x65,
|
||||
0x61, 0x72, 0x63, 0x68, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x20, 0x82, 0xd3,
|
||||
0xe4, 0x93, 0x02, 0x1a, 0x3a, 0x01, 0x2a, 0x22, 0x15, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x76, 0x30,
|
||||
0x2f, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2f, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x12, 0x96,
|
||||
0x01, 0x0a, 0x0a, 0x49, 0x6e, 0x64, 0x65, 0x78, 0x53, 0x70, 0x61, 0x63, 0x65, 0x12, 0x2f, 0x2e,
|
||||
0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2e, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63,
|
||||
0x65, 0x73, 0x2e, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2e, 0x76, 0x30, 0x2e, 0x49, 0x6e, 0x64,
|
||||
0x65, 0x78, 0x53, 0x70, 0x61, 0x63, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x30,
|
||||
0x2e, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2e, 0x73, 0x65, 0x72, 0x76, 0x69,
|
||||
0x63, 0x65, 0x73, 0x2e, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2e, 0x76, 0x30, 0x2e, 0x49, 0x6e,
|
||||
0x64, 0x65, 0x78, 0x53, 0x70, 0x61, 0x63, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
|
||||
0x22, 0x25, 0x82, 0xd3, 0xe4, 0x93, 0x02, 0x1f, 0x3a, 0x01, 0x2a, 0x22, 0x1a, 0x2f, 0x61, 0x70,
|
||||
0x69, 0x2f, 0x76, 0x30, 0x2f, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2f, 0x69, 0x6e, 0x64, 0x65,
|
||||
0x78, 0x2d, 0x73, 0x70, 0x61, 0x63, 0x65, 0x32, 0xa7, 0x01, 0x0a, 0x0d, 0x49, 0x6e, 0x64, 0x65,
|
||||
0x78, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x12, 0x95, 0x01, 0x0a, 0x06, 0x53, 0x65,
|
||||
0x61, 0x72, 0x63, 0x68, 0x12, 0x30, 0x2e, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64,
|
||||
0x94, 0x01, 0x0a, 0x11, 0x49, 0x6e, 0x64, 0x65, 0x78, 0x53, 0x70, 0x61, 0x63, 0x65, 0x52, 0x65,
|
||||
0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x19, 0x0a, 0x08, 0x73, 0x70, 0x61, 0x63, 0x65, 0x5f, 0x69,
|
||||
0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x73, 0x70, 0x61, 0x63, 0x65, 0x49, 0x64,
|
||||
0x12, 0x17, 0x0a, 0x07, 0x75, 0x73, 0x65, 0x72, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28,
|
||||
0x09, 0x52, 0x06, 0x75, 0x73, 0x65, 0x72, 0x49, 0x64, 0x12, 0x23, 0x0a, 0x0d, 0x66, 0x6f, 0x72,
|
||||
0x63, 0x65, 0x5f, 0x72, 0x65, 0x69, 0x6e, 0x64, 0x65, 0x78, 0x18, 0x03, 0x20, 0x01, 0x28, 0x08,
|
||||
0x52, 0x0c, 0x66, 0x6f, 0x72, 0x63, 0x65, 0x52, 0x65, 0x69, 0x6e, 0x64, 0x65, 0x78, 0x12, 0x26,
|
||||
0x0a, 0x0b, 0x63, 0x6f, 0x6e, 0x63, 0x75, 0x72, 0x72, 0x65, 0x6e, 0x63, 0x79, 0x18, 0x04, 0x20,
|
||||
0x01, 0x28, 0x05, 0x42, 0x04, 0xe2, 0x41, 0x01, 0x01, 0x52, 0x0b, 0x63, 0x6f, 0x6e, 0x63, 0x75,
|
||||
0x72, 0x72, 0x65, 0x6e, 0x63, 0x79, 0x22, 0xd1, 0x01, 0x0a, 0x12, 0x49, 0x6e, 0x64, 0x65, 0x78,
|
||||
0x53, 0x70, 0x61, 0x63, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x19, 0x0a,
|
||||
0x08, 0x73, 0x70, 0x61, 0x63, 0x65, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52,
|
||||
0x07, 0x73, 0x70, 0x61, 0x63, 0x65, 0x49, 0x64, 0x12, 0x40, 0x0a, 0x0e, 0x73, 0x70, 0x61, 0x63,
|
||||
0x65, 0x5f, 0x64, 0x75, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b,
|
||||
0x32, 0x19, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62,
|
||||
0x75, 0x66, 0x2e, 0x44, 0x75, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x0d, 0x73, 0x70, 0x61,
|
||||
0x63, 0x65, 0x44, 0x75, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x25, 0x0a, 0x0e, 0x69, 0x6e,
|
||||
0x64, 0x65, 0x78, 0x65, 0x64, 0x5f, 0x73, 0x70, 0x61, 0x63, 0x65, 0x73, 0x18, 0x03, 0x20, 0x01,
|
||||
0x28, 0x03, 0x52, 0x0d, 0x69, 0x6e, 0x64, 0x65, 0x78, 0x65, 0x64, 0x53, 0x70, 0x61, 0x63, 0x65,
|
||||
0x73, 0x12, 0x21, 0x0a, 0x0c, 0x74, 0x6f, 0x74, 0x61, 0x6c, 0x5f, 0x73, 0x70, 0x61, 0x63, 0x65,
|
||||
0x73, 0x18, 0x04, 0x20, 0x01, 0x28, 0x03, 0x52, 0x0b, 0x74, 0x6f, 0x74, 0x61, 0x6c, 0x53, 0x70,
|
||||
0x61, 0x63, 0x65, 0x73, 0x12, 0x14, 0x0a, 0x05, 0x65, 0x72, 0x72, 0x6f, 0x72, 0x18, 0x05, 0x20,
|
||||
0x01, 0x28, 0x09, 0x52, 0x05, 0x65, 0x72, 0x72, 0x6f, 0x72, 0x32, 0xb3, 0x02, 0x0a, 0x0e, 0x53,
|
||||
0x65, 0x61, 0x72, 0x63, 0x68, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x12, 0x85, 0x01,
|
||||
0x0a, 0x06, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x12, 0x2b, 0x2e, 0x6f, 0x70, 0x65, 0x6e, 0x63,
|
||||
0x6c, 0x6f, 0x75, 0x64, 0x2e, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x73, 0x2e, 0x73, 0x65,
|
||||
0x61, 0x72, 0x63, 0x68, 0x2e, 0x76, 0x30, 0x2e, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x52, 0x65,
|
||||
0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2c, 0x2e, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75,
|
||||
0x64, 0x2e, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x73, 0x2e, 0x73, 0x65, 0x61, 0x72, 0x63,
|
||||
0x68, 0x2e, 0x76, 0x30, 0x2e, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x52, 0x65, 0x73, 0x70, 0x6f,
|
||||
0x6e, 0x73, 0x65, 0x22, 0x20, 0x82, 0xd3, 0xe4, 0x93, 0x02, 0x1a, 0x3a, 0x01, 0x2a, 0x22, 0x15,
|
||||
0x2f, 0x61, 0x70, 0x69, 0x2f, 0x76, 0x30, 0x2f, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2f, 0x73,
|
||||
0x65, 0x61, 0x72, 0x63, 0x68, 0x12, 0x98, 0x01, 0x0a, 0x0a, 0x49, 0x6e, 0x64, 0x65, 0x78, 0x53,
|
||||
0x70, 0x61, 0x63, 0x65, 0x12, 0x2f, 0x2e, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64,
|
||||
0x2e, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x73, 0x2e, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68,
|
||||
0x2e, 0x76, 0x30, 0x2e, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x49, 0x6e, 0x64, 0x65, 0x78, 0x52,
|
||||
0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x31, 0x2e, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f,
|
||||
0x75, 0x64, 0x2e, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x73, 0x2e, 0x73, 0x65, 0x61, 0x72,
|
||||
0x63, 0x68, 0x2e, 0x76, 0x30, 0x2e, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x49, 0x6e, 0x64, 0x65,
|
||||
0x78, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x26, 0x82, 0xd3, 0xe4, 0x93, 0x02,
|
||||
0x20, 0x3a, 0x01, 0x2a, 0x22, 0x1b, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x76, 0x30, 0x2f, 0x73, 0x65,
|
||||
0x61, 0x72, 0x63, 0x68, 0x2f, 0x69, 0x6e, 0x64, 0x65, 0x78, 0x2f, 0x73, 0x65, 0x61, 0x72, 0x63,
|
||||
0x68, 0x42, 0xf2, 0x02, 0x5a, 0x4a, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d,
|
||||
0x2f, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2d, 0x65, 0x75, 0x2f, 0x6f, 0x70,
|
||||
0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x67, 0x65, 0x6e,
|
||||
0x2f, 0x67, 0x65, 0x6e, 0x2f, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2f, 0x73,
|
||||
0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x2f, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2f, 0x76, 0x30,
|
||||
0x92, 0x41, 0xa2, 0x02, 0x12, 0xb7, 0x01, 0x0a, 0x10, 0x4f, 0x70, 0x65, 0x6e, 0x43, 0x6c, 0x6f,
|
||||
0x75, 0x64, 0x20, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x22, 0x51, 0x0a, 0x0e, 0x4f, 0x70, 0x65,
|
||||
0x6e, 0x43, 0x6c, 0x6f, 0x75, 0x64, 0x20, 0x47, 0x6d, 0x62, 0x48, 0x12, 0x29, 0x68, 0x74, 0x74,
|
||||
0x70, 0x73, 0x3a, 0x2f, 0x2f, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f,
|
||||
0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2d, 0x65, 0x75, 0x2f, 0x6f, 0x70, 0x65,
|
||||
0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x1a, 0x14, 0x73, 0x75, 0x70, 0x70, 0x6f, 0x72, 0x74, 0x40,
|
||||
0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2e, 0x65, 0x75, 0x2a, 0x49, 0x0a, 0x0a,
|
||||
0x41, 0x70, 0x61, 0x63, 0x68, 0x65, 0x2d, 0x32, 0x2e, 0x30, 0x12, 0x3b, 0x68, 0x74, 0x74, 0x70,
|
||||
0x73, 0x3a, 0x2f, 0x2f, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x6f,
|
||||
0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2d, 0x65, 0x75, 0x2f, 0x6f, 0x70, 0x65, 0x6e,
|
||||
0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2f, 0x62, 0x6c, 0x6f, 0x62, 0x2f, 0x6d, 0x61, 0x69, 0x6e, 0x2f,
|
||||
0x4c, 0x49, 0x43, 0x45, 0x4e, 0x53, 0x45, 0x32, 0x05, 0x31, 0x2e, 0x30, 0x2e, 0x30, 0x2a, 0x02,
|
||||
0x01, 0x02, 0x32, 0x10, 0x61, 0x70, 0x70, 0x6c, 0x69, 0x63, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x2f,
|
||||
0x6a, 0x73, 0x6f, 0x6e, 0x3a, 0x10, 0x61, 0x70, 0x70, 0x6c, 0x69, 0x63, 0x61, 0x74, 0x69, 0x6f,
|
||||
0x6e, 0x2f, 0x6a, 0x73, 0x6f, 0x6e, 0x72, 0x3e, 0x0a, 0x10, 0x44, 0x65, 0x76, 0x65, 0x6c, 0x6f,
|
||||
0x70, 0x65, 0x72, 0x20, 0x4d, 0x61, 0x6e, 0x75, 0x61, 0x6c, 0x12, 0x2a, 0x68, 0x74, 0x74, 0x70,
|
||||
0x73, 0x3a, 0x2f, 0x2f, 0x64, 0x6f, 0x63, 0x73, 0x2e, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f,
|
||||
0x75, 0x64, 0x2e, 0x65, 0x75, 0x2f, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x73, 0x2f, 0x73,
|
||||
0x65, 0x61, 0x72, 0x63, 0x68, 0x2f, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
|
||||
0x2e, 0x76, 0x30, 0x2e, 0x49, 0x6e, 0x64, 0x65, 0x78, 0x53, 0x70, 0x61, 0x63, 0x65, 0x52, 0x65,
|
||||
0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x30, 0x2e, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75,
|
||||
0x64, 0x2e, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x73, 0x2e, 0x73, 0x65, 0x61, 0x72, 0x63,
|
||||
0x68, 0x2e, 0x76, 0x30, 0x2e, 0x49, 0x6e, 0x64, 0x65, 0x78, 0x53, 0x70, 0x61, 0x63, 0x65, 0x52,
|
||||
0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x25, 0x82, 0xd3, 0xe4, 0x93, 0x02, 0x1f, 0x3a,
|
||||
0x01, 0x2a, 0x22, 0x1a, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x76, 0x30, 0x2f, 0x73, 0x65, 0x61, 0x72,
|
||||
0x63, 0x68, 0x2f, 0x69, 0x6e, 0x64, 0x65, 0x78, 0x2d, 0x73, 0x70, 0x61, 0x63, 0x65, 0x30, 0x01,
|
||||
0x32, 0xa7, 0x01, 0x0a, 0x0d, 0x49, 0x6e, 0x64, 0x65, 0x78, 0x50, 0x72, 0x6f, 0x76, 0x69, 0x64,
|
||||
0x65, 0x72, 0x12, 0x95, 0x01, 0x0a, 0x06, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x12, 0x30, 0x2e,
|
||||
0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2e, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63,
|
||||
0x65, 0x73, 0x2e, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2e, 0x76, 0x30, 0x2e, 0x53, 0x65, 0x61,
|
||||
0x72, 0x63, 0x68, 0x49, 0x6e, 0x64, 0x65, 0x78, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a,
|
||||
0x31, 0x2e, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2e, 0x73, 0x65, 0x72, 0x76,
|
||||
0x69, 0x63, 0x65, 0x73, 0x2e, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2e, 0x76, 0x30, 0x2e, 0x53,
|
||||
0x65, 0x61, 0x72, 0x63, 0x68, 0x49, 0x6e, 0x64, 0x65, 0x78, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e,
|
||||
0x73, 0x65, 0x22, 0x26, 0x82, 0xd3, 0xe4, 0x93, 0x02, 0x20, 0x3a, 0x01, 0x2a, 0x22, 0x1b, 0x2f,
|
||||
0x61, 0x70, 0x69, 0x2f, 0x76, 0x30, 0x2f, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2f, 0x69, 0x6e,
|
||||
0x64, 0x65, 0x78, 0x2f, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x42, 0xf2, 0x02, 0x5a, 0x4a, 0x67,
|
||||
0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c,
|
||||
0x6f, 0x75, 0x64, 0x2d, 0x65, 0x75, 0x2f, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64,
|
||||
0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x67, 0x65, 0x6e, 0x2f, 0x67, 0x65, 0x6e, 0x2f, 0x6f, 0x70,
|
||||
0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2f, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x2f,
|
||||
0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2f, 0x76, 0x30, 0x92, 0x41, 0xa2, 0x02, 0x12, 0xb7, 0x01,
|
||||
0x0a, 0x10, 0x4f, 0x70, 0x65, 0x6e, 0x43, 0x6c, 0x6f, 0x75, 0x64, 0x20, 0x73, 0x65, 0x61, 0x72,
|
||||
0x63, 0x68, 0x22, 0x51, 0x0a, 0x0e, 0x4f, 0x70, 0x65, 0x6e, 0x43, 0x6c, 0x6f, 0x75, 0x64, 0x20,
|
||||
0x47, 0x6d, 0x62, 0x48, 0x12, 0x29, 0x68, 0x74, 0x74, 0x70, 0x73, 0x3a, 0x2f, 0x2f, 0x67, 0x69,
|
||||
0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f,
|
||||
0x75, 0x64, 0x2d, 0x65, 0x75, 0x2f, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x1a,
|
||||
0x14, 0x73, 0x75, 0x70, 0x70, 0x6f, 0x72, 0x74, 0x40, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f,
|
||||
0x75, 0x64, 0x2e, 0x65, 0x75, 0x2a, 0x49, 0x0a, 0x0a, 0x41, 0x70, 0x61, 0x63, 0x68, 0x65, 0x2d,
|
||||
0x32, 0x2e, 0x30, 0x12, 0x3b, 0x68, 0x74, 0x74, 0x70, 0x73, 0x3a, 0x2f, 0x2f, 0x67, 0x69, 0x74,
|
||||
0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75,
|
||||
0x64, 0x2d, 0x65, 0x75, 0x2f, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2f, 0x62,
|
||||
0x6c, 0x6f, 0x62, 0x2f, 0x6d, 0x61, 0x69, 0x6e, 0x2f, 0x4c, 0x49, 0x43, 0x45, 0x4e, 0x53, 0x45,
|
||||
0x32, 0x05, 0x31, 0x2e, 0x30, 0x2e, 0x30, 0x2a, 0x02, 0x01, 0x02, 0x32, 0x10, 0x61, 0x70, 0x70,
|
||||
0x6c, 0x69, 0x63, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x2f, 0x6a, 0x73, 0x6f, 0x6e, 0x3a, 0x10, 0x61,
|
||||
0x70, 0x70, 0x6c, 0x69, 0x63, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x2f, 0x6a, 0x73, 0x6f, 0x6e, 0x72,
|
||||
0x3e, 0x0a, 0x10, 0x44, 0x65, 0x76, 0x65, 0x6c, 0x6f, 0x70, 0x65, 0x72, 0x20, 0x4d, 0x61, 0x6e,
|
||||
0x75, 0x61, 0x6c, 0x12, 0x2a, 0x68, 0x74, 0x74, 0x70, 0x73, 0x3a, 0x2f, 0x2f, 0x64, 0x6f, 0x63,
|
||||
0x73, 0x2e, 0x6f, 0x70, 0x65, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2e, 0x65, 0x75, 0x2f, 0x73,
|
||||
0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x73, 0x2f, 0x73, 0x65, 0x61, 0x72, 0x63, 0x68, 0x2f, 0x62,
|
||||
0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
|
||||
}
|
||||
|
||||
var (
|
||||
@@ -551,23 +623,25 @@ var file_opencloud_services_search_v0_search_proto_goTypes = []interface{}{
|
||||
(*IndexSpaceResponse)(nil), // 5: opencloud.services.search.v0.IndexSpaceResponse
|
||||
(*v0.Reference)(nil), // 6: opencloud.messages.search.v0.Reference
|
||||
(*v0.Match)(nil), // 7: opencloud.messages.search.v0.Match
|
||||
(*durationpb.Duration)(nil), // 8: google.protobuf.Duration
|
||||
}
|
||||
var file_opencloud_services_search_v0_search_proto_depIdxs = []int32{
|
||||
6, // 0: opencloud.services.search.v0.SearchRequest.ref:type_name -> opencloud.messages.search.v0.Reference
|
||||
7, // 1: opencloud.services.search.v0.SearchResponse.matches:type_name -> opencloud.messages.search.v0.Match
|
||||
6, // 2: opencloud.services.search.v0.SearchIndexRequest.ref:type_name -> opencloud.messages.search.v0.Reference
|
||||
7, // 3: opencloud.services.search.v0.SearchIndexResponse.matches:type_name -> opencloud.messages.search.v0.Match
|
||||
0, // 4: opencloud.services.search.v0.SearchProvider.Search:input_type -> opencloud.services.search.v0.SearchRequest
|
||||
4, // 5: opencloud.services.search.v0.SearchProvider.IndexSpace:input_type -> opencloud.services.search.v0.IndexSpaceRequest
|
||||
2, // 6: opencloud.services.search.v0.IndexProvider.Search:input_type -> opencloud.services.search.v0.SearchIndexRequest
|
||||
1, // 7: opencloud.services.search.v0.SearchProvider.Search:output_type -> opencloud.services.search.v0.SearchResponse
|
||||
5, // 8: opencloud.services.search.v0.SearchProvider.IndexSpace:output_type -> opencloud.services.search.v0.IndexSpaceResponse
|
||||
3, // 9: opencloud.services.search.v0.IndexProvider.Search:output_type -> opencloud.services.search.v0.SearchIndexResponse
|
||||
7, // [7:10] is the sub-list for method output_type
|
||||
4, // [4:7] is the sub-list for method input_type
|
||||
4, // [4:4] is the sub-list for extension type_name
|
||||
4, // [4:4] is the sub-list for extension extendee
|
||||
0, // [0:4] is the sub-list for field type_name
|
||||
8, // 4: opencloud.services.search.v0.IndexSpaceResponse.space_duration:type_name -> google.protobuf.Duration
|
||||
0, // 5: opencloud.services.search.v0.SearchProvider.Search:input_type -> opencloud.services.search.v0.SearchRequest
|
||||
4, // 6: opencloud.services.search.v0.SearchProvider.IndexSpace:input_type -> opencloud.services.search.v0.IndexSpaceRequest
|
||||
2, // 7: opencloud.services.search.v0.IndexProvider.Search:input_type -> opencloud.services.search.v0.SearchIndexRequest
|
||||
1, // 8: opencloud.services.search.v0.SearchProvider.Search:output_type -> opencloud.services.search.v0.SearchResponse
|
||||
5, // 9: opencloud.services.search.v0.SearchProvider.IndexSpace:output_type -> opencloud.services.search.v0.IndexSpaceResponse
|
||||
3, // 10: opencloud.services.search.v0.IndexProvider.Search:output_type -> opencloud.services.search.v0.SearchIndexResponse
|
||||
8, // [8:11] is the sub-list for method output_type
|
||||
5, // [5:8] is the sub-list for method input_type
|
||||
5, // [5:5] is the sub-list for extension type_name
|
||||
5, // [5:5] is the sub-list for extension extendee
|
||||
0, // [0:5] is the sub-list for field type_name
|
||||
}
|
||||
|
||||
func init() { file_opencloud_services_search_v0_search_proto_init() }
|
||||
|
||||
@@ -9,6 +9,7 @@ import (
|
||||
_ "github.com/opencloud-eu/opencloud/protogen/gen/opencloud/messages/search/v0"
|
||||
_ "google.golang.org/genproto/googleapis/api/annotations"
|
||||
proto "google.golang.org/protobuf/proto"
|
||||
_ "google.golang.org/protobuf/types/known/durationpb"
|
||||
_ "google.golang.org/protobuf/types/known/fieldmaskpb"
|
||||
math "math"
|
||||
)
|
||||
@@ -45,6 +46,7 @@ func NewSearchProviderEndpoints() []*api.Endpoint {
|
||||
Name: "SearchProvider.IndexSpace",
|
||||
Path: []string{"/api/v0/search/index-space"},
|
||||
Method: []string{"POST"},
|
||||
Stream: true,
|
||||
Handler: "rpc",
|
||||
},
|
||||
}
|
||||
@@ -54,7 +56,9 @@ func NewSearchProviderEndpoints() []*api.Endpoint {
|
||||
|
||||
type SearchProviderService interface {
|
||||
Search(ctx context.Context, in *SearchRequest, opts ...client.CallOption) (*SearchResponse, error)
|
||||
IndexSpace(ctx context.Context, in *IndexSpaceRequest, opts ...client.CallOption) (*IndexSpaceResponse, error)
|
||||
// IndexSpace (re)indexes one or all spaces. The response is streamed, sending
|
||||
// progress information after each space has been indexed.
|
||||
IndexSpace(ctx context.Context, in *IndexSpaceRequest, opts ...client.CallOption) (SearchProvider_IndexSpaceService, error)
|
||||
}
|
||||
|
||||
type searchProviderService struct {
|
||||
@@ -79,27 +83,73 @@ func (c *searchProviderService) Search(ctx context.Context, in *SearchRequest, o
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *searchProviderService) IndexSpace(ctx context.Context, in *IndexSpaceRequest, opts ...client.CallOption) (*IndexSpaceResponse, error) {
|
||||
req := c.c.NewRequest(c.name, "SearchProvider.IndexSpace", in)
|
||||
out := new(IndexSpaceResponse)
|
||||
err := c.c.Call(ctx, req, out, opts...)
|
||||
func (c *searchProviderService) IndexSpace(ctx context.Context, in *IndexSpaceRequest, opts ...client.CallOption) (SearchProvider_IndexSpaceService, error) {
|
||||
req := c.c.NewRequest(c.name, "SearchProvider.IndexSpace", &IndexSpaceRequest{})
|
||||
stream, err := c.c.Stream(ctx, req, opts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
if err := stream.Send(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &searchProviderServiceIndexSpace{stream}, nil
|
||||
}
|
||||
|
||||
type SearchProvider_IndexSpaceService interface {
|
||||
Context() context.Context
|
||||
SendMsg(interface{}) error
|
||||
RecvMsg(interface{}) error
|
||||
CloseSend() error
|
||||
Close() error
|
||||
Recv() (*IndexSpaceResponse, error)
|
||||
}
|
||||
|
||||
type searchProviderServiceIndexSpace struct {
|
||||
stream client.Stream
|
||||
}
|
||||
|
||||
func (x *searchProviderServiceIndexSpace) CloseSend() error {
|
||||
return x.stream.CloseSend()
|
||||
}
|
||||
|
||||
func (x *searchProviderServiceIndexSpace) Close() error {
|
||||
return x.stream.Close()
|
||||
}
|
||||
|
||||
func (x *searchProviderServiceIndexSpace) Context() context.Context {
|
||||
return x.stream.Context()
|
||||
}
|
||||
|
||||
func (x *searchProviderServiceIndexSpace) SendMsg(m interface{}) error {
|
||||
return x.stream.Send(m)
|
||||
}
|
||||
|
||||
func (x *searchProviderServiceIndexSpace) RecvMsg(m interface{}) error {
|
||||
return x.stream.Recv(m)
|
||||
}
|
||||
|
||||
func (x *searchProviderServiceIndexSpace) Recv() (*IndexSpaceResponse, error) {
|
||||
m := new(IndexSpaceResponse)
|
||||
err := x.stream.Recv(m)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
// Server API for SearchProvider service
|
||||
|
||||
type SearchProviderHandler interface {
|
||||
Search(context.Context, *SearchRequest, *SearchResponse) error
|
||||
IndexSpace(context.Context, *IndexSpaceRequest, *IndexSpaceResponse) error
|
||||
// IndexSpace (re)indexes one or all spaces. The response is streamed, sending
|
||||
// progress information after each space has been indexed.
|
||||
IndexSpace(context.Context, *IndexSpaceRequest, SearchProvider_IndexSpaceStream) error
|
||||
}
|
||||
|
||||
func RegisterSearchProviderHandler(s server.Server, hdlr SearchProviderHandler, opts ...server.HandlerOption) error {
|
||||
type searchProvider interface {
|
||||
Search(ctx context.Context, in *SearchRequest, out *SearchResponse) error
|
||||
IndexSpace(ctx context.Context, in *IndexSpaceRequest, out *IndexSpaceResponse) error
|
||||
IndexSpace(ctx context.Context, stream server.Stream) error
|
||||
}
|
||||
type SearchProvider struct {
|
||||
searchProvider
|
||||
@@ -115,6 +165,7 @@ func RegisterSearchProviderHandler(s server.Server, hdlr SearchProviderHandler,
|
||||
Name: "SearchProvider.IndexSpace",
|
||||
Path: []string{"/api/v0/search/index-space"},
|
||||
Method: []string{"POST"},
|
||||
Stream: true,
|
||||
Handler: "rpc",
|
||||
}))
|
||||
return s.Handle(s.NewHandler(&SearchProvider{h}, opts...))
|
||||
@@ -128,8 +179,44 @@ func (h *searchProviderHandler) Search(ctx context.Context, in *SearchRequest, o
|
||||
return h.SearchProviderHandler.Search(ctx, in, out)
|
||||
}
|
||||
|
||||
func (h *searchProviderHandler) IndexSpace(ctx context.Context, in *IndexSpaceRequest, out *IndexSpaceResponse) error {
|
||||
return h.SearchProviderHandler.IndexSpace(ctx, in, out)
|
||||
func (h *searchProviderHandler) IndexSpace(ctx context.Context, stream server.Stream) error {
|
||||
m := new(IndexSpaceRequest)
|
||||
if err := stream.Recv(m); err != nil {
|
||||
return err
|
||||
}
|
||||
return h.SearchProviderHandler.IndexSpace(ctx, m, &searchProviderIndexSpaceStream{stream})
|
||||
}
|
||||
|
||||
type SearchProvider_IndexSpaceStream interface {
|
||||
Context() context.Context
|
||||
SendMsg(interface{}) error
|
||||
RecvMsg(interface{}) error
|
||||
Close() error
|
||||
Send(*IndexSpaceResponse) error
|
||||
}
|
||||
|
||||
type searchProviderIndexSpaceStream struct {
|
||||
stream server.Stream
|
||||
}
|
||||
|
||||
func (x *searchProviderIndexSpaceStream) Close() error {
|
||||
return x.stream.Close()
|
||||
}
|
||||
|
||||
func (x *searchProviderIndexSpaceStream) Context() context.Context {
|
||||
return x.stream.Context()
|
||||
}
|
||||
|
||||
func (x *searchProviderIndexSpaceStream) SendMsg(m interface{}) error {
|
||||
return x.stream.Send(m)
|
||||
}
|
||||
|
||||
func (x *searchProviderIndexSpaceStream) RecvMsg(m interface{}) error {
|
||||
return x.stream.Recv(m)
|
||||
}
|
||||
|
||||
func (x *searchProviderIndexSpaceStream) Send(m *IndexSpaceResponse) error {
|
||||
return x.stream.Send(m)
|
||||
}
|
||||
|
||||
// Api Endpoints for IndexProvider service
|
||||
|
||||
@@ -1,346 +0,0 @@
|
||||
// Code generated by protoc-gen-microweb. DO NOT EDIT.
|
||||
// source: v0.proto
|
||||
|
||||
package v0
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
|
||||
"github.com/go-chi/chi/v5"
|
||||
"github.com/go-chi/render"
|
||||
"github.com/golang/protobuf/jsonpb"
|
||||
merrors "go-micro.dev/v4/errors"
|
||||
)
|
||||
|
||||
type webSearchProviderHandler struct {
|
||||
r chi.Router
|
||||
h SearchProviderHandler
|
||||
}
|
||||
|
||||
func (h *webSearchProviderHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
h.r.ServeHTTP(w, r)
|
||||
}
|
||||
|
||||
func (h *webSearchProviderHandler) Search(w http.ResponseWriter, r *http.Request) {
|
||||
req := &SearchRequest{}
|
||||
resp := &SearchResponse{}
|
||||
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
http.Error(w, err.Error(), http.StatusPreconditionFailed)
|
||||
return
|
||||
}
|
||||
|
||||
if err := h.h.Search(
|
||||
r.Context(),
|
||||
req,
|
||||
resp,
|
||||
); err != nil {
|
||||
if merr, ok := merrors.As(err); ok && merr.Code == http.StatusNotFound {
|
||||
http.Error(w, err.Error(), http.StatusNotFound)
|
||||
} else {
|
||||
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
render.Status(r, http.StatusCreated)
|
||||
render.JSON(w, r, resp)
|
||||
}
|
||||
|
||||
func (h *webSearchProviderHandler) IndexSpace(w http.ResponseWriter, r *http.Request) {
|
||||
req := &IndexSpaceRequest{}
|
||||
resp := &IndexSpaceResponse{}
|
||||
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
http.Error(w, err.Error(), http.StatusPreconditionFailed)
|
||||
return
|
||||
}
|
||||
|
||||
if err := h.h.IndexSpace(
|
||||
r.Context(),
|
||||
req,
|
||||
resp,
|
||||
); err != nil {
|
||||
if merr, ok := merrors.As(err); ok && merr.Code == http.StatusNotFound {
|
||||
http.Error(w, err.Error(), http.StatusNotFound)
|
||||
} else {
|
||||
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
render.Status(r, http.StatusCreated)
|
||||
render.JSON(w, r, resp)
|
||||
}
|
||||
|
||||
func RegisterSearchProviderWeb(r chi.Router, i SearchProviderHandler, middlewares ...func(http.Handler) http.Handler) {
|
||||
handler := &webSearchProviderHandler{
|
||||
r: r,
|
||||
h: i,
|
||||
}
|
||||
|
||||
r.MethodFunc("POST", "/api/v0/search/search", handler.Search)
|
||||
r.MethodFunc("POST", "/api/v0/search/index-space", handler.IndexSpace)
|
||||
}
|
||||
|
||||
type webIndexProviderHandler struct {
|
||||
r chi.Router
|
||||
h IndexProviderHandler
|
||||
}
|
||||
|
||||
func (h *webIndexProviderHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
h.r.ServeHTTP(w, r)
|
||||
}
|
||||
|
||||
func (h *webIndexProviderHandler) Search(w http.ResponseWriter, r *http.Request) {
|
||||
req := &SearchIndexRequest{}
|
||||
resp := &SearchIndexResponse{}
|
||||
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
http.Error(w, err.Error(), http.StatusPreconditionFailed)
|
||||
return
|
||||
}
|
||||
|
||||
if err := h.h.Search(
|
||||
r.Context(),
|
||||
req,
|
||||
resp,
|
||||
); err != nil {
|
||||
if merr, ok := merrors.As(err); ok && merr.Code == http.StatusNotFound {
|
||||
http.Error(w, err.Error(), http.StatusNotFound)
|
||||
} else {
|
||||
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
render.Status(r, http.StatusCreated)
|
||||
render.JSON(w, r, resp)
|
||||
}
|
||||
|
||||
func RegisterIndexProviderWeb(r chi.Router, i IndexProviderHandler, middlewares ...func(http.Handler) http.Handler) {
|
||||
handler := &webIndexProviderHandler{
|
||||
r: r,
|
||||
h: i,
|
||||
}
|
||||
|
||||
r.MethodFunc("POST", "/api/v0/search/index/search", handler.Search)
|
||||
}
|
||||
|
||||
// SearchRequestJSONMarshaler describes the default jsonpb.Marshaler used by all
|
||||
// instances of SearchRequest. This struct is safe to replace or modify but
|
||||
// should not be done so concurrently.
|
||||
var SearchRequestJSONMarshaler = new(jsonpb.Marshaler)
|
||||
|
||||
// MarshalJSON satisfies the encoding/json Marshaler interface. This method
|
||||
// uses the more correct jsonpb package to correctly marshal the message.
|
||||
func (m *SearchRequest) MarshalJSON() ([]byte, error) {
|
||||
if m == nil {
|
||||
return json.Marshal(nil)
|
||||
}
|
||||
|
||||
buf := &bytes.Buffer{}
|
||||
|
||||
if err := SearchRequestJSONMarshaler.Marshal(buf, m); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return buf.Bytes(), nil
|
||||
}
|
||||
|
||||
var _ json.Marshaler = (*SearchRequest)(nil)
|
||||
|
||||
// SearchRequestJSONUnmarshaler describes the default jsonpb.Unmarshaler used by all
|
||||
// instances of SearchRequest. This struct is safe to replace or modify but
|
||||
// should not be done so concurrently.
|
||||
var SearchRequestJSONUnmarshaler = new(jsonpb.Unmarshaler)
|
||||
|
||||
// UnmarshalJSON satisfies the encoding/json Unmarshaler interface. This method
|
||||
// uses the more correct jsonpb package to correctly unmarshal the message.
|
||||
func (m *SearchRequest) UnmarshalJSON(b []byte) error {
|
||||
return SearchRequestJSONUnmarshaler.Unmarshal(bytes.NewReader(b), m)
|
||||
}
|
||||
|
||||
var _ json.Unmarshaler = (*SearchRequest)(nil)
|
||||
|
||||
// SearchResponseJSONMarshaler describes the default jsonpb.Marshaler used by all
|
||||
// instances of SearchResponse. This struct is safe to replace or modify but
|
||||
// should not be done so concurrently.
|
||||
var SearchResponseJSONMarshaler = new(jsonpb.Marshaler)
|
||||
|
||||
// MarshalJSON satisfies the encoding/json Marshaler interface. This method
|
||||
// uses the more correct jsonpb package to correctly marshal the message.
|
||||
func (m *SearchResponse) MarshalJSON() ([]byte, error) {
|
||||
if m == nil {
|
||||
return json.Marshal(nil)
|
||||
}
|
||||
|
||||
buf := &bytes.Buffer{}
|
||||
|
||||
if err := SearchResponseJSONMarshaler.Marshal(buf, m); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return buf.Bytes(), nil
|
||||
}
|
||||
|
||||
var _ json.Marshaler = (*SearchResponse)(nil)
|
||||
|
||||
// SearchResponseJSONUnmarshaler describes the default jsonpb.Unmarshaler used by all
|
||||
// instances of SearchResponse. This struct is safe to replace or modify but
|
||||
// should not be done so concurrently.
|
||||
var SearchResponseJSONUnmarshaler = new(jsonpb.Unmarshaler)
|
||||
|
||||
// UnmarshalJSON satisfies the encoding/json Unmarshaler interface. This method
|
||||
// uses the more correct jsonpb package to correctly unmarshal the message.
|
||||
func (m *SearchResponse) UnmarshalJSON(b []byte) error {
|
||||
return SearchResponseJSONUnmarshaler.Unmarshal(bytes.NewReader(b), m)
|
||||
}
|
||||
|
||||
var _ json.Unmarshaler = (*SearchResponse)(nil)
|
||||
|
||||
// SearchIndexRequestJSONMarshaler describes the default jsonpb.Marshaler used by all
|
||||
// instances of SearchIndexRequest. This struct is safe to replace or modify but
|
||||
// should not be done so concurrently.
|
||||
var SearchIndexRequestJSONMarshaler = new(jsonpb.Marshaler)
|
||||
|
||||
// MarshalJSON satisfies the encoding/json Marshaler interface. This method
|
||||
// uses the more correct jsonpb package to correctly marshal the message.
|
||||
func (m *SearchIndexRequest) MarshalJSON() ([]byte, error) {
|
||||
if m == nil {
|
||||
return json.Marshal(nil)
|
||||
}
|
||||
|
||||
buf := &bytes.Buffer{}
|
||||
|
||||
if err := SearchIndexRequestJSONMarshaler.Marshal(buf, m); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return buf.Bytes(), nil
|
||||
}
|
||||
|
||||
var _ json.Marshaler = (*SearchIndexRequest)(nil)
|
||||
|
||||
// SearchIndexRequestJSONUnmarshaler describes the default jsonpb.Unmarshaler used by all
|
||||
// instances of SearchIndexRequest. This struct is safe to replace or modify but
|
||||
// should not be done so concurrently.
|
||||
var SearchIndexRequestJSONUnmarshaler = new(jsonpb.Unmarshaler)
|
||||
|
||||
// UnmarshalJSON satisfies the encoding/json Unmarshaler interface. This method
|
||||
// uses the more correct jsonpb package to correctly unmarshal the message.
|
||||
func (m *SearchIndexRequest) UnmarshalJSON(b []byte) error {
|
||||
return SearchIndexRequestJSONUnmarshaler.Unmarshal(bytes.NewReader(b), m)
|
||||
}
|
||||
|
||||
var _ json.Unmarshaler = (*SearchIndexRequest)(nil)
|
||||
|
||||
// SearchIndexResponseJSONMarshaler describes the default jsonpb.Marshaler used by all
|
||||
// instances of SearchIndexResponse. This struct is safe to replace or modify but
|
||||
// should not be done so concurrently.
|
||||
var SearchIndexResponseJSONMarshaler = new(jsonpb.Marshaler)
|
||||
|
||||
// MarshalJSON satisfies the encoding/json Marshaler interface. This method
|
||||
// uses the more correct jsonpb package to correctly marshal the message.
|
||||
func (m *SearchIndexResponse) MarshalJSON() ([]byte, error) {
|
||||
if m == nil {
|
||||
return json.Marshal(nil)
|
||||
}
|
||||
|
||||
buf := &bytes.Buffer{}
|
||||
|
||||
if err := SearchIndexResponseJSONMarshaler.Marshal(buf, m); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return buf.Bytes(), nil
|
||||
}
|
||||
|
||||
var _ json.Marshaler = (*SearchIndexResponse)(nil)
|
||||
|
||||
// SearchIndexResponseJSONUnmarshaler describes the default jsonpb.Unmarshaler used by all
|
||||
// instances of SearchIndexResponse. This struct is safe to replace or modify but
|
||||
// should not be done so concurrently.
|
||||
var SearchIndexResponseJSONUnmarshaler = new(jsonpb.Unmarshaler)
|
||||
|
||||
// UnmarshalJSON satisfies the encoding/json Unmarshaler interface. This method
|
||||
// uses the more correct jsonpb package to correctly unmarshal the message.
|
||||
func (m *SearchIndexResponse) UnmarshalJSON(b []byte) error {
|
||||
return SearchIndexResponseJSONUnmarshaler.Unmarshal(bytes.NewReader(b), m)
|
||||
}
|
||||
|
||||
var _ json.Unmarshaler = (*SearchIndexResponse)(nil)
|
||||
|
||||
// IndexSpaceRequestJSONMarshaler describes the default jsonpb.Marshaler used by all
|
||||
// instances of IndexSpaceRequest. This struct is safe to replace or modify but
|
||||
// should not be done so concurrently.
|
||||
var IndexSpaceRequestJSONMarshaler = new(jsonpb.Marshaler)
|
||||
|
||||
// MarshalJSON satisfies the encoding/json Marshaler interface. This method
|
||||
// uses the more correct jsonpb package to correctly marshal the message.
|
||||
func (m *IndexSpaceRequest) MarshalJSON() ([]byte, error) {
|
||||
if m == nil {
|
||||
return json.Marshal(nil)
|
||||
}
|
||||
|
||||
buf := &bytes.Buffer{}
|
||||
|
||||
if err := IndexSpaceRequestJSONMarshaler.Marshal(buf, m); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return buf.Bytes(), nil
|
||||
}
|
||||
|
||||
var _ json.Marshaler = (*IndexSpaceRequest)(nil)
|
||||
|
||||
// IndexSpaceRequestJSONUnmarshaler describes the default jsonpb.Unmarshaler used by all
|
||||
// instances of IndexSpaceRequest. This struct is safe to replace or modify but
|
||||
// should not be done so concurrently.
|
||||
var IndexSpaceRequestJSONUnmarshaler = new(jsonpb.Unmarshaler)
|
||||
|
||||
// UnmarshalJSON satisfies the encoding/json Unmarshaler interface. This method
|
||||
// uses the more correct jsonpb package to correctly unmarshal the message.
|
||||
func (m *IndexSpaceRequest) UnmarshalJSON(b []byte) error {
|
||||
return IndexSpaceRequestJSONUnmarshaler.Unmarshal(bytes.NewReader(b), m)
|
||||
}
|
||||
|
||||
var _ json.Unmarshaler = (*IndexSpaceRequest)(nil)
|
||||
|
||||
// IndexSpaceResponseJSONMarshaler describes the default jsonpb.Marshaler used by all
|
||||
// instances of IndexSpaceResponse. This struct is safe to replace or modify but
|
||||
// should not be done so concurrently.
|
||||
var IndexSpaceResponseJSONMarshaler = new(jsonpb.Marshaler)
|
||||
|
||||
// MarshalJSON satisfies the encoding/json Marshaler interface. This method
|
||||
// uses the more correct jsonpb package to correctly marshal the message.
|
||||
func (m *IndexSpaceResponse) MarshalJSON() ([]byte, error) {
|
||||
if m == nil {
|
||||
return json.Marshal(nil)
|
||||
}
|
||||
|
||||
buf := &bytes.Buffer{}
|
||||
|
||||
if err := IndexSpaceResponseJSONMarshaler.Marshal(buf, m); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return buf.Bytes(), nil
|
||||
}
|
||||
|
||||
var _ json.Marshaler = (*IndexSpaceResponse)(nil)
|
||||
|
||||
// IndexSpaceResponseJSONUnmarshaler describes the default jsonpb.Unmarshaler used by all
|
||||
// instances of IndexSpaceResponse. This struct is safe to replace or modify but
|
||||
// should not be done so concurrently.
|
||||
var IndexSpaceResponseJSONUnmarshaler = new(jsonpb.Unmarshaler)
|
||||
|
||||
// UnmarshalJSON satisfies the encoding/json Unmarshaler interface. This method
|
||||
// uses the more correct jsonpb package to correctly unmarshal the message.
|
||||
func (m *IndexSpaceResponse) UnmarshalJSON(b []byte) error {
|
||||
return IndexSpaceResponseJSONUnmarshaler.Unmarshal(bytes.NewReader(b), m)
|
||||
}
|
||||
|
||||
var _ json.Unmarshaler = (*IndexSpaceResponse)(nil)
|
||||
@@ -34,12 +34,22 @@
|
||||
"paths": {
|
||||
"/api/v0/search/index-space": {
|
||||
"post": {
|
||||
"summary": "IndexSpace (re)indexes one or all spaces. The response is streamed, sending\nprogress information after each space has been indexed.",
|
||||
"operationId": "SearchProvider_IndexSpace",
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "A successful response.",
|
||||
"description": "A successful response.(streaming responses)",
|
||||
"schema": {
|
||||
"$ref": "#/definitions/v0IndexSpaceResponse"
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"result": {
|
||||
"$ref": "#/definitions/v0IndexSpaceResponse"
|
||||
},
|
||||
"error": {
|
||||
"$ref": "#/definitions/rpcStatus"
|
||||
}
|
||||
},
|
||||
"title": "Stream result of v0IndexSpaceResponse"
|
||||
}
|
||||
},
|
||||
"default": {
|
||||
@@ -332,11 +342,39 @@
|
||||
},
|
||||
"forceReindex": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"concurrency": {
|
||||
"type": "integer",
|
||||
"format": "int32"
|
||||
}
|
||||
}
|
||||
},
|
||||
"v0IndexSpaceResponse": {
|
||||
"type": "object"
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"spaceId": {
|
||||
"type": "string",
|
||||
"description": "The id of the space that has just been indexed."
|
||||
},
|
||||
"spaceDuration": {
|
||||
"type": "string",
|
||||
"description": "The duration it took to index this space."
|
||||
},
|
||||
"indexedSpaces": {
|
||||
"type": "string",
|
||||
"format": "int64",
|
||||
"description": "The number of spaces that have been indexed so far."
|
||||
},
|
||||
"totalSpaces": {
|
||||
"type": "string",
|
||||
"format": "int64",
|
||||
"description": "The total number of spaces that are being indexed."
|
||||
},
|
||||
"error": {
|
||||
"type": "string",
|
||||
"description": "Contains an error message in case indexing this particular space failed."
|
||||
}
|
||||
}
|
||||
},
|
||||
"v0Match": {
|
||||
"type": "object",
|
||||
|
||||
@@ -28,7 +28,9 @@ const (
|
||||
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
|
||||
type SearchProviderClient interface {
|
||||
Search(ctx context.Context, in *SearchRequest, opts ...grpc.CallOption) (*SearchResponse, error)
|
||||
IndexSpace(ctx context.Context, in *IndexSpaceRequest, opts ...grpc.CallOption) (*IndexSpaceResponse, error)
|
||||
// IndexSpace (re)indexes one or all spaces. The response is streamed, sending
|
||||
// progress information after each space has been indexed.
|
||||
IndexSpace(ctx context.Context, in *IndexSpaceRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[IndexSpaceResponse], error)
|
||||
}
|
||||
|
||||
type searchProviderClient struct {
|
||||
@@ -49,22 +51,33 @@ func (c *searchProviderClient) Search(ctx context.Context, in *SearchRequest, op
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *searchProviderClient) IndexSpace(ctx context.Context, in *IndexSpaceRequest, opts ...grpc.CallOption) (*IndexSpaceResponse, error) {
|
||||
func (c *searchProviderClient) IndexSpace(ctx context.Context, in *IndexSpaceRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[IndexSpaceResponse], error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(IndexSpaceResponse)
|
||||
err := c.cc.Invoke(ctx, SearchProvider_IndexSpace_FullMethodName, in, out, cOpts...)
|
||||
stream, err := c.cc.NewStream(ctx, &SearchProvider_ServiceDesc.Streams[0], SearchProvider_IndexSpace_FullMethodName, cOpts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
x := &grpc.GenericClientStream[IndexSpaceRequest, IndexSpaceResponse]{ClientStream: stream}
|
||||
if err := x.ClientStream.SendMsg(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := x.ClientStream.CloseSend(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return x, nil
|
||||
}
|
||||
|
||||
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
|
||||
type SearchProvider_IndexSpaceClient = grpc.ServerStreamingClient[IndexSpaceResponse]
|
||||
|
||||
// SearchProviderServer is the server API for SearchProvider service.
|
||||
// All implementations must embed UnimplementedSearchProviderServer
|
||||
// for forward compatibility.
|
||||
type SearchProviderServer interface {
|
||||
Search(context.Context, *SearchRequest) (*SearchResponse, error)
|
||||
IndexSpace(context.Context, *IndexSpaceRequest) (*IndexSpaceResponse, error)
|
||||
// IndexSpace (re)indexes one or all spaces. The response is streamed, sending
|
||||
// progress information after each space has been indexed.
|
||||
IndexSpace(*IndexSpaceRequest, grpc.ServerStreamingServer[IndexSpaceResponse]) error
|
||||
mustEmbedUnimplementedSearchProviderServer()
|
||||
}
|
||||
|
||||
@@ -78,8 +91,8 @@ type UnimplementedSearchProviderServer struct{}
|
||||
func (UnimplementedSearchProviderServer) Search(context.Context, *SearchRequest) (*SearchResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method Search not implemented")
|
||||
}
|
||||
func (UnimplementedSearchProviderServer) IndexSpace(context.Context, *IndexSpaceRequest) (*IndexSpaceResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method IndexSpace not implemented")
|
||||
func (UnimplementedSearchProviderServer) IndexSpace(*IndexSpaceRequest, grpc.ServerStreamingServer[IndexSpaceResponse]) error {
|
||||
return status.Error(codes.Unimplemented, "method IndexSpace not implemented")
|
||||
}
|
||||
func (UnimplementedSearchProviderServer) mustEmbedUnimplementedSearchProviderServer() {}
|
||||
func (UnimplementedSearchProviderServer) testEmbeddedByValue() {}
|
||||
@@ -120,24 +133,17 @@ func _SearchProvider_Search_Handler(srv interface{}, ctx context.Context, dec fu
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _SearchProvider_IndexSpace_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(IndexSpaceRequest)
|
||||
if err := dec(in); err != nil {
|
||||
return nil, err
|
||||
func _SearchProvider_IndexSpace_Handler(srv interface{}, stream grpc.ServerStream) error {
|
||||
m := new(IndexSpaceRequest)
|
||||
if err := stream.RecvMsg(m); err != nil {
|
||||
return err
|
||||
}
|
||||
if interceptor == nil {
|
||||
return srv.(SearchProviderServer).IndexSpace(ctx, in)
|
||||
}
|
||||
info := &grpc.UnaryServerInfo{
|
||||
Server: srv,
|
||||
FullMethod: SearchProvider_IndexSpace_FullMethodName,
|
||||
}
|
||||
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
|
||||
return srv.(SearchProviderServer).IndexSpace(ctx, req.(*IndexSpaceRequest))
|
||||
}
|
||||
return interceptor(ctx, in, info, handler)
|
||||
return srv.(SearchProviderServer).IndexSpace(m, &grpc.GenericServerStream[IndexSpaceRequest, IndexSpaceResponse]{ServerStream: stream})
|
||||
}
|
||||
|
||||
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
|
||||
type SearchProvider_IndexSpaceServer = grpc.ServerStreamingServer[IndexSpaceResponse]
|
||||
|
||||
// SearchProvider_ServiceDesc is the grpc.ServiceDesc for SearchProvider service.
|
||||
// It's only intended for direct use with grpc.RegisterService,
|
||||
// and not to be introspected or modified (even as a copy)
|
||||
@@ -149,12 +155,14 @@ var SearchProvider_ServiceDesc = grpc.ServiceDesc{
|
||||
MethodName: "Search",
|
||||
Handler: _SearchProvider_Search_Handler,
|
||||
},
|
||||
},
|
||||
Streams: []grpc.StreamDesc{
|
||||
{
|
||||
MethodName: "IndexSpace",
|
||||
Handler: _SearchProvider_IndexSpace_Handler,
|
||||
StreamName: "IndexSpace",
|
||||
Handler: _SearchProvider_IndexSpace_Handler,
|
||||
ServerStreams: true,
|
||||
},
|
||||
},
|
||||
Streams: []grpc.StreamDesc{},
|
||||
Metadata: "opencloud/services/search/v0/search.proto",
|
||||
}
|
||||
|
||||
|
||||
@@ -31,7 +31,9 @@ plugins:
|
||||
opencloud.services.eventhistory.v0;\
|
||||
opencloud.messages.eventhistory.v0;\
|
||||
opencloud.services.policies.v0;\
|
||||
opencloud.messages.policies.v0"
|
||||
opencloud.messages.policies.v0;\
|
||||
opencloud.services.search.v0;\
|
||||
opencloud.messages.search.v0"
|
||||
|
||||
- name: openapiv2
|
||||
path: ../../.bingo/protoc-gen-openapiv2
|
||||
|
||||
@@ -9,6 +9,7 @@ import "protoc-gen-openapiv2/options/annotations.proto";
|
||||
import "google/api/field_behavior.proto";
|
||||
import "google/api/annotations.proto";
|
||||
import "google/protobuf/field_mask.proto";
|
||||
import "google/protobuf/duration.proto";
|
||||
|
||||
option (grpc.gateway.protoc_gen_openapiv2.options.openapiv2_swagger) = {
|
||||
info: {
|
||||
@@ -41,7 +42,9 @@ service SearchProvider {
|
||||
body: "*"
|
||||
};
|
||||
};
|
||||
rpc IndexSpace(IndexSpaceRequest) returns (IndexSpaceResponse) {
|
||||
// IndexSpace (re)indexes one or all spaces. The response is streamed, sending
|
||||
// progress information after each space has been indexed.
|
||||
rpc IndexSpace(IndexSpaceRequest) returns (stream IndexSpaceResponse) {
|
||||
option (google.api.http) = {
|
||||
post: "/api/v0/search/index-space",
|
||||
body: "*"
|
||||
@@ -105,7 +108,18 @@ message IndexSpaceRequest {
|
||||
string space_id = 1;
|
||||
string user_id = 2;
|
||||
bool force_reindex = 3;
|
||||
int32 concurrency = 4 [(google.api.field_behavior) = OPTIONAL];
|
||||
}
|
||||
|
||||
message IndexSpaceResponse {
|
||||
// The id of the space that has just been indexed.
|
||||
string space_id = 1;
|
||||
// The duration it took to index this space.
|
||||
google.protobuf.Duration space_duration = 2;
|
||||
// The number of spaces that have been indexed so far.
|
||||
int64 indexed_spaces = 3;
|
||||
// The total number of spaces that are being indexed.
|
||||
int64 total_spaces = 4;
|
||||
// Contains an error message in case indexing this particular space failed.
|
||||
string error = 5;
|
||||
}
|
||||
+10
-10
@@ -30,19 +30,19 @@
|
||||
},
|
||||
"dependencies": {
|
||||
"@material-ui/core": "^4.12.4",
|
||||
"@types/node": "^25.7.0",
|
||||
"@types/react": "^17.0.91",
|
||||
"@types/node": "^25.9.5",
|
||||
"@types/react": "^17.0.93",
|
||||
"@types/react-dom": "^17.0.26",
|
||||
"@types/react-redux": "^7.1.34",
|
||||
"@types/redux-logger": "^3.0.13",
|
||||
"axios": "^1.18.1",
|
||||
"i18next": "^26.3.0",
|
||||
"axios": "^1.19.0",
|
||||
"i18next": "^26.3.6",
|
||||
"i18next-browser-languagedetector": "^8.2.1",
|
||||
"i18next-resources-to-backend": "^1.2.1",
|
||||
"query-string": "^9.3.1",
|
||||
"i18next-resources-to-backend": "^1.2.3",
|
||||
"query-string": "^9.4.1",
|
||||
"react": "^17.0.2",
|
||||
"react-dom": "^17.0.2",
|
||||
"react-i18next": "^17.0.8",
|
||||
"react-i18next": "^17.0.11",
|
||||
"react-redux": "^8.1.3",
|
||||
"react-router": "^5.3.4",
|
||||
"react-router-dom": "5.2.1",
|
||||
@@ -61,9 +61,9 @@
|
||||
"css-minimizer-webpack-plugin": "^8.0.0",
|
||||
"dotenv": "17.4.2",
|
||||
"dotenv-expand": "^13.0.0",
|
||||
"gettext-parser": "^9.0.2",
|
||||
"html-webpack-plugin": "^5.6.7",
|
||||
"i18next-cli": "^1.65.0",
|
||||
"gettext-parser": "^9.1.1",
|
||||
"html-webpack-plugin": "^5.6.8",
|
||||
"i18next-cli": "^1.67.3",
|
||||
"i18next-conv": "^17.0.0",
|
||||
"license-checker-rseidelsohn": "5.0.1",
|
||||
"mini-css-extract-plugin": "2.9.2",
|
||||
|
||||
Generated
+764
-744
File diff suppressed because it is too large.
Load diff
@@ -20,11 +20,14 @@ supportedArchitectures:
|
||||
- musl
|
||||
|
||||
overrides:
|
||||
fast-uri: ">=3.1.2"
|
||||
undici: ">=7.28.0"
|
||||
postcss: ">=8.5.10"
|
||||
brace-expansion: ">=5.0.7"
|
||||
fast-uri: ">=3.1.4"
|
||||
js-yaml: ">=4.3.0"
|
||||
postcss: ">=8.5.18"
|
||||
"serialize-javascript@<7.0.3": ">=7.0.3"
|
||||
shell-quote: ">=1.8.4"
|
||||
shell-quote: ">=1.9.0"
|
||||
svgo: ">=4.0.2"
|
||||
undici: ">=7.28.0"
|
||||
"@babel/plugin-transform-modules-systemjs": ">=7.29.4"
|
||||
"@xmldom/xmldom": "^0.8.13"
|
||||
|
||||
|
||||
@@ -137,6 +137,10 @@ func DefaultPolicies() []config.Policy {
|
||||
Endpoint: "/branding/logo",
|
||||
Service: "eu.opencloud.web.web",
|
||||
},
|
||||
{
|
||||
Endpoint: "/announcement",
|
||||
Service: "eu.opencloud.web.web",
|
||||
},
|
||||
{
|
||||
Endpoint: "/konnect/",
|
||||
Service: "eu.opencloud.web.idp",
|
||||
|
||||
@@ -5,6 +5,10 @@ import (
|
||||
"crypto/tls"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/opencloud-eu/opencloud/pkg/config/configlog"
|
||||
@@ -33,9 +37,14 @@ func Index(cfg *config.Config) *cobra.Command {
|
||||
forceRescanFlag, _ := cmd.Flags().GetBool("force-rescan")
|
||||
endpointFlag, _ := cmd.Flags().GetString("endpoint")
|
||||
insecureFlag, _ := cmd.Flags().GetBool("insecure")
|
||||
concurrencyFlag, _ := cmd.Flags().GetInt("concurrency")
|
||||
|
||||
if spaceFlag == "" && !allSpacesFlag {
|
||||
return errors.New("either --space or --all-spaces is required")
|
||||
}
|
||||
if int(concurrencyFlag) > cfg.ReindexMaxConcurrency {
|
||||
return fmt.Errorf("concurrency %d exceeds max allowed %d", concurrencyFlag, cfg.ReindexMaxConcurrency)
|
||||
}
|
||||
|
||||
var dialOpts []grpc.DialOption
|
||||
if cfg.GRPCClientTLS.Mode == "insecure" || insecureFlag {
|
||||
@@ -54,17 +63,47 @@ func Index(cfg *config.Config) *cobra.Command {
|
||||
|
||||
c := searchsvc.NewSearchProviderClient(conn)
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
|
||||
// Cancel the operation when the user presses Ctrl+C (SIGINT) or the
|
||||
// process receives SIGTERM. The cancellation propagates over the
|
||||
// gRPC stream so the server stops indexing.
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
ctx, cancel := context.WithTimeout(ctx, 10*time.Minute)
|
||||
defer cancel()
|
||||
|
||||
_, err = c.IndexSpace(ctx, &searchsvc.IndexSpaceRequest{
|
||||
stream, err := c.IndexSpace(ctx, &searchsvc.IndexSpaceRequest{
|
||||
SpaceId: spaceFlag,
|
||||
ForceReindex: forceRescanFlag,
|
||||
Concurrency: int32(concurrencyFlag),
|
||||
})
|
||||
if err != nil {
|
||||
fmt.Println("failed to index space: " + err.Error())
|
||||
return err
|
||||
}
|
||||
|
||||
for {
|
||||
progress, err := stream.Recv()
|
||||
if errors.Is(err, io.EOF) {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
// The user aborted (Ctrl+C / SIGTERM). Exit quietly instead
|
||||
// of dumping a "context canceled" gRPC error.
|
||||
if errors.Is(ctx.Err(), context.Canceled) {
|
||||
fmt.Println("aborted, indexing has been stopped")
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
if progress.GetError() != "" {
|
||||
fmt.Printf("[%d/%d] failed to index space %s: %s\n",
|
||||
progress.GetIndexedSpaces(), progress.GetTotalSpaces(), progress.GetSpaceId(), progress.GetError())
|
||||
continue
|
||||
}
|
||||
|
||||
fmt.Printf("[%d/%d] indexed space %s in %s\n",
|
||||
progress.GetIndexedSpaces(), progress.GetTotalSpaces(), progress.GetSpaceId(), progress.GetSpaceDuration().AsDuration())
|
||||
}
|
||||
return nil
|
||||
},
|
||||
}
|
||||
@@ -94,6 +133,11 @@ func Index(cfg *config.Config) *cobra.Command {
|
||||
false,
|
||||
"disable TLS for the gRPC connection.",
|
||||
)
|
||||
indexCmd.Flags().Int(
|
||||
"concurrency",
|
||||
3,
|
||||
"the number of concurrent indexing operations.",
|
||||
)
|
||||
|
||||
return indexCmd
|
||||
}
|
||||
@@ -28,6 +28,7 @@ type Config struct {
|
||||
Extractor Extractor `yaml:"extractor"`
|
||||
ContentExtractionSizeLimit uint64 `yaml:"content_extraction_size_limit" env:"SEARCH_CONTENT_EXTRACTION_SIZE_LIMIT" desc:"Maximum file size in bytes that is allowed for content extraction." introductionVersion:"1.0.0"`
|
||||
BatchSize int `yaml:"batch_size" env:"SEARCH_BATCH_SIZE" desc:"The number of documents to process in a single batch. Defaults to 500." introductionVersion:"1.0.0"`
|
||||
ReindexMaxConcurrency int `yaml:"reindex_concurrency" env:"SEARCH_REINDEX_MAX_CONCURRENCY" desc:"The maximum number of spaces that are reindexed concurrently when reindexing all spaces." introductionVersion:"7.4.0"`
|
||||
|
||||
ServiceAccount ServiceAccount `yaml:"service_account"`
|
||||
|
||||
|
||||
@@ -65,6 +65,7 @@ func DefaultConfig() *config.Config {
|
||||
},
|
||||
ContentExtractionSizeLimit: 20 * 1024 * 1024, // Limit content extraction to <20MB files by default
|
||||
BatchSize: 50,
|
||||
ReindexMaxConcurrency: 3,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1"
|
||||
@@ -19,7 +20,9 @@ import (
|
||||
"github.com/opencloud-eu/reva/v2/pkg/utils"
|
||||
merrors "go-micro.dev/v4/errors"
|
||||
"go-micro.dev/v4/metadata"
|
||||
"golang.org/x/sync/errgroup"
|
||||
grpcmetadata "google.golang.org/grpc/metadata"
|
||||
"google.golang.org/protobuf/types/known/durationpb"
|
||||
|
||||
"github.com/opencloud-eu/opencloud/pkg/log"
|
||||
v0 "github.com/opencloud-eu/opencloud/protogen/gen/opencloud/messages/search/v0"
|
||||
@@ -118,10 +121,27 @@ func (s Service) Search(ctx context.Context, in *searchsvc.SearchRequest, out *s
|
||||
return nil
|
||||
}
|
||||
|
||||
// IndexSpace (re)indexes all resources of a given space.
|
||||
func (s Service) IndexSpace(_ context.Context, in *searchsvc.IndexSpaceRequest, _ *searchsvc.IndexSpaceResponse) error {
|
||||
// IndexSpace (re)indexes all resources of a given space. Progress information is
|
||||
// streamed back to the caller after every space that has been indexed.
|
||||
func (s Service) IndexSpace(_ context.Context, in *searchsvc.IndexSpaceRequest, stream searchsvc.SearchProvider_IndexSpaceStream) error {
|
||||
// Use the stream's context so that indexing stops when the client cancels
|
||||
// the request or disconnects.
|
||||
ctx := stream.Context()
|
||||
|
||||
if in.GetSpaceId() != "" {
|
||||
return s.searcher.IndexSpace(&provider.StorageSpaceId{OpaqueId: in.GetSpaceId()}, in.GetForceReindex())
|
||||
err := s.searcher.IndexSpace(&provider.StorageSpaceId{OpaqueId: in.GetSpaceId()}, in.GetForceReindex())
|
||||
resp := &searchsvc.IndexSpaceResponse{
|
||||
SpaceId: in.GetSpaceId(),
|
||||
IndexedSpaces: 1,
|
||||
TotalSpaces: 1,
|
||||
}
|
||||
if err != nil {
|
||||
resp.Error = err.Error()
|
||||
}
|
||||
if sendErr := stream.Send(resp); sendErr != nil {
|
||||
return sendErr
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// index all spaces instead
|
||||
@@ -130,7 +150,7 @@ func (s Service) IndexSpace(_ context.Context, in *searchsvc.IndexSpaceRequest,
|
||||
return err
|
||||
}
|
||||
|
||||
ctx, err := utils.GetServiceUserContext(s.cfg.ServiceAccount.ServiceAccountID, gwc, s.cfg.ServiceAccount.ServiceAccountSecret)
|
||||
ctx, err = utils.GetServiceUserContextWithContext(ctx, gwc, s.cfg.ServiceAccount.ServiceAccountID, s.cfg.ServiceAccount.ServiceAccountSecret)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -144,13 +164,65 @@ func (s Service) IndexSpace(_ context.Context, in *searchsvc.IndexSpaceRequest,
|
||||
return errors.New(resp.GetStatus().GetMessage())
|
||||
}
|
||||
|
||||
for _, space := range resp.GetStorageSpaces() {
|
||||
if err := s.searcher.IndexSpace(space.GetId(), in.GetForceReindex()); err != nil {
|
||||
spaces := resp.GetStorageSpaces()
|
||||
totalSpaces := int64(len(spaces))
|
||||
|
||||
// Index all spaces concurrently, limited to a configurable number of spaces
|
||||
// being reindexed at the same time. The errgroup context is cancelled as
|
||||
// soon as the client goes away or a stream send fails, so the remaining
|
||||
// goroutines stop indexing early.
|
||||
concurrency := max(s.cfg.ReindexMaxConcurrency, 1)
|
||||
concurrency = min(concurrency, int(in.GetConcurrency()))
|
||||
|
||||
g, ctx := errgroup.WithContext(ctx)
|
||||
g.SetLimit(concurrency)
|
||||
|
||||
// Serialize progress updates on the stream, as gRPC streams must not be
|
||||
// written to from multiple goroutines concurrently.
|
||||
var (
|
||||
mu sync.Mutex
|
||||
indexedCount int64
|
||||
)
|
||||
|
||||
for _, space := range spaces {
|
||||
// Stop early if the client cancelled the request.
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
g.Go(func() error {
|
||||
s.log.Info().Str("space_id", space.GetId().GetOpaqueId()).Msg("indexing space")
|
||||
t := time.Now()
|
||||
|
||||
indexErr := s.searcher.IndexSpace(space.GetId(), in.GetForceReindex())
|
||||
if indexErr != nil {
|
||||
s.log.Error().Err(indexErr).Str("space_id", space.GetId().GetOpaqueId()).Msg("failed to index space")
|
||||
} else {
|
||||
s.log.Info().Str("space_id", space.GetId().GetOpaqueId()).Msg("finished indexing space")
|
||||
}
|
||||
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
|
||||
// Don't try to send progress on an already cancelled stream.
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
indexedCount++
|
||||
progress := &searchsvc.IndexSpaceResponse{
|
||||
SpaceId: space.GetId().GetOpaqueId(),
|
||||
IndexedSpaces: indexedCount,
|
||||
TotalSpaces: totalSpaces,
|
||||
SpaceDuration: durationpb.New(time.Since(t)),
|
||||
}
|
||||
if indexErr != nil {
|
||||
progress.Error = indexErr.Error()
|
||||
}
|
||||
return stream.Send(progress)
|
||||
})
|
||||
}
|
||||
|
||||
return nil
|
||||
return g.Wait()
|
||||
}
|
||||
|
||||
// FromCache pulls a search result from cache
|
||||
|
||||
@@ -12,7 +12,7 @@ msgid ""
|
||||
msgstr ""
|
||||
"Project-Id-Version: \n"
|
||||
"Report-Msgid-Bugs-To: EMAIL\n"
|
||||
"POT-Creation-Date: 2026-07-09 23:15+0000\n"
|
||||
"POT-Creation-Date: 2026-07-29 23:16+0000\n"
|
||||
"PO-Revision-Date: 2025-01-27 10:17+0000\n"
|
||||
"Last-Translator: Andre Nunes, 2026\n"
|
||||
"Language-Team: Portuguese (https://app.transifex.com/opencloud-eu/teams/204053/pt/)\n"
|
||||
|
||||
@@ -80,6 +80,7 @@ func ServiceAccountBundle() *settingsmsg.Bundle {
|
||||
},
|
||||
Settings: []*settingsmsg.Setting{
|
||||
AccountManagementPermission(All),
|
||||
AnnouncementReadWritePermission(All),
|
||||
ChangeLogoPermission(All),
|
||||
CollaborationPublishNotificationPermission(All),
|
||||
CollaborationManageFontsPermission(All),
|
||||
@@ -117,6 +118,7 @@ func generateBundleAdminRole() *settingsmsg.Bundle {
|
||||
},
|
||||
Settings: []*settingsmsg.Setting{
|
||||
AccountManagementPermission(All),
|
||||
AnnouncementReadWritePermission(All),
|
||||
AutoAcceptSharesPermission(Own),
|
||||
ChangeLogoPermission(All),
|
||||
CollaborationPublishNotificationPermission(All),
|
||||
|
||||
@@ -29,6 +29,25 @@ func AccountManagementPermission(c settingsmsg.Permission_Constraint) *settingsm
|
||||
}
|
||||
}
|
||||
|
||||
// AnnouncementReadWritePermission is the permission to read and manage the web announcement banner
|
||||
func AnnouncementReadWritePermission(c settingsmsg.Permission_Constraint) *settingsmsg.Setting {
|
||||
return &settingsmsg.Setting{
|
||||
Id: "52b1994b-1bdb-4c8d-a887-1967dbe8cb11",
|
||||
Name: "Announcement.ReadWrite",
|
||||
DisplayName: "Manage announcement",
|
||||
Description: "This permission permits to read and manage the announcement banner shown to all users.",
|
||||
Resource: &settingsmsg.Resource{
|
||||
Type: settingsmsg.Resource_TYPE_SYSTEM,
|
||||
},
|
||||
Value: &settingsmsg.Setting_PermissionValue{
|
||||
PermissionValue: &settingsmsg.Permission{
|
||||
Operation: settingsmsg.Permission_OPERATION_READWRITE,
|
||||
Constraint: c,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// AutoAcceptSharesPermission is the permission to enable share auto-accept
|
||||
func AutoAcceptSharesPermission(c settingsmsg.Permission_Constraint) *settingsmsg.Setting {
|
||||
return &settingsmsg.Setting{
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
# maintain v2 separate mocks dir
|
||||
dir: "{{.InterfaceDir}}/mocks"
|
||||
structname: "{{.InterfaceName}}"
|
||||
filename: "{{.InterfaceName | snakecase }}.go"
|
||||
pkgname: mocks
|
||||
|
||||
template: testify
|
||||
packages:
|
||||
github.com/nats-io/nats.go/jetstream:
|
||||
config:
|
||||
dir: mocks
|
||||
interfaces:
|
||||
KeyValue: {}
|
||||
KeyValueEntry: {}
|
||||
@@ -1,6 +1,6 @@
|
||||
SHELL := bash
|
||||
NAME := web
|
||||
WEB_ASSETS_VERSION = v7.2.0
|
||||
WEB_ASSETS_VERSION = v7.3.0
|
||||
WEB_ASSETS_BRANCH = main
|
||||
|
||||
ifneq (, $(shell command -v go 2> /dev/null)) # suppress `command not found warnings` for non go targets in CI
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,349 @@
|
||||
// Code generated by mockery; DO NOT EDIT.
|
||||
// github.com/vektra/mockery
|
||||
// template: testify
|
||||
|
||||
package mocks
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
mock "github.com/stretchr/testify/mock"
|
||||
)
|
||||
|
||||
// NewKeyValueEntry creates a new instance of KeyValueEntry. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
|
||||
// The first argument is typically a *testing.T value.
|
||||
func NewKeyValueEntry(t interface {
|
||||
mock.TestingT
|
||||
Cleanup(func())
|
||||
}) *KeyValueEntry {
|
||||
mock := &KeyValueEntry{}
|
||||
mock.Mock.Test(t)
|
||||
|
||||
t.Cleanup(func() { mock.AssertExpectations(t) })
|
||||
|
||||
return mock
|
||||
}
|
||||
|
||||
// KeyValueEntry is an autogenerated mock type for the KeyValueEntry type
|
||||
type KeyValueEntry struct {
|
||||
mock.Mock
|
||||
}
|
||||
|
||||
type KeyValueEntry_Expecter struct {
|
||||
mock *mock.Mock
|
||||
}
|
||||
|
||||
func (_m *KeyValueEntry) EXPECT() *KeyValueEntry_Expecter {
|
||||
return &KeyValueEntry_Expecter{mock: &_m.Mock}
|
||||
}
|
||||
|
||||
// Bucket provides a mock function for the type KeyValueEntry
|
||||
func (_mock *KeyValueEntry) Bucket() string {
|
||||
ret := _mock.Called()
|
||||
|
||||
if len(ret) == 0 {
|
||||
panic("no return value specified for Bucket")
|
||||
}
|
||||
|
||||
var r0 string
|
||||
if returnFunc, ok := ret.Get(0).(func() string); ok {
|
||||
r0 = returnFunc()
|
||||
} else {
|
||||
r0 = ret.Get(0).(string)
|
||||
}
|
||||
return r0
|
||||
}
|
||||
|
||||
// KeyValueEntry_Bucket_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Bucket'
|
||||
type KeyValueEntry_Bucket_Call struct {
|
||||
*mock.Call
|
||||
}
|
||||
|
||||
// Bucket is a helper method to define mock.On call
|
||||
func (_e *KeyValueEntry_Expecter) Bucket() *KeyValueEntry_Bucket_Call {
|
||||
return &KeyValueEntry_Bucket_Call{Call: _e.mock.On("Bucket")}
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Bucket_Call) Run(run func()) *KeyValueEntry_Bucket_Call {
|
||||
_c.Call.Run(func(args mock.Arguments) {
|
||||
run()
|
||||
})
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Bucket_Call) Return(s string) *KeyValueEntry_Bucket_Call {
|
||||
_c.Call.Return(s)
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Bucket_Call) RunAndReturn(run func() string) *KeyValueEntry_Bucket_Call {
|
||||
_c.Call.Return(run)
|
||||
return _c
|
||||
}
|
||||
|
||||
// Created provides a mock function for the type KeyValueEntry
|
||||
func (_mock *KeyValueEntry) Created() time.Time {
|
||||
ret := _mock.Called()
|
||||
|
||||
if len(ret) == 0 {
|
||||
panic("no return value specified for Created")
|
||||
}
|
||||
|
||||
var r0 time.Time
|
||||
if returnFunc, ok := ret.Get(0).(func() time.Time); ok {
|
||||
r0 = returnFunc()
|
||||
} else {
|
||||
r0 = ret.Get(0).(time.Time)
|
||||
}
|
||||
return r0
|
||||
}
|
||||
|
||||
// KeyValueEntry_Created_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Created'
|
||||
type KeyValueEntry_Created_Call struct {
|
||||
*mock.Call
|
||||
}
|
||||
|
||||
// Created is a helper method to define mock.On call
|
||||
func (_e *KeyValueEntry_Expecter) Created() *KeyValueEntry_Created_Call {
|
||||
return &KeyValueEntry_Created_Call{Call: _e.mock.On("Created")}
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Created_Call) Run(run func()) *KeyValueEntry_Created_Call {
|
||||
_c.Call.Run(func(args mock.Arguments) {
|
||||
run()
|
||||
})
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Created_Call) Return(time1 time.Time) *KeyValueEntry_Created_Call {
|
||||
_c.Call.Return(time1)
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Created_Call) RunAndReturn(run func() time.Time) *KeyValueEntry_Created_Call {
|
||||
_c.Call.Return(run)
|
||||
return _c
|
||||
}
|
||||
|
||||
// Delta provides a mock function for the type KeyValueEntry
|
||||
func (_mock *KeyValueEntry) Delta() uint64 {
|
||||
ret := _mock.Called()
|
||||
|
||||
if len(ret) == 0 {
|
||||
panic("no return value specified for Delta")
|
||||
}
|
||||
|
||||
var r0 uint64
|
||||
if returnFunc, ok := ret.Get(0).(func() uint64); ok {
|
||||
r0 = returnFunc()
|
||||
} else {
|
||||
r0 = ret.Get(0).(uint64)
|
||||
}
|
||||
return r0
|
||||
}
|
||||
|
||||
// KeyValueEntry_Delta_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Delta'
|
||||
type KeyValueEntry_Delta_Call struct {
|
||||
*mock.Call
|
||||
}
|
||||
|
||||
// Delta is a helper method to define mock.On call
|
||||
func (_e *KeyValueEntry_Expecter) Delta() *KeyValueEntry_Delta_Call {
|
||||
return &KeyValueEntry_Delta_Call{Call: _e.mock.On("Delta")}
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Delta_Call) Run(run func()) *KeyValueEntry_Delta_Call {
|
||||
_c.Call.Run(func(args mock.Arguments) {
|
||||
run()
|
||||
})
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Delta_Call) Return(v uint64) *KeyValueEntry_Delta_Call {
|
||||
_c.Call.Return(v)
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Delta_Call) RunAndReturn(run func() uint64) *KeyValueEntry_Delta_Call {
|
||||
_c.Call.Return(run)
|
||||
return _c
|
||||
}
|
||||
|
||||
// Key provides a mock function for the type KeyValueEntry
|
||||
func (_mock *KeyValueEntry) Key() string {
|
||||
ret := _mock.Called()
|
||||
|
||||
if len(ret) == 0 {
|
||||
panic("no return value specified for Key")
|
||||
}
|
||||
|
||||
var r0 string
|
||||
if returnFunc, ok := ret.Get(0).(func() string); ok {
|
||||
r0 = returnFunc()
|
||||
} else {
|
||||
r0 = ret.Get(0).(string)
|
||||
}
|
||||
return r0
|
||||
}
|
||||
|
||||
// KeyValueEntry_Key_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Key'
|
||||
type KeyValueEntry_Key_Call struct {
|
||||
*mock.Call
|
||||
}
|
||||
|
||||
// Key is a helper method to define mock.On call
|
||||
func (_e *KeyValueEntry_Expecter) Key() *KeyValueEntry_Key_Call {
|
||||
return &KeyValueEntry_Key_Call{Call: _e.mock.On("Key")}
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Key_Call) Run(run func()) *KeyValueEntry_Key_Call {
|
||||
_c.Call.Run(func(args mock.Arguments) {
|
||||
run()
|
||||
})
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Key_Call) Return(s string) *KeyValueEntry_Key_Call {
|
||||
_c.Call.Return(s)
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Key_Call) RunAndReturn(run func() string) *KeyValueEntry_Key_Call {
|
||||
_c.Call.Return(run)
|
||||
return _c
|
||||
}
|
||||
|
||||
// Operation provides a mock function for the type KeyValueEntry
|
||||
func (_mock *KeyValueEntry) Operation() jetstream.KeyValueOp {
|
||||
ret := _mock.Called()
|
||||
|
||||
if len(ret) == 0 {
|
||||
panic("no return value specified for Operation")
|
||||
}
|
||||
|
||||
var r0 jetstream.KeyValueOp
|
||||
if returnFunc, ok := ret.Get(0).(func() jetstream.KeyValueOp); ok {
|
||||
r0 = returnFunc()
|
||||
} else {
|
||||
r0 = ret.Get(0).(jetstream.KeyValueOp)
|
||||
}
|
||||
return r0
|
||||
}
|
||||
|
||||
// KeyValueEntry_Operation_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Operation'
|
||||
type KeyValueEntry_Operation_Call struct {
|
||||
*mock.Call
|
||||
}
|
||||
|
||||
// Operation is a helper method to define mock.On call
|
||||
func (_e *KeyValueEntry_Expecter) Operation() *KeyValueEntry_Operation_Call {
|
||||
return &KeyValueEntry_Operation_Call{Call: _e.mock.On("Operation")}
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Operation_Call) Run(run func()) *KeyValueEntry_Operation_Call {
|
||||
_c.Call.Run(func(args mock.Arguments) {
|
||||
run()
|
||||
})
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Operation_Call) Return(keyValueOp jetstream.KeyValueOp) *KeyValueEntry_Operation_Call {
|
||||
_c.Call.Return(keyValueOp)
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Operation_Call) RunAndReturn(run func() jetstream.KeyValueOp) *KeyValueEntry_Operation_Call {
|
||||
_c.Call.Return(run)
|
||||
return _c
|
||||
}
|
||||
|
||||
// Revision provides a mock function for the type KeyValueEntry
|
||||
func (_mock *KeyValueEntry) Revision() uint64 {
|
||||
ret := _mock.Called()
|
||||
|
||||
if len(ret) == 0 {
|
||||
panic("no return value specified for Revision")
|
||||
}
|
||||
|
||||
var r0 uint64
|
||||
if returnFunc, ok := ret.Get(0).(func() uint64); ok {
|
||||
r0 = returnFunc()
|
||||
} else {
|
||||
r0 = ret.Get(0).(uint64)
|
||||
}
|
||||
return r0
|
||||
}
|
||||
|
||||
// KeyValueEntry_Revision_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Revision'
|
||||
type KeyValueEntry_Revision_Call struct {
|
||||
*mock.Call
|
||||
}
|
||||
|
||||
// Revision is a helper method to define mock.On call
|
||||
func (_e *KeyValueEntry_Expecter) Revision() *KeyValueEntry_Revision_Call {
|
||||
return &KeyValueEntry_Revision_Call{Call: _e.mock.On("Revision")}
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Revision_Call) Run(run func()) *KeyValueEntry_Revision_Call {
|
||||
_c.Call.Run(func(args mock.Arguments) {
|
||||
run()
|
||||
})
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Revision_Call) Return(v uint64) *KeyValueEntry_Revision_Call {
|
||||
_c.Call.Return(v)
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Revision_Call) RunAndReturn(run func() uint64) *KeyValueEntry_Revision_Call {
|
||||
_c.Call.Return(run)
|
||||
return _c
|
||||
}
|
||||
|
||||
// Value provides a mock function for the type KeyValueEntry
|
||||
func (_mock *KeyValueEntry) Value() []byte {
|
||||
ret := _mock.Called()
|
||||
|
||||
if len(ret) == 0 {
|
||||
panic("no return value specified for Value")
|
||||
}
|
||||
|
||||
var r0 []byte
|
||||
if returnFunc, ok := ret.Get(0).(func() []byte); ok {
|
||||
r0 = returnFunc()
|
||||
} else {
|
||||
if ret.Get(0) != nil {
|
||||
r0 = ret.Get(0).([]byte)
|
||||
}
|
||||
}
|
||||
return r0
|
||||
}
|
||||
|
||||
// KeyValueEntry_Value_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Value'
|
||||
type KeyValueEntry_Value_Call struct {
|
||||
*mock.Call
|
||||
}
|
||||
|
||||
// Value is a helper method to define mock.On call
|
||||
func (_e *KeyValueEntry_Expecter) Value() *KeyValueEntry_Value_Call {
|
||||
return &KeyValueEntry_Value_Call{Call: _e.mock.On("Value")}
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Value_Call) Run(run func()) *KeyValueEntry_Value_Call {
|
||||
_c.Call.Run(func(args mock.Arguments) {
|
||||
run()
|
||||
})
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Value_Call) Return(bytes []byte) *KeyValueEntry_Value_Call {
|
||||
_c.Call.Return(bytes)
|
||||
return _c
|
||||
}
|
||||
|
||||
func (_c *KeyValueEntry_Value_Call) RunAndReturn(run func() []byte) *KeyValueEntry_Value_Call {
|
||||
_c.Call.Return(run)
|
||||
return _c
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
// Package announcement persists and serves the web announcement banner.
|
||||
package announcement
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
)
|
||||
|
||||
// _storeKey is the single key under which the announcement is persisted.
|
||||
const _storeKey = "announcement"
|
||||
|
||||
// Announcement is a banner message shown above the top bar to all users.
|
||||
type Announcement struct {
|
||||
// Enabled controls whether the announcement is live (injected into config.json).
|
||||
Enabled bool `json:"enabled"`
|
||||
BannerText string `json:"bannerText"`
|
||||
InfoText string `json:"infoText"`
|
||||
}
|
||||
|
||||
// Store persists a single announcement in a NATS JetStream key-value bucket.
|
||||
type Store struct {
|
||||
kv jetstream.KeyValue
|
||||
}
|
||||
|
||||
// NewStore returns a new announcement Store backed by the given key-value bucket.
|
||||
func NewStore(kv jetstream.KeyValue) *Store {
|
||||
return &Store{kv: kv}
|
||||
}
|
||||
|
||||
// Get returns the currently stored announcement. An unset announcement is returned as the zero value.
|
||||
func (s *Store) Get(ctx context.Context) (Announcement, error) {
|
||||
var a Announcement
|
||||
|
||||
entry, err := s.kv.Get(ctx, _storeKey)
|
||||
if err != nil {
|
||||
if errors.Is(err, jetstream.ErrKeyNotFound) {
|
||||
return a, nil
|
||||
}
|
||||
return a, err
|
||||
}
|
||||
|
||||
if err := json.Unmarshal(entry.Value(), &a); err != nil {
|
||||
return a, err
|
||||
}
|
||||
|
||||
return a, nil
|
||||
}
|
||||
|
||||
// Set persists the given announcement, overwriting any existing one.
|
||||
func (s *Store) Set(ctx context.Context, a Announcement) error {
|
||||
value, err := json.Marshal(a)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
_, err = s.kv.Put(ctx, _storeKey, value)
|
||||
return err
|
||||
}
|
||||
|
||||
// Delete removes the stored announcement. Deleting a missing announcement is a no-op.
|
||||
func (s *Store) Delete(ctx context.Context) error {
|
||||
if err := s.kv.Delete(ctx, _storeKey); err != nil && !errors.Is(err, jetstream.ErrKeyNotFound) {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
package announcement_test
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
func TestAnnouncement(t *testing.T) {
|
||||
RegisterFailHandler(Fail)
|
||||
RunSpecs(t, "Announcement Suite")
|
||||
}
|
||||
@@ -0,0 +1,207 @@
|
||||
package announcement_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
|
||||
gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1"
|
||||
userpb "github.com/cs3org/go-cs3apis/cs3/identity/user/v1beta1"
|
||||
cs3permissions "github.com/cs3org/go-cs3apis/cs3/permissions/v1beta1"
|
||||
cs3rpc "github.com/cs3org/go-cs3apis/cs3/rpc/v1beta1"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
revactx "github.com/opencloud-eu/reva/v2/pkg/ctx"
|
||||
"github.com/opencloud-eu/reva/v2/pkg/rgrpc/todo/pool"
|
||||
cs3mocks "github.com/opencloud-eu/reva/v2/tests/cs3mocks/mocks"
|
||||
"github.com/stretchr/testify/mock"
|
||||
"google.golang.org/grpc"
|
||||
|
||||
"github.com/opencloud-eu/opencloud/pkg/log"
|
||||
"github.com/opencloud-eu/opencloud/services/web/mocks"
|
||||
"github.com/opencloud-eu/opencloud/services/web/pkg/announcement"
|
||||
)
|
||||
|
||||
func newGatewaySelector(allowed bool) pool.Selectable[gateway.GatewayAPIClient] {
|
||||
code := cs3rpc.Code_CODE_OK
|
||||
name := "announcement-test-allowed"
|
||||
if !allowed {
|
||||
code = cs3rpc.Code_CODE_PERMISSION_DENIED
|
||||
name = "announcement-test-denied"
|
||||
}
|
||||
|
||||
client := &cs3mocks.GatewayAPIClient{}
|
||||
client.On("CheckPermission", mock.Anything, mock.Anything).Return(
|
||||
&cs3permissions.CheckPermissionResponse{Status: &cs3rpc.Status{Code: code}}, nil)
|
||||
|
||||
// pool.GetSelector caches by name, so allow/deny must use distinct names
|
||||
return pool.GetSelector[gateway.GatewayAPIClient](
|
||||
name,
|
||||
"eu.opencloud.api.gateway",
|
||||
func(cc grpc.ClientConnInterface) gateway.GatewayAPIClient { return client },
|
||||
)
|
||||
}
|
||||
|
||||
func withUser(r *http.Request) *http.Request {
|
||||
return r.WithContext(revactx.ContextSetUser(r.Context(), &userpb.User{
|
||||
Id: &userpb.UserId{OpaqueId: "user"},
|
||||
}))
|
||||
}
|
||||
|
||||
func newService(store *announcement.Store, allowed bool) announcement.Service {
|
||||
svc, err := announcement.NewService(announcement.ServiceOptions{}.
|
||||
WithLogger(log.NopLogger()).
|
||||
WithStore(store).
|
||||
WithGatewaySelector(newGatewaySelector(allowed)))
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
return svc
|
||||
}
|
||||
|
||||
var _ = Describe("Store", func() {
|
||||
It("reads the stored announcement", func() {
|
||||
entry := mocks.NewKeyValueEntry(GinkgoT())
|
||||
entry.EXPECT().Value().Return([]byte(`{"enabled":true,"bannerText":"hello","infoText":"world"}`))
|
||||
kv := mocks.NewKeyValue(GinkgoT())
|
||||
kv.EXPECT().Get(mock.Anything, "announcement").Return(entry, nil)
|
||||
|
||||
got, err := announcement.NewStore(kv).Get(context.Background())
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(got.Enabled).To(BeTrue())
|
||||
Expect(got.BannerText).To(Equal("hello"))
|
||||
Expect(got.InfoText).To(Equal("world"))
|
||||
})
|
||||
|
||||
It("returns the zero value when unset", func() {
|
||||
kv := mocks.NewKeyValue(GinkgoT())
|
||||
kv.EXPECT().Get(mock.Anything, "announcement").Return(nil, jetstream.ErrKeyNotFound)
|
||||
|
||||
got, err := announcement.NewStore(kv).Get(context.Background())
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(got.BannerText).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("writes the announcement", func() {
|
||||
kv := mocks.NewKeyValue(GinkgoT())
|
||||
kv.EXPECT().Put(mock.Anything, "announcement", mock.Anything).Return(uint64(1), nil)
|
||||
|
||||
Expect(announcement.NewStore(kv).Set(context.Background(), announcement.Announcement{BannerText: "hello"})).To(Succeed())
|
||||
})
|
||||
|
||||
It("deletes the announcement", func() {
|
||||
kv := mocks.NewKeyValue(GinkgoT())
|
||||
kv.EXPECT().Delete(mock.Anything, "announcement").Return(nil)
|
||||
|
||||
Expect(announcement.NewStore(kv).Delete(context.Background())).To(Succeed())
|
||||
})
|
||||
|
||||
It("treats deleting a missing announcement as a no-op", func() {
|
||||
kv := mocks.NewKeyValue(GinkgoT())
|
||||
kv.EXPECT().Delete(mock.Anything, "announcement").Return(jetstream.ErrKeyNotFound)
|
||||
|
||||
Expect(announcement.NewStore(kv).Delete(context.Background())).To(Succeed())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("Service", func() {
|
||||
Describe("NewService", func() {
|
||||
It("fails when options are missing", func() {
|
||||
_, err := announcement.NewService(announcement.ServiceOptions{})
|
||||
Expect(err).To(HaveOccurred())
|
||||
})
|
||||
|
||||
It("succeeds when options are valid", func() {
|
||||
_, err := announcement.NewService(announcement.ServiceOptions{}.
|
||||
WithStore(announcement.NewStore(mocks.NewKeyValue(GinkgoT()))).
|
||||
WithGatewaySelector(newGatewaySelector(true)))
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
})
|
||||
})
|
||||
|
||||
Describe("Get", func() {
|
||||
It("returns the full stored announcement when permitted", func() {
|
||||
entry := mocks.NewKeyValueEntry(GinkgoT())
|
||||
entry.EXPECT().Value().Return([]byte(`{"enabled":true,"bannerText":"hello","infoText":"world"}`))
|
||||
kv := mocks.NewKeyValue(GinkgoT())
|
||||
kv.EXPECT().Get(mock.Anything, "announcement").Return(entry, nil)
|
||||
|
||||
req := withUser(httptest.NewRequest(http.MethodGet, "/announcement", nil))
|
||||
resp := httptest.NewRecorder()
|
||||
|
||||
newService(announcement.NewStore(kv), true).Get(resp, req)
|
||||
|
||||
Expect(resp.Code).To(Equal(http.StatusOK))
|
||||
var got announcement.Announcement
|
||||
Expect(json.Unmarshal(resp.Body.Bytes(), &got)).To(Succeed())
|
||||
Expect(got.Enabled).To(BeTrue())
|
||||
Expect(got.BannerText).To(Equal("hello"))
|
||||
Expect(got.InfoText).To(Equal("world"))
|
||||
})
|
||||
|
||||
It("is forbidden without permission", func() {
|
||||
req := withUser(httptest.NewRequest(http.MethodGet, "/announcement", nil))
|
||||
resp := httptest.NewRecorder()
|
||||
|
||||
newService(announcement.NewStore(mocks.NewKeyValue(GinkgoT())), false).Get(resp, req)
|
||||
|
||||
Expect(resp.Code).To(Equal(http.StatusForbidden))
|
||||
})
|
||||
})
|
||||
|
||||
Describe("Set", func() {
|
||||
It("persists the message when permitted", func() {
|
||||
kv := mocks.NewKeyValue(GinkgoT())
|
||||
kv.EXPECT().Put(mock.Anything, "announcement", mock.Anything).Return(uint64(1), nil)
|
||||
|
||||
req := withUser(httptest.NewRequest(http.MethodPut, "/announcement", strings.NewReader(`{"bannerText":"hello"}`)))
|
||||
resp := httptest.NewRecorder()
|
||||
|
||||
newService(announcement.NewStore(kv), true).Set(resp, req)
|
||||
|
||||
Expect(resp.Code).To(Equal(http.StatusNoContent))
|
||||
})
|
||||
|
||||
It("is forbidden without permission", func() {
|
||||
req := withUser(httptest.NewRequest(http.MethodPut, "/announcement", strings.NewReader(`{"bannerText":"hello"}`)))
|
||||
resp := httptest.NewRecorder()
|
||||
|
||||
newService(announcement.NewStore(mocks.NewKeyValue(GinkgoT())), false).Set(resp, req)
|
||||
|
||||
Expect(resp.Code).To(Equal(http.StatusForbidden))
|
||||
})
|
||||
|
||||
It("rejects an invalid body", func() {
|
||||
req := withUser(httptest.NewRequest(http.MethodPut, "/announcement", strings.NewReader(`not json`)))
|
||||
resp := httptest.NewRecorder()
|
||||
|
||||
newService(announcement.NewStore(mocks.NewKeyValue(GinkgoT())), true).Set(resp, req)
|
||||
|
||||
Expect(resp.Code).To(Equal(http.StatusBadRequest))
|
||||
})
|
||||
|
||||
It("rejects an oversized body", func() {
|
||||
body := `{"bannerText":"` + strings.Repeat("a", 300000) + `"}`
|
||||
req := withUser(httptest.NewRequest(http.MethodPut, "/announcement", strings.NewReader(body)))
|
||||
resp := httptest.NewRecorder()
|
||||
|
||||
newService(announcement.NewStore(mocks.NewKeyValue(GinkgoT())), true).Set(resp, req)
|
||||
|
||||
Expect(resp.Code).To(Equal(http.StatusRequestEntityTooLarge))
|
||||
})
|
||||
})
|
||||
|
||||
Describe("Set with an empty banner text", func() {
|
||||
It("removes the stored announcement", func() {
|
||||
kv := mocks.NewKeyValue(GinkgoT())
|
||||
kv.EXPECT().Delete(mock.Anything, "announcement").Return(nil)
|
||||
|
||||
req := withUser(httptest.NewRequest(http.MethodPut, "/announcement", strings.NewReader(`{"enabled":false,"bannerText":"","infoText":""}`)))
|
||||
resp := httptest.NewRecorder()
|
||||
newService(announcement.NewStore(kv), true).Set(resp, req)
|
||||
|
||||
Expect(resp.Code).To(Equal(http.StatusNoContent))
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,192 @@
|
||||
package announcement
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
|
||||
gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1"
|
||||
permissionsapi "github.com/cs3org/go-cs3apis/cs3/permissions/v1beta1"
|
||||
rpc "github.com/cs3org/go-cs3apis/cs3/rpc/v1beta1"
|
||||
"github.com/pkg/errors"
|
||||
|
||||
revactx "github.com/opencloud-eu/reva/v2/pkg/ctx"
|
||||
"github.com/opencloud-eu/reva/v2/pkg/rgrpc/todo/pool"
|
||||
|
||||
"github.com/opencloud-eu/opencloud/pkg/log"
|
||||
)
|
||||
|
||||
// _permission is the settings permission required to read and manage the announcement.
|
||||
const _permission = "Announcement.ReadWrite"
|
||||
|
||||
// _maxBodySize caps the announcement request body. The info text is Markdown and ends up in
|
||||
// the public config.json that every client loads on bootstrap, so it must stay small.
|
||||
const _maxBodySize = 50 << 10 // 50 KiB
|
||||
|
||||
// ServiceOptions defines the options to configure the Service.
|
||||
type ServiceOptions struct {
|
||||
logger log.Logger
|
||||
store *Store
|
||||
gatewaySelector pool.Selectable[gateway.GatewayAPIClient]
|
||||
}
|
||||
|
||||
// WithLogger sets the logger.
|
||||
func (o ServiceOptions) WithLogger(l log.Logger) ServiceOptions {
|
||||
o.logger = l
|
||||
return o
|
||||
}
|
||||
|
||||
// WithStore sets the announcement store.
|
||||
func (o ServiceOptions) WithStore(s *Store) ServiceOptions {
|
||||
o.store = s
|
||||
return o
|
||||
}
|
||||
|
||||
// WithGatewaySelector sets the gateway selector.
|
||||
func (o ServiceOptions) WithGatewaySelector(gws pool.Selectable[gateway.GatewayAPIClient]) ServiceOptions {
|
||||
o.gatewaySelector = gws
|
||||
return o
|
||||
}
|
||||
|
||||
// validate validates the input parameters.
|
||||
func (o ServiceOptions) validate() error {
|
||||
if o.store == nil {
|
||||
return errors.New("store is required")
|
||||
}
|
||||
|
||||
if o.gatewaySelector == nil {
|
||||
return errors.New("gatewaySelector is required")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// Service exposes the http handlers to manage the announcement.
|
||||
type Service struct {
|
||||
logger log.Logger
|
||||
store *Store
|
||||
gatewaySelector pool.Selectable[gateway.GatewayAPIClient]
|
||||
}
|
||||
|
||||
// NewService initializes a new Service.
|
||||
func NewService(options ServiceOptions) (Service, error) {
|
||||
if err := options.validate(); err != nil {
|
||||
return Service{}, err
|
||||
}
|
||||
|
||||
return Service{
|
||||
logger: options.logger,
|
||||
store: options.store,
|
||||
gatewaySelector: options.gatewaySelector,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// logError logs a server-side error together with the request id, so it can be correlated with
|
||||
// the request log line emitted by the logging middleware.
|
||||
func (s Service) logError(r *http.Request, err error, msg string) {
|
||||
s.logger.Error().Err(err).Str(log.RequestIDString, r.Header.Get("X-Request-ID")).Msg(msg)
|
||||
}
|
||||
|
||||
// Get returns the full stored announcement (including disabled ones) for management.
|
||||
func (s Service) Get(w http.ResponseWriter, r *http.Request) {
|
||||
gatewayClient, err := s.gatewaySelector.Next()
|
||||
if err != nil {
|
||||
s.logError(r, err, "could not select next gateway client")
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
user, ok := revactx.ContextGetUser(r.Context())
|
||||
if !ok {
|
||||
w.WriteHeader(http.StatusUnauthorized)
|
||||
return
|
||||
}
|
||||
rsp, err := gatewayClient.CheckPermission(r.Context(), &permissionsapi.CheckPermissionRequest{
|
||||
Permission: _permission,
|
||||
SubjectRef: &permissionsapi.SubjectReference{
|
||||
Spec: &permissionsapi.SubjectReference_UserId{
|
||||
UserId: user.GetId(),
|
||||
},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
s.logError(r, err, "could not check permission")
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
if rsp.GetStatus().GetCode() != rpc.Code_CODE_OK {
|
||||
w.WriteHeader(http.StatusForbidden)
|
||||
return
|
||||
}
|
||||
|
||||
a, err := s.store.Get(r.Context())
|
||||
if err != nil {
|
||||
s.logError(r, err, "could not read announcement from store")
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
if err := json.NewEncoder(w).Encode(a); err != nil {
|
||||
s.logError(r, err, "could not encode announcement")
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
}
|
||||
}
|
||||
|
||||
// Set persists the announcement provided in the request body.
|
||||
func (s Service) Set(w http.ResponseWriter, r *http.Request) {
|
||||
gatewayClient, err := s.gatewaySelector.Next()
|
||||
if err != nil {
|
||||
s.logError(r, err, "could not select next gateway client")
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
user, ok := revactx.ContextGetUser(r.Context())
|
||||
if !ok {
|
||||
w.WriteHeader(http.StatusUnauthorized)
|
||||
return
|
||||
}
|
||||
rsp, err := gatewayClient.CheckPermission(r.Context(), &permissionsapi.CheckPermissionRequest{
|
||||
Permission: _permission,
|
||||
SubjectRef: &permissionsapi.SubjectReference{
|
||||
Spec: &permissionsapi.SubjectReference_UserId{
|
||||
UserId: user.GetId(),
|
||||
},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
s.logError(r, err, "could not check permission")
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
if rsp.GetStatus().GetCode() != rpc.Code_CODE_OK {
|
||||
w.WriteHeader(http.StatusForbidden)
|
||||
return
|
||||
}
|
||||
|
||||
var body Announcement
|
||||
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, _maxBodySize)).Decode(&body); err != nil {
|
||||
var maxBytesErr *http.MaxBytesError
|
||||
if errors.As(err, &maxBytesErr) {
|
||||
w.WriteHeader(http.StatusRequestEntityTooLarge)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
// an announcement without a banner text is nothing to show, so remove it entirely
|
||||
if body.BannerText == "" {
|
||||
if err := s.store.Delete(r.Context()); err != nil {
|
||||
s.logError(r, err, "could not delete announcement from store")
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
} else if err := s.store.Set(r.Context(), body); err != nil {
|
||||
s.logError(r, err, "could not write announcement to store")
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
@@ -25,9 +25,22 @@ type Config struct {
|
||||
TokenManager *TokenManager `yaml:"token_manager"`
|
||||
|
||||
GatewayAddress string `yaml:"gateway_addr" env:"WEB_GATEWAY_GRPC_ADDR" desc:"The bind address of the GRPC service." introductionVersion:"1.0.0"`
|
||||
Store Store `yaml:"store"`
|
||||
Context context.Context `yaml:"-"`
|
||||
}
|
||||
|
||||
// Store configures the NATS JetStream key-value store used to keep runtime managed web settings,
|
||||
// e.g. the announcement banner.
|
||||
type Store struct {
|
||||
Nodes []string `yaml:"nodes" env:"OC_PERSISTENT_STORE_NODES;WEB_STORE_NODES" desc:"A list of nodes to access the NATS JetStream store. See the Environment Variable Types description for more details." introductionVersion:"7.4.0"`
|
||||
Database string `yaml:"database" env:"WEB_STORE_DATABASE" desc:"The bucket name the store should use." introductionVersion:"7.4.0"`
|
||||
AuthUsername string `yaml:"username" env:"OC_PERSISTENT_STORE_AUTH_USERNAME;WEB_STORE_AUTH_USERNAME" desc:"The username to authenticate with the store." introductionVersion:"7.4.0"`
|
||||
AuthPassword string `yaml:"password" env:"OC_PERSISTENT_STORE_AUTH_PASSWORD;WEB_STORE_AUTH_PASSWORD" desc:"The password to authenticate with the store." introductionVersion:"7.4.0"`
|
||||
EnableTLS bool `yaml:"enable_tls" env:"OC_PERSISTENT_STORE_ENABLE_TLS;WEB_STORE_ENABLE_TLS" desc:"Enable TLS for the connection to the store." introductionVersion:"7.4.0"`
|
||||
TLSInsecure bool `yaml:"tls_insecure" env:"OC_INSECURE;OC_PERSISTENT_STORE_TLS_INSECURE;WEB_STORE_TLS_INSECURE" desc:"Whether to verify the server TLS certificates." introductionVersion:"7.4.0"`
|
||||
TLSRootCACertificate string `yaml:"tls_root_ca_certificate" env:"OC_PERSISTENT_STORE_TLS_ROOT_CA_CERTIFICATE;WEB_STORE_TLS_ROOT_CA_CERTIFICATE" desc:"The root CA certificate used to validate the server's TLS certificate. If provided WEB_STORE_TLS_INSECURE will be seen as false." introductionVersion:"7.4.0"`
|
||||
}
|
||||
|
||||
// Asset defines the available asset configuration.
|
||||
type Asset struct {
|
||||
CorePath string `yaml:"core_path" env:"WEB_ASSET_CORE_PATH" desc:"Serve OpenCloud Web assets from a path on the filesystem instead of the builtin assets. If not defined, the root directory derives from $OC_BASE_DATA_PATH/web/assets/core" introductionVersion:"1.0.0"`
|
||||
|
||||
@@ -85,6 +85,10 @@ func DefaultConfig() *config.Config {
|
||||
ThemesPath: filepath.Join(defaults.BaseDataPath(), "web/assets/themes"),
|
||||
},
|
||||
GatewayAddress: "eu.opencloud.api.gateway",
|
||||
Store: config.Store{
|
||||
Nodes: []string{"127.0.0.1:9233"},
|
||||
Database: "web",
|
||||
},
|
||||
Web: config.Web{
|
||||
ThemeServer: "https://localhost:9200",
|
||||
ThemePath: "/themes/opencloud/theme.json",
|
||||
|
||||
@@ -2,6 +2,7 @@ package config
|
||||
|
||||
// Options are the option for the web
|
||||
type Options struct {
|
||||
Announcement *Announcement `json:"announcement,omitempty" yaml:"-"`
|
||||
AccountEditLink *AccountEditLink `json:"accountEditLink,omitempty" yaml:"accountEditLink"`
|
||||
DisableFeedbackLink bool `json:"disableFeedbackLink,omitempty" yaml:"disableFeedbackLink" env:"WEB_OPTION_DISABLE_FEEDBACK_LINK" desc:"Set this option to 'true' to disable the feedback link in the top bar. Keeping it enabled by setting the value to 'false' or with the absence of the option, allows OpenCloud to get feedback from your user base through a dedicated survey website." introductionVersion:"1.0.0"`
|
||||
DisableSponsorLink bool `json:"disableSponsorLink,omitempty" yaml:"disableSponsorLink" env:"WEB_OPTION_DISABLE_SPONSOR_LINK" desc:"Set this option to 'true' to disable the sponsor link in the left sidebar. Keeping it enabled by setting the value to 'false' or by leaving the option unset allows OpenCloud to get support from the community through a dedicated sponsorship program on GitHub." introductionVersion:"7.3.0"`
|
||||
@@ -23,6 +24,16 @@ type Options struct {
|
||||
OxAppSuite *OxAppSuite `json:"oxAppSuite,omitempty" yaml:"oxAppSuite"`
|
||||
}
|
||||
|
||||
// Announcement is a banner message shown above the top bar to all users. It is managed at runtime
|
||||
// (via the web service's store and the admin settings UI) and injected into config.json here; a
|
||||
// value configured statically is ignored.
|
||||
type Announcement struct {
|
||||
// BannerText is the short line shown in the banner.
|
||||
BannerText string `json:"bannerText,omitempty" yaml:"-"`
|
||||
// InfoText is the (Markdown) detail shown in a dialog when the banner is clicked.
|
||||
InfoText string `json:"infoText,omitempty" yaml:"-"`
|
||||
}
|
||||
|
||||
// AccountEditLink are the AccountEditLink options
|
||||
type AccountEditLink struct {
|
||||
Href string `json:"href,omitempty" yaml:"href" env:"WEB_OPTION_ACCOUNT_EDIT_LINK_HREF" desc:"Set a different target URL for the edit link. Make sure to prepend it with 'http(s)://'." introductionVersion:"1.0.0"`
|
||||
|
||||
@@ -1,20 +1,26 @@
|
||||
package http
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"path"
|
||||
"strings"
|
||||
|
||||
chimiddleware "github.com/go-chi/chi/v5/middleware"
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
"github.com/opencloud-eu/reva/v2/pkg/rgrpc/todo/pool"
|
||||
"go-micro.dev/v4"
|
||||
|
||||
"github.com/opencloud-eu/opencloud/pkg/cors"
|
||||
"github.com/opencloud-eu/opencloud/pkg/middleware"
|
||||
natspkg "github.com/opencloud-eu/opencloud/pkg/nats"
|
||||
"github.com/opencloud-eu/opencloud/pkg/registry"
|
||||
"github.com/opencloud-eu/opencloud/pkg/service/http"
|
||||
"github.com/opencloud-eu/opencloud/pkg/version"
|
||||
"github.com/opencloud-eu/opencloud/pkg/x/io/fsx"
|
||||
"github.com/opencloud-eu/opencloud/services/web"
|
||||
"github.com/opencloud-eu/opencloud/services/web/pkg/announcement"
|
||||
"github.com/opencloud-eu/opencloud/services/web/pkg/apps"
|
||||
svc "github.com/opencloud-eu/opencloud/services/web/pkg/service/v0"
|
||||
)
|
||||
@@ -76,11 +82,38 @@ func Server(opts ...Option) (http.Service, error) {
|
||||
fsx.NewBasePathFs(fsx.FromIOFS(web.Assets), "assets/themes"),
|
||||
)
|
||||
|
||||
// NATS JetStream key-value store for runtime managed web settings, e.g. the announcement banner.
|
||||
// Connect eagerly and fail fast: an unreachable store means the feature would be broken, so a
|
||||
// clear startup error is preferable to silently degrading.
|
||||
natsConn, err := nats.Connect(
|
||||
strings.Join(options.Config.Store.Nodes, ","),
|
||||
natspkg.Secure(options.Config.Store.EnableTLS, options.Config.Store.TLSInsecure, options.Config.Store.TLSRootCACertificate),
|
||||
nats.UserInfo(options.Config.Store.AuthUsername, options.Config.Store.AuthPassword),
|
||||
)
|
||||
if err != nil {
|
||||
return http.Service{}, fmt.Errorf("could not connect to nats for the announcement store: %w", err)
|
||||
}
|
||||
js, err := jetstream.New(natsConn)
|
||||
if err != nil {
|
||||
return http.Service{}, fmt.Errorf("could not create jetstream context for the announcement store: %w", err)
|
||||
}
|
||||
kv, err := js.KeyValue(options.Context, options.Config.Store.Database)
|
||||
if err != nil {
|
||||
if !errors.Is(err, jetstream.ErrBucketNotFound) {
|
||||
return http.Service{}, fmt.Errorf("could not open the announcement store bucket %q: %w", options.Config.Store.Database, err)
|
||||
}
|
||||
if kv, err = js.CreateKeyValue(options.Context, jetstream.KeyValueConfig{Bucket: options.Config.Store.Database}); err != nil {
|
||||
return http.Service{}, fmt.Errorf("could not create the announcement store bucket %q: %w", options.Config.Store.Database, err)
|
||||
}
|
||||
}
|
||||
announcementStore := announcement.NewStore(kv)
|
||||
|
||||
handle, err := svc.NewService(
|
||||
svc.Logger(options.Logger),
|
||||
svc.CoreFS(coreFS.IOFS()),
|
||||
svc.AppFS(appsFS.IOFS()),
|
||||
svc.ThemeFS(themeFS),
|
||||
svc.AnnouncementStore(announcementStore),
|
||||
svc.AppsHTTPEndpoint(_customAppsEndpoint),
|
||||
svc.Config(options.Config),
|
||||
svc.GatewaySelector(gatewaySelector),
|
||||
|
||||
@@ -10,6 +10,7 @@ import (
|
||||
|
||||
"github.com/opencloud-eu/opencloud/pkg/log"
|
||||
"github.com/opencloud-eu/opencloud/pkg/x/io/fsx"
|
||||
"github.com/opencloud-eu/opencloud/services/web/pkg/announcement"
|
||||
"github.com/opencloud-eu/opencloud/services/web/pkg/config"
|
||||
)
|
||||
|
||||
@@ -18,15 +19,16 @@ type Option func(o *Options)
|
||||
|
||||
// Options define the available options for this package.
|
||||
type Options struct {
|
||||
Logger log.Logger
|
||||
Config *config.Config
|
||||
Middleware []func(http.Handler) http.Handler
|
||||
GatewaySelector pool.Selectable[gateway.GatewayAPIClient]
|
||||
TraceProvider trace.TracerProvider
|
||||
AppsHTTPEndpoint string
|
||||
CoreFS fs.FS
|
||||
AppFS fs.FS
|
||||
ThemeFS *fsx.FallbackFS
|
||||
Logger log.Logger
|
||||
Config *config.Config
|
||||
Middleware []func(http.Handler) http.Handler
|
||||
GatewaySelector pool.Selectable[gateway.GatewayAPIClient]
|
||||
TraceProvider trace.TracerProvider
|
||||
AppsHTTPEndpoint string
|
||||
CoreFS fs.FS
|
||||
AppFS fs.FS
|
||||
ThemeFS *fsx.FallbackFS
|
||||
AnnouncementStore *announcement.Store
|
||||
}
|
||||
|
||||
// newOptions initializes the available default options.
|
||||
@@ -89,6 +91,13 @@ func ThemeFS(val *fsx.FallbackFS) Option {
|
||||
}
|
||||
}
|
||||
|
||||
// AnnouncementStore provides a function to set the announcement store option.
|
||||
func AnnouncementStore(val *announcement.Store) Option {
|
||||
return func(o *Options) {
|
||||
o.AnnouncementStore = val
|
||||
}
|
||||
}
|
||||
|
||||
// AppsHTTPEndpoint provides a function to set the appsHTTPEndpoint option.
|
||||
func AppsHTTPEndpoint(val string) Option {
|
||||
return func(o *Options) {
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package svc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io/fs"
|
||||
"net/http"
|
||||
@@ -19,6 +20,7 @@ import (
|
||||
"github.com/opencloud-eu/opencloud/pkg/log"
|
||||
"github.com/opencloud-eu/opencloud/pkg/middleware"
|
||||
"github.com/opencloud-eu/opencloud/pkg/tracing"
|
||||
"github.com/opencloud-eu/opencloud/services/web/pkg/announcement"
|
||||
"github.com/opencloud-eu/opencloud/services/web/pkg/assets"
|
||||
"github.com/opencloud-eu/opencloud/services/web/pkg/config"
|
||||
"github.com/opencloud-eu/opencloud/services/web/pkg/theme"
|
||||
@@ -50,10 +52,11 @@ func NewService(opts ...Option) (Service, error) {
|
||||
)
|
||||
|
||||
svc := Web{
|
||||
logger: options.Logger,
|
||||
config: options.Config,
|
||||
mux: m,
|
||||
gatewaySelector: options.GatewaySelector,
|
||||
logger: options.Logger,
|
||||
config: options.Config,
|
||||
mux: m,
|
||||
gatewaySelector: options.GatewaySelector,
|
||||
announcementStore: options.AnnouncementStore,
|
||||
}
|
||||
|
||||
themeService, err := theme.NewService(
|
||||
@@ -65,6 +68,16 @@ func NewService(opts ...Option) (Service, error) {
|
||||
return svc, err
|
||||
}
|
||||
|
||||
announcementService, err := announcement.NewService(
|
||||
announcement.ServiceOptions{}.
|
||||
WithLogger(options.Logger).
|
||||
WithStore(options.AnnouncementStore).
|
||||
WithGatewaySelector(options.GatewaySelector),
|
||||
)
|
||||
if err != nil {
|
||||
return svc, err
|
||||
}
|
||||
|
||||
m.Route(options.Config.HTTP.Root, func(r chi.Router) {
|
||||
r.Get("/config.json", svc.Config)
|
||||
r.Route("/branding/logo", func(r chi.Router) {
|
||||
@@ -75,6 +88,14 @@ func NewService(opts ...Option) (Service, error) {
|
||||
r.Post("/", themeService.LogoUpload)
|
||||
r.Delete("/", themeService.LogoReset)
|
||||
})
|
||||
r.Route("/announcement", func(r chi.Router) {
|
||||
r.Use(middleware.ExtractAccountUUID(
|
||||
account.Logger(options.Logger),
|
||||
account.JWTSecret(options.Config.TokenManager.JWTSecret),
|
||||
))
|
||||
r.Get("/", announcementService.Get)
|
||||
r.Put("/", announcementService.Set)
|
||||
})
|
||||
r.Route("/themes", func(r chi.Router) {
|
||||
r.Get("/{id}/theme.json", themeService.Get)
|
||||
r.Mount("/", svc.Static(
|
||||
@@ -104,10 +125,11 @@ func NewService(opts ...Option) (Service, error) {
|
||||
|
||||
// Web defines the handlers for the web service.
|
||||
type Web struct {
|
||||
logger log.Logger
|
||||
config *config.Config
|
||||
mux *chi.Mux
|
||||
gatewaySelector pool.Selectable[gateway.GatewayAPIClient]
|
||||
logger log.Logger
|
||||
config *config.Config
|
||||
mux *chi.Mux
|
||||
gatewaySelector pool.Selectable[gateway.GatewayAPIClient]
|
||||
announcementStore *announcement.Store
|
||||
}
|
||||
|
||||
// ServeHTTP implements the Service interface.
|
||||
@@ -115,31 +137,58 @@ func (p Web) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
p.mux.ServeHTTP(w, r)
|
||||
}
|
||||
|
||||
func (p Web) getPayload() (payload []byte, err error) {
|
||||
// render dynamically using config
|
||||
func (p Web) getPayload(ctx context.Context) (payload []byte, err error) {
|
||||
// render dynamically using a copy of the config, so per-request values (e.g. the
|
||||
// announcement) are not written into the shared config concurrently.
|
||||
webConfig := p.config.Web.Config
|
||||
|
||||
// build theme url
|
||||
if themeServer, err := url.Parse(p.config.Web.ThemeServer); err == nil {
|
||||
p.config.Web.Config.Theme = themeServer.String() + p.config.Web.ThemePath
|
||||
webConfig.Theme = themeServer.String() + p.config.Web.ThemePath
|
||||
} else {
|
||||
p.config.Web.Config.Theme = p.config.Web.ThemePath
|
||||
webConfig.Theme = p.config.Web.ThemePath
|
||||
}
|
||||
|
||||
// make apps render as empty array if it is empty
|
||||
// TODO remove once https://github.com/golang/go/issues/27589 is fixed
|
||||
if len(p.config.Web.Config.Apps) == 0 {
|
||||
p.config.Web.Config.Apps = make([]string, 0)
|
||||
if len(webConfig.Apps) == 0 {
|
||||
webConfig.Apps = make([]string, 0)
|
||||
}
|
||||
|
||||
// ensure that the server url has a trailing slash
|
||||
p.config.Web.Config.Server = strings.TrimRight(p.config.Web.Config.Server, "/") + "/"
|
||||
webConfig.Server = strings.TrimRight(webConfig.Server, "/") + "/"
|
||||
|
||||
return json.Marshal(p.config.Web.Config)
|
||||
// the runtime store is the single source of truth for the announcement banner: expose it
|
||||
// when live, clear it otherwise. A statically configured value is not supported.
|
||||
webConfig.Options.Announcement = p.currentAnnouncement(ctx)
|
||||
|
||||
return json.Marshal(webConfig)
|
||||
}
|
||||
|
||||
// currentAnnouncement returns the stored announcement for config.json, or nil if unset, disabled
|
||||
// or unavailable.
|
||||
func (p Web) currentAnnouncement(ctx context.Context) *config.Announcement {
|
||||
if p.announcementStore == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
a, err := p.announcementStore.Get(ctx)
|
||||
if err != nil {
|
||||
p.logger.Error().Err(err).Msg("could not read announcement from store")
|
||||
return nil
|
||||
}
|
||||
|
||||
// only live (enabled) announcements with a banner line are exposed in the public config.json
|
||||
if !a.Enabled || a.BannerText == "" {
|
||||
return nil
|
||||
}
|
||||
|
||||
return &config.Announcement{BannerText: a.BannerText, InfoText: a.InfoText}
|
||||
}
|
||||
|
||||
// Config implements the Service interface.
|
||||
func (p Web) Config(w http.ResponseWriter, _ *http.Request) {
|
||||
payload, err := p.getPayload()
|
||||
func (p Web) Config(w http.ResponseWriter, r *http.Request) {
|
||||
payload, err := p.getPayload(r.Context())
|
||||
if err != nil {
|
||||
http.Error(w, ErrConfigInvalid, http.StatusUnprocessableEntity)
|
||||
return
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
package svc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
"github.com/stretchr/testify/mock"
|
||||
"github.com/stretchr/testify/require"
|
||||
|
||||
"github.com/opencloud-eu/opencloud/pkg/log"
|
||||
"github.com/opencloud-eu/opencloud/services/web/mocks"
|
||||
"github.com/opencloud-eu/opencloud/services/web/pkg/announcement"
|
||||
"github.com/opencloud-eu/opencloud/services/web/pkg/config"
|
||||
)
|
||||
|
||||
func TestCurrentAnnouncement(t *testing.T) {
|
||||
newWeb := func(store *announcement.Store) Web {
|
||||
return Web{logger: log.NopLogger(), announcementStore: store}
|
||||
}
|
||||
// storeReturning builds a store whose backing bucket returns the given JSON for the announcement key.
|
||||
storeReturning := func(t *testing.T, value string) *announcement.Store {
|
||||
entry := mocks.NewKeyValueEntry(t)
|
||||
entry.EXPECT().Value().Return([]byte(value))
|
||||
kv := mocks.NewKeyValue(t)
|
||||
kv.EXPECT().Get(mock.Anything, "announcement").Return(entry, nil)
|
||||
return announcement.NewStore(kv)
|
||||
}
|
||||
// emptyStore builds a store whose backing bucket has no announcement.
|
||||
emptyStore := func(t *testing.T) *announcement.Store {
|
||||
kv := mocks.NewKeyValue(t)
|
||||
kv.EXPECT().Get(mock.Anything, "announcement").Return(nil, jetstream.ErrKeyNotFound)
|
||||
return announcement.NewStore(kv)
|
||||
}
|
||||
|
||||
t.Run("nil when there is no store", func(t *testing.T) {
|
||||
require.Nil(t, newWeb(nil).currentAnnouncement(context.Background()))
|
||||
})
|
||||
|
||||
t.Run("nil when the store is empty", func(t *testing.T) {
|
||||
require.Nil(t, newWeb(emptyStore(t)).currentAnnouncement(context.Background()))
|
||||
})
|
||||
|
||||
t.Run("nil when disabled", func(t *testing.T) {
|
||||
s := storeReturning(t, `{"enabled":false,"bannerText":"hi","infoText":"info"}`)
|
||||
require.Nil(t, newWeb(s).currentAnnouncement(context.Background()))
|
||||
})
|
||||
|
||||
t.Run("nil when enabled but the banner text is empty", func(t *testing.T) {
|
||||
s := storeReturning(t, `{"enabled":true,"bannerText":"","infoText":"info"}`)
|
||||
require.Nil(t, newWeb(s).currentAnnouncement(context.Background()))
|
||||
})
|
||||
|
||||
t.Run("returns banner and info text when enabled with a banner text", func(t *testing.T) {
|
||||
s := storeReturning(t, `{"enabled":true,"bannerText":"hi","infoText":"info"}`)
|
||||
require.Equal(t, &config.Announcement{BannerText: "hi", InfoText: "info"}, newWeb(s).currentAnnouncement(context.Background()))
|
||||
})
|
||||
}
|
||||
@@ -1031,19 +1031,7 @@ class SharingNgContext implements Context {
|
||||
$this->featureContext->shareNgAddToCreatedUserGroupShares($this->getDrivePermissionsList($sharer, $space));
|
||||
$permissionID = $this->featureContext->shareNgGetLastCreatedUserGroupShareID();
|
||||
} elseif ($shareType == 'group' && !isset($recipient)) {
|
||||
// https://github.com/opencloud-eu/opencloud/pull/3179#issuecomment-5103212045
|
||||
$retried = 0;
|
||||
do {
|
||||
$response = $this->getDrivePermissionsList($sharer, $space);
|
||||
$tryAgain = $response->getStatusCode() === 404
|
||||
&& $retried < HttpRequestHelper::numRetriesOnHttpTooEarly();
|
||||
if ($tryAgain) {
|
||||
$retried += 1;
|
||||
echo "Drive permissions of space '$space' not available for user '$sharer' yet, retrying ($retried)...\n";
|
||||
// wait 500ms and try again
|
||||
\usleep(500 * 1000);
|
||||
}
|
||||
} while ($tryAgain);
|
||||
$response = $this->getDrivePermissionsList($sharer, $space);
|
||||
$permissionID = $this->featureContext->getJsonDecodedResponse($response)['value'][0]['id'];
|
||||
} else {
|
||||
$permissionID = match ($shareType) {
|
||||
|
||||
@@ -141,7 +141,7 @@ Feature: Remove access to a drive
|
||||
Then the HTTP status code should be "403"
|
||||
And the user "Alice" should have a space called "NewSpace"
|
||||
|
||||
|
||||
@flaky @issue-3193
|
||||
Scenario: user of a group cannot remove own group from project space if it is the last manager using root endpoint
|
||||
Given the administrator has assigned the role "Space Admin" to user "Alice" using the Graph API
|
||||
And group "group1" has been created
|
||||
|
||||
+2
-2
@@ -1,4 +1,4 @@
|
||||
//go:build enable_antithesis_sdk
|
||||
//go:build !no_antithesis_sdk
|
||||
|
||||
// Package assert enables defining [test properties] about your program or [workload]. It is part of the [Antithesis Go SDK], which enables Go applications to integrate with the [Antithesis platform].
|
||||
//
|
||||
@@ -6,7 +6,7 @@
|
||||
//
|
||||
// These functions are no-ops with minimal performance overhead when called outside of the Antithesis environment. However, if the environment variable ANTITHESIS_SDK_LOCAL_OUTPUT is set, these functions will log to the file pointed to by that variable using a structured JSON format defined [here]. This allows you to make use of the Antithesis assertions package in your regular testing, or even in production. In particular, very few assertions frameworks offer a convenient way to define [Sometimes assertions], but they can be quite useful even outside Antithesis.
|
||||
//
|
||||
// Each function in this package takes a parameter called message, which is a human readable identifier used to aggregate assertions. Antithesis generates one test property per unique message and this test property will be named "<message>" in the [triage report].
|
||||
// Each function in this package takes a parameter called message, which is a human readable identifier used to aggregate assertions. Antithesis generates one test property per unique message and this test property will be named "<message>" in the [triage report]. Message must be provided as a string literal.
|
||||
//
|
||||
// This test property either passes or fails, which depends upon the evaluation of every assertion that shares its message. Different assertions in different parts of the code should have different message, but the same assertion should always have the same message even if it is moved to a different file.
|
||||
//
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
//go:build !enable_antithesis_sdk
|
||||
//go:build no_antithesis_sdk
|
||||
|
||||
package assert
|
||||
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
//go:build enable_antithesis_sdk
|
||||
//go:build !no_antithesis_sdk
|
||||
|
||||
package assert
|
||||
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
//go:build enable_antithesis_sdk
|
||||
//go:build !no_antithesis_sdk
|
||||
|
||||
package assert
|
||||
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
//go:build enable_antithesis_sdk
|
||||
//go:build !no_antithesis_sdk
|
||||
|
||||
package assert
|
||||
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
//go:build enable_antithesis_sdk
|
||||
//go:build !no_antithesis_sdk
|
||||
|
||||
package assert
|
||||
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
//go:build !enable_antithesis_sdk
|
||||
//go:build no_antithesis_sdk
|
||||
|
||||
package assert
|
||||
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
//go:build enable_antithesis_sdk
|
||||
//go:build !no_antithesis_sdk
|
||||
|
||||
package assert
|
||||
|
||||
|
||||
+2
-2
@@ -1,4 +1,4 @@
|
||||
//go:build enable_antithesis_sdk
|
||||
//go:build !no_antithesis_sdk
|
||||
|
||||
package internal
|
||||
|
||||
@@ -38,7 +38,7 @@ type libHandler interface {
|
||||
}
|
||||
|
||||
const (
|
||||
errorLogLinePrefix = "[* antithesis-sdk-go *]"
|
||||
errorLogLinePrefix = "[* antithesis-sdk-go *]"
|
||||
)
|
||||
|
||||
var handler libHandler
|
||||
|
||||
+1
-1
@@ -3,7 +3,7 @@ package internal
|
||||
// --------------------------------------------------------------------------------
|
||||
// Versions
|
||||
// --------------------------------------------------------------------------------
|
||||
const SDK_Version = "0.7.0"
|
||||
const SDK_Version = "0.7.2"
|
||||
const Protocol_Version = "1.1.0"
|
||||
|
||||
// --------------------------------------------------------------------------------
|
||||
|
||||
+2
-2
@@ -1,11 +1,11 @@
|
||||
//go:build enable_antithesis_sdk && linux && amd64 && cgo
|
||||
//go:build !no_antithesis_sdk && linux && amd64 && cgo
|
||||
|
||||
package internal
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"unsafe"
|
||||
"os"
|
||||
"unsafe"
|
||||
)
|
||||
|
||||
// --------------------------------------------------------------------------------
|
||||
|
||||
Generated
Vendored
+1
-1
@@ -1,4 +1,4 @@
|
||||
//go:build enable_antithesis_sdk && (!linux || !amd64 || !cgo)
|
||||
//go:build !no_antithesis_sdk && (!linux || !amd64 || !cgo)
|
||||
|
||||
package internal
|
||||
|
||||
|
||||
+24
@@ -1,3 +1,27 @@
|
||||
Release 1.7.0
|
||||
=============
|
||||
|
||||
**Changes**
|
||||
|
||||
**Breaking changes**
|
||||
|
||||
* To address a security issue, it was necessary to add a `MaxDepth` option to
|
||||
`ReadSettings` to limit the depth of XML trees during parsing. A generous
|
||||
default value of 1024 was chosen to avoid breaking most existing code.
|
||||
However, if your code is processing XML hierarchies with a depth greater
|
||||
than 1024, you will need to assign your `Document` a `ReadSettings` that has
|
||||
a `MaxDepth` set to a higher value.
|
||||
|
||||
**Security Fixes**
|
||||
|
||||
* Limited the depth of XML trees processed by all `ReadFrom` functions during
|
||||
parsing.
|
||||
* Fixed a `CompilePath` index-out-of-range panic that could be caused by a
|
||||
missing path filter key.
|
||||
* Sanitized the contents of XML text, comment, ProcInst and Directive tokens
|
||||
provided by the user.
|
||||
|
||||
|
||||
Release 1.6.0
|
||||
=============
|
||||
|
||||
|
||||
+31
-17
@@ -13,6 +13,7 @@ import (
|
||||
"errors"
|
||||
"io"
|
||||
"iter"
|
||||
"maps"
|
||||
"os"
|
||||
"slices"
|
||||
"strings"
|
||||
@@ -27,6 +28,10 @@ const (
|
||||
// ErrXML is returned when XML parsing fails due to incorrect formatting.
|
||||
var ErrXML = errors.New("etree: invalid XML format")
|
||||
|
||||
// ErrMaxDepth is returned when the depth of the XML tree being read exceeds
|
||||
// the maximum depth allowed by ReadSettings.MaxDepth.
|
||||
var ErrMaxDepth = errors.New("etree: XML tree exceeds maximum depth")
|
||||
|
||||
// cdataPrefix is used to detect CDATA text when ReadSettings.PreserveCData is
|
||||
// true.
|
||||
var cdataPrefix = []byte("<![CDATA[")
|
||||
@@ -76,8 +81,18 @@ type ReadSettings struct {
|
||||
// whether an end element is present. Commonly set to xml.HTMLAutoClose.
|
||||
// Default: nil.
|
||||
AutoClose []string
|
||||
|
||||
// MaxDepth is the maximum depth of the XML tree to parse. If the depth of
|
||||
// the XML tree exceeds this value, all ReadFrom* functions return the
|
||||
// error ErrMaxDepth. If MaxDepth is zero or negative, a depth limit of
|
||||
// 1024 is used. Default: 0 (i.e., a limit of 1024).
|
||||
MaxDepth int
|
||||
}
|
||||
|
||||
// defaultMaxDepth is the maximum depth of an XML tree parsed by ReadFrom*
|
||||
// functions when ReadSettings.MaxDepth is not set to a positive value.
|
||||
const defaultMaxDepth = 1024
|
||||
|
||||
// defaultCharsetReader is used by the xml decoder when the ReadSettings
|
||||
// CharsetReader value is nil. It behaves as a "pass-through", ignoring
|
||||
// the requested charset parameter and skipping conversion altogether.
|
||||
@@ -87,18 +102,9 @@ func defaultCharsetReader(charset string, input io.Reader) (io.Reader, error) {
|
||||
|
||||
// dup creates a duplicate of the ReadSettings object.
|
||||
func (s *ReadSettings) dup() ReadSettings {
|
||||
var entityCopy map[string]string
|
||||
if s.Entity != nil {
|
||||
entityCopy = make(map[string]string)
|
||||
for k, v := range s.Entity {
|
||||
entityCopy[k] = v
|
||||
}
|
||||
}
|
||||
return ReadSettings{
|
||||
CharsetReader: s.CharsetReader,
|
||||
Permissive: s.Permissive,
|
||||
Entity: entityCopy,
|
||||
}
|
||||
c := *s
|
||||
c.Entity = maps.Clone(s.Entity)
|
||||
return c
|
||||
}
|
||||
|
||||
// WriteSettings determine the behavior of the Document's WriteTo* functions.
|
||||
@@ -913,6 +919,11 @@ func (e *Element) readFrom(ri io.Reader, settings ReadSettings) (n int64, err er
|
||||
attrCheck := make(map[xml.Name]int)
|
||||
dec := newDecoder(r, settings)
|
||||
|
||||
maxDepth := settings.MaxDepth
|
||||
if maxDepth <= 0 {
|
||||
maxDepth = defaultMaxDepth
|
||||
}
|
||||
|
||||
var stack stack[*Element]
|
||||
stack.push(e)
|
||||
for {
|
||||
@@ -942,6 +953,9 @@ func (e *Element) readFrom(ri io.Reader, settings ReadSettings) (n int64, err er
|
||||
|
||||
switch t := t.(type) {
|
||||
case xml.StartElement:
|
||||
if len(stack.data) > maxDepth {
|
||||
return r.Bytes(), ErrMaxDepth
|
||||
}
|
||||
e := newElement(t.Name.Space, t.Name.Local, top)
|
||||
if settings.PreserveDuplicateAttrs || len(t.Attr) < 2 {
|
||||
for _, a := range t.Attr {
|
||||
@@ -1622,7 +1636,7 @@ func (c *CharData) Index() int {
|
||||
func (c *CharData) WriteTo(w Writer, s *WriteSettings) {
|
||||
if c.IsCData() {
|
||||
w.WriteString(`<![CDATA[`)
|
||||
w.WriteString(c.Data)
|
||||
sanitizeCData(w, c.Data)
|
||||
w.WriteString(`]]>`)
|
||||
} else {
|
||||
var m escapeMode
|
||||
@@ -1704,7 +1718,7 @@ func (c *Comment) Index() int {
|
||||
// WriteTo serialies the comment to the writer.
|
||||
func (c *Comment) WriteTo(w Writer, s *WriteSettings) {
|
||||
w.WriteString("<!--")
|
||||
w.WriteString(c.Data)
|
||||
sanitizeComment(w, c.Data)
|
||||
w.WriteString("-->")
|
||||
}
|
||||
|
||||
@@ -1769,7 +1783,7 @@ func (d *Directive) Index() int {
|
||||
// WriteTo serializes the XML directive to the writer.
|
||||
func (d *Directive) WriteTo(w Writer, s *WriteSettings) {
|
||||
w.WriteString("<!")
|
||||
w.WriteString(d.Data)
|
||||
sanitizeDirective(w, d.Data)
|
||||
w.WriteString(">")
|
||||
}
|
||||
|
||||
@@ -1837,10 +1851,10 @@ func (p *ProcInst) Index() int {
|
||||
// WriteTo serializes the processing instruction to the writer.
|
||||
func (p *ProcInst) WriteTo(w Writer, s *WriteSettings) {
|
||||
w.WriteString("<?")
|
||||
w.WriteString(p.Target)
|
||||
sanitizeProcInst(w, p.Target)
|
||||
if p.Inst != "" {
|
||||
w.WriteByte(' ')
|
||||
w.WriteString(p.Inst)
|
||||
sanitizeProcInst(w, p.Inst)
|
||||
}
|
||||
w.WriteString("?>")
|
||||
}
|
||||
|
||||
+114
@@ -384,6 +384,120 @@ func escapeString(w Writer, s string, m escapeMode) {
|
||||
w.WriteString(s[last:])
|
||||
}
|
||||
|
||||
// sanitizeCData writes the sanitized contents of a CDATA section to the
|
||||
// writer. XML provides no way to escape the "]]>" sequence within a CDATA
|
||||
// section, so any occurrence of it is split across two CDATA sections.
|
||||
func sanitizeCData(w Writer, s string) {
|
||||
for {
|
||||
i := strings.Index(s, "]]>")
|
||||
if i < 0 {
|
||||
break
|
||||
}
|
||||
w.WriteString(s[:i+2])
|
||||
w.WriteString("]]><![CDATA[")
|
||||
s = s[i+2:]
|
||||
}
|
||||
w.WriteString(s)
|
||||
}
|
||||
|
||||
// sanitizeComment writes the sanitized contents of a comment to the writer.
|
||||
// An XML comment may not contain the string "--", and it may not end with a
|
||||
// '-'. Because XML provides no way to escape these sequences, spaces are
|
||||
// inserted where necessary.
|
||||
func sanitizeComment(w Writer, s string) {
|
||||
last, hyphen := 0, false
|
||||
for i := 0; i < len(s); i++ {
|
||||
if s[i] != '-' {
|
||||
hyphen = false
|
||||
continue
|
||||
}
|
||||
if hyphen {
|
||||
w.WriteString(s[last:i])
|
||||
w.WriteByte(' ')
|
||||
last = i
|
||||
}
|
||||
hyphen = true
|
||||
}
|
||||
w.WriteString(s[last:])
|
||||
if hyphen {
|
||||
w.WriteByte(' ')
|
||||
}
|
||||
}
|
||||
|
||||
// sanitizeProcInst writes the contents of a sanitized processing instruction
|
||||
// to the writer. XML provides no way to escape the "?>" sequence within a
|
||||
// processing instruction, so a space is inserted between the two characters.
|
||||
func sanitizeProcInst(w Writer, s string) {
|
||||
for {
|
||||
i := strings.Index(s, "?>")
|
||||
if i < 0 {
|
||||
break
|
||||
}
|
||||
w.WriteString(s[:i+1])
|
||||
w.WriteByte(' ')
|
||||
s = s[i+1:]
|
||||
}
|
||||
w.WriteString(s)
|
||||
}
|
||||
|
||||
// sanitizeDirective writes the sanitized contents of an XML directive to the
|
||||
// writer.
|
||||
func sanitizeDirective(w Writer, s string) {
|
||||
// The XML decoder reserves the character following "<!" for comments
|
||||
// ('-') and CDATA sections ('['), and it treats "<!>" as an unterminated
|
||||
// directive. Insert a space to avoid conflicts with reserved sequences.
|
||||
scan := s
|
||||
if s == "" || s[0] == '-' || s[0] == '[' {
|
||||
w.WriteByte(' ')
|
||||
} else {
|
||||
scan = s[1:]
|
||||
}
|
||||
|
||||
// A directive's contents may legitimately contain '<' and '>' characters,
|
||||
// so write them without modification when they are balanced.
|
||||
if isDirectiveBalanced(scan) {
|
||||
w.WriteString(s)
|
||||
return
|
||||
}
|
||||
|
||||
// The contents are unbalanced, so escape every character in the string.
|
||||
escapeString(w, s, escapeNormal)
|
||||
}
|
||||
|
||||
// isDirectiveBalanced returns true if the interpreted portion of an XML
|
||||
// directive's contents may be enclosed by "<!" and ">" without changing the
|
||||
// extents of the resulting directive.
|
||||
func isDirectiveBalanced(s string) bool {
|
||||
var quote byte
|
||||
var depth int
|
||||
for i := 0; i < len(s); i++ {
|
||||
switch c := s[i]; {
|
||||
case quote != 0:
|
||||
if c == quote {
|
||||
quote = 0
|
||||
}
|
||||
case c == '\'' || c == '"':
|
||||
quote = c
|
||||
case c == '>':
|
||||
if depth == 0 {
|
||||
return false
|
||||
}
|
||||
depth--
|
||||
case c == '<':
|
||||
if !strings.HasPrefix(s[i+1:], "!--") {
|
||||
depth++
|
||||
break
|
||||
}
|
||||
j := strings.Index(s[i+4:], "-->")
|
||||
if j < 0 {
|
||||
return false
|
||||
}
|
||||
i += 4 + j + 2
|
||||
}
|
||||
}
|
||||
return quote == 0 && depth == 0
|
||||
}
|
||||
|
||||
func isInCharacterRange(r rune) bool {
|
||||
return r == 0x09 ||
|
||||
r == 0x0A ||
|
||||
|
||||
+14
-2
@@ -281,7 +281,11 @@ func (c *compiler) parseSegment(path string) segment {
|
||||
c.err = ErrPath("path has invalid filter [brackets].")
|
||||
break
|
||||
}
|
||||
seg.filters = append(seg.filters, c.parseFilter(fpath[:len(fpath)-1]))
|
||||
filter := c.parseFilter(fpath[:len(fpath)-1])
|
||||
if c.err != ErrPath("") {
|
||||
break
|
||||
}
|
||||
seg.filters = append(seg.filters, filter)
|
||||
}
|
||||
return seg
|
||||
}
|
||||
@@ -320,7 +324,11 @@ func (c *compiler) parseFilter(path string) filter {
|
||||
// Filter contains [@attr='val'], [@attr="val"], [fn()='val'],
|
||||
// [fn()="val"], [tag='val'] or [tag="val"]?
|
||||
eqindex := strings.IndexByte(path, '=')
|
||||
if eqindex >= 0 && eqindex+1 < len(path) {
|
||||
if eqindex == 0 {
|
||||
c.err = ErrPath("path contains a filter expression with no key.")
|
||||
return nil
|
||||
}
|
||||
if eqindex > 0 && eqindex+1 < len(path) {
|
||||
quote := path[eqindex+1]
|
||||
if quote == '\'' || quote == '"' {
|
||||
rindex := nextIndex(path, quote, eqindex+2)
|
||||
@@ -334,6 +342,10 @@ func (c *compiler) parseFilter(path string) filter {
|
||||
|
||||
switch {
|
||||
case key[0] == '@':
|
||||
if len(key) == 1 {
|
||||
c.err = ErrPath("path contains a filter expression with no key.")
|
||||
return nil
|
||||
}
|
||||
return newFilterAttrVal(key[1:], value)
|
||||
case strings.HasSuffix(key, "()"):
|
||||
name := key[:len(key)-2]
|
||||
|
||||
+7
@@ -7,6 +7,12 @@ linters:
|
||||
exclusions:
|
||||
presets:
|
||||
- std-error-handling
|
||||
rules:
|
||||
# Test fixtures construct CDF binary blobs from known small constants, so
|
||||
# gosec's integer-overflow checks (G115) add no value there.
|
||||
- path: internal/cdf/cdf_test\.go
|
||||
linters:
|
||||
- gosec
|
||||
enable:
|
||||
- gosec # Detects security problems.
|
||||
# Keep all extras disabled for now to focus on the integer overflow problem.
|
||||
@@ -31,6 +37,7 @@ linters:
|
||||
- unused
|
||||
- usestdlibvars # Detects the possibility to use variables/constants from the Go standard library.
|
||||
- usetesting # Reports uses of functions with replacement inside the testing package.
|
||||
- asciicheck # https://daniel.haxx.se/blog/2025/05/16/detecting-malicious-unicode/
|
||||
settings:
|
||||
govet:
|
||||
disable:
|
||||
|
||||
+9
-2
@@ -13,8 +13,8 @@
|
||||
<a href="https://pkg.go.dev/github.com/gabriel-vasile/mimetype">
|
||||
<img alt="Go Reference" src="https://pkg.go.dev/badge/github.com/gabriel-vasile/mimetype.svg">
|
||||
</a>
|
||||
<a href="https://goreportcard.com/report/github.com/gabriel-vasile/mimetype">
|
||||
<img alt="Go report card" src="https://goreportcard.com/badge/github.com/gabriel-vasile/mimetype">
|
||||
<a href="https://codecov.io/gh/gabriel-vasile/mimetype">
|
||||
<img alt="Code coverage" src="https://codecov.io/gh/gabriel-vasile/mimetype/graph/badge.svg">
|
||||
</a>
|
||||
<a href="LICENSE">
|
||||
<img alt="License" src="https://img.shields.io/badge/License-MIT-green.svg">
|
||||
@@ -103,3 +103,10 @@ shows which file formats are most often misidentified and can help prioritise.
|
||||
When submitting a PR for detection of a new file format, please make sure to
|
||||
add a record to the list of testcases in [mimetype_test.go](mimetype_test.go).
|
||||
For complex files a record can be added in the [testdata](testdata) directory.
|
||||
Code contributions must respect following rules:
|
||||
- code must be test covered
|
||||
- code must be formatted using the `gofmt` tool
|
||||
- exported names must be documented
|
||||
|
||||
**Important**: By submitting a pull request, you agree to allow the project
|
||||
owner to license your work under the same license as that used by the project.
|
||||
+1
@@ -0,0 +1 @@
|
||||
comment: false
|
||||
+667
@@ -0,0 +1,667 @@
|
||||
// Package cdf implements parsing of CDF (OLE2) files. It is greatly inspired
|
||||
// by src/readcdf.c from libmagic. One difference is this implementation is
|
||||
// permissive of truncated inputs. See readLimit in mimetype.go for the
|
||||
// reason why truncated inputs need to be handled.
|
||||
// http://sc.openoffice.org/compdocfileformat.pdf
|
||||
package cdf
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/binary"
|
||||
|
||||
"github.com/gabriel-vasile/mimetype/internal/scan"
|
||||
)
|
||||
|
||||
type CDFType int8
|
||||
|
||||
const (
|
||||
CDFTypeGeneric CDFType = iota
|
||||
CDFTypeInstaller
|
||||
CDFTypeDoc
|
||||
CDFTypePpt
|
||||
CDFTypeXls
|
||||
CDFTypeMsg
|
||||
)
|
||||
|
||||
// Detect parses raw as a CDF (OLE2) compound file and returns the document type
|
||||
// it contains. It returns CDFTypeGeneric for input that is not a CDF file or
|
||||
// whose type cannot be narrowed down.
|
||||
func Detect(raw []byte) CDFType {
|
||||
if len(raw) < 512 {
|
||||
return CDFTypeGeneric
|
||||
}
|
||||
var c cdf
|
||||
if !parse(raw, &c) {
|
||||
return CDFTypeGeneric
|
||||
}
|
||||
return c.detect()
|
||||
}
|
||||
|
||||
// cdf holds everything we need from a CDF file to do detection.
|
||||
type cdf struct {
|
||||
data []byte
|
||||
secSize int
|
||||
shortSecSize int
|
||||
minStdStream uint32
|
||||
satSecs int32s // list of SAT sector ids; usually a sub-slice of raw input
|
||||
satEntries int // number of valid SAT entries reachable through satSecs
|
||||
firstSSAT int32
|
||||
dirRaw []byte // directory stream bytes (entries are decoded on demand)
|
||||
sst []byte // short-stream pool (root storage's stream)
|
||||
sstBuilt bool // whether sst was already loaded (it is loaded lazily)
|
||||
rootStreamFirst int32 // first sector of the root storage short-stream pool
|
||||
rootStreamSize uint32 // size of the root storage short-stream pool
|
||||
rootStorageUUID []byte
|
||||
}
|
||||
|
||||
// parse reads the entire on-disk structure required for type detection. It
|
||||
// returns true on success and false if the header does not look like a CDF file.
|
||||
// Truncated or partially malformed bodies are tolerated: sector reads degrade
|
||||
// to whatever could be collected so detection can still succeed from partial data.
|
||||
func parse(raw []byte, c *cdf) bool {
|
||||
if len(raw) < 512 || binary.LittleEndian.Uint64(raw) != cdfMagic {
|
||||
return false
|
||||
}
|
||||
secP2 := binary.LittleEndian.Uint16(raw[30:32])
|
||||
shortP2 := binary.LittleEndian.Uint16(raw[32:34])
|
||||
if secP2 > 20 || shortP2 > 20 {
|
||||
return false
|
||||
}
|
||||
c.data = raw
|
||||
c.secSize = 1 << secP2
|
||||
c.shortSecSize = 1 << shortP2
|
||||
c.minStdStream = binary.LittleEndian.Uint32(raw[56:60])
|
||||
if c.secSize < dirEntrySize {
|
||||
return false
|
||||
}
|
||||
firstDirSec := readSecID(raw[48:52])
|
||||
c.firstSSAT = readSecID(raw[60:64])
|
||||
firstMSAT := readSecID(raw[68:72])
|
||||
nMSAT := binary.LittleEndian.Uint32(raw[72:76])
|
||||
masterSAT := int32s{b: raw[76 : 76+4*masterSATSize]}
|
||||
|
||||
c.buildSAT(masterSAT, firstMSAT, nMSAT)
|
||||
c.dirRaw = c.readLong(firstDirSec, 0)
|
||||
|
||||
c.rootStreamFirst = -1
|
||||
var d dirEntry
|
||||
for i, n := 0, c.dirLen(); i < n; i++ {
|
||||
c.dirAt(i, &d)
|
||||
if d.typ != dirTypeRootStorage || d.streamFirst < 0 {
|
||||
continue
|
||||
}
|
||||
c.rootStorageUUID = d.storageUUID[:]
|
||||
// Record where the short-stream pool lives; it is loaded lazily by
|
||||
// shortStream the first time a short stream is actually read.
|
||||
c.rootStreamFirst = d.streamFirst
|
||||
c.rootStreamSize = d.size
|
||||
break
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func (c *cdf) detect() CDFType {
|
||||
for _, name := range []string{"\x05SummaryInformation", "\x05DocumentSummaryInformation"} {
|
||||
if t, ok := c.detectFromSummary(name); ok {
|
||||
return t
|
||||
}
|
||||
}
|
||||
var d dirEntry
|
||||
for i, n := 0, c.dirLen(); i < n; i++ {
|
||||
c.dirAt(i, &d)
|
||||
if t, ok := lookupSection(d.nameBytes(), d.typ); ok {
|
||||
return t
|
||||
}
|
||||
}
|
||||
return CDFTypeGeneric
|
||||
}
|
||||
|
||||
// detectFromSummary inspects a (Doc)SummaryInformation stream and tries to
|
||||
// derive a CDFType from the root-storage CLSID, the property NameOfApplication,
|
||||
// and finally the names of sibling user streams.
|
||||
func (c *cdf) detectFromSummary(streamName string) (CDFType, bool) {
|
||||
if c.rootStorageUUID != nil && bytes.Equal(c.rootStorageUUID, msiCLSID) {
|
||||
return CDFTypeInstaller, true
|
||||
}
|
||||
raw, ok := c.userStream(streamName)
|
||||
if !ok {
|
||||
return CDFTypeGeneric, false
|
||||
}
|
||||
if app := summaryAppName(raw); len(app) > 0 {
|
||||
if t, ok := lookupSubstring(app, app2type); ok {
|
||||
return t, true
|
||||
}
|
||||
}
|
||||
for i, n := 0, c.dirLen(); i < n; i++ {
|
||||
var d dirEntry
|
||||
c.dirAt(i, &d)
|
||||
if d.nameLen == 0 {
|
||||
continue
|
||||
}
|
||||
if t, ok := lookupSubstring(d.nameBytes(), name2type); ok {
|
||||
return t, true
|
||||
}
|
||||
}
|
||||
return CDFTypeGeneric, true
|
||||
}
|
||||
|
||||
const (
|
||||
cdfMagic uint64 = 0xE11AB1A1E011CFD0
|
||||
|
||||
dirTypeUserStorage = 1
|
||||
dirTypeUserStream = 2
|
||||
dirTypeRootStorage = 5
|
||||
|
||||
dirEntrySize = 128
|
||||
masterSATSize = 109 // first 109 SAT secids live in the file header
|
||||
)
|
||||
|
||||
// dirEntry is a single CDF directory record. The UTF-16LE name is pre-decoded
|
||||
// into an inline ASCII buffer at parse time, avoiding a per-entry heap
|
||||
// allocation while keeping comparisons trivial. CDF names are at most 32
|
||||
// UTF-16 code units, so 32 bytes always suffice.
|
||||
type dirEntry struct {
|
||||
name [32]byte
|
||||
nameLen uint8
|
||||
typ uint8
|
||||
streamFirst int32
|
||||
size uint32
|
||||
storageUUID [16]byte
|
||||
}
|
||||
|
||||
// nameBytes returns the decoded ASCII name without copying.
|
||||
func (d *dirEntry) nameBytes() []byte { return d.name[:d.nameLen] }
|
||||
|
||||
func (c *cdf) ssatAt(i int32) int32 {
|
||||
for sid := c.firstSSAT; sid >= 0; {
|
||||
if int(sid) >= c.satLen() {
|
||||
break // SAT is truncated; stop collecting
|
||||
}
|
||||
buf, ok := c.sector(sid)
|
||||
if !ok {
|
||||
break
|
||||
}
|
||||
lbuf := int32(len(buf) / 4) //nolint:gosec // anything divided by 4 fits int32
|
||||
if i < lbuf {
|
||||
return int32(binary.LittleEndian.Uint32(buf[4*i:])) //nolint:gosec // intentional two's-complement reinterpretation of a sector id
|
||||
}
|
||||
i -= lbuf
|
||||
sid = c.satAt(sid)
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
// shortStream returns the root storage short-stream pool, loading it on first
|
||||
// use. Detection often finishes (e.g. via the root CLSID or a long-stream
|
||||
// summary) without ever reading a short stream, so building this eagerly would
|
||||
// be wasted work.
|
||||
func (c *cdf) shortStream() []byte {
|
||||
if !c.sstBuilt {
|
||||
c.sstBuilt = true
|
||||
if c.rootStreamFirst >= 0 {
|
||||
c.sst = c.readLong(c.rootStreamFirst, c.rootStreamSize)
|
||||
}
|
||||
}
|
||||
return c.sst
|
||||
}
|
||||
|
||||
// int32s works like a slice of LE int32 and is backed by a slice of bytes.
|
||||
// int32s could very well be type int32s []byte, but that would mean
|
||||
// len function can be called on it. We don't want that, we always want to use
|
||||
// the len method.
|
||||
type int32s struct {
|
||||
b []byte
|
||||
}
|
||||
|
||||
func (b int32s) at(i int) int32 {
|
||||
//nolint:gosec // intentional two's-complement reinterpretation of a sector id
|
||||
return int32(binary.LittleEndian.Uint32(b.b[4*i:]))
|
||||
}
|
||||
func (b int32s) len() int {
|
||||
return len(b.b) / 4
|
||||
}
|
||||
|
||||
// readSecID reinterprets four little-endian bytes as a signed sector id.
|
||||
// Every 32-bit pattern is a valid id (values >= 0 are sector numbers,
|
||||
// negatives are CDF sentinels such as -2 end-of-chain), so the conversion is
|
||||
// an intentional two's-complement reinterpretation rather than an overflow.
|
||||
func readSecID(b []byte) int32 {
|
||||
return int32(binary.LittleEndian.Uint32(b)) //nolint:gosec // intentional two's-complement reinterpretation
|
||||
}
|
||||
|
||||
// satLen is the number of sector ids reachable through the SAT.
|
||||
func (c *cdf) satLen() int { return c.satEntries }
|
||||
|
||||
// satAt returns the i-th sector id from the SAT. Callers must ensure
|
||||
// i < satLen(). The SAT is not materialized; the entry is fetched directly
|
||||
// from the input by translating i into (SAT sector index, entry offset).
|
||||
func (c *cdf) satAt(i int32) int32 {
|
||||
perSec := c.secSize / 4
|
||||
secIdx := int(i) / perSec
|
||||
entryIdx := int(i) % perSec
|
||||
secID := c.satSecs.at(secIdx)
|
||||
off := c.secSize*(1+int(secID)) + 4*entryIdx
|
||||
return readSecID(c.data[off:])
|
||||
}
|
||||
|
||||
// sector returns the bytes of long sector secid. If the file is truncated
|
||||
// inside the requested sector the result is the available bytes (no padding).
|
||||
// If the sector starts past EOF or secid is negative, then ok is false.
|
||||
func (c *cdf) sector(secid int32) (_ []byte, ok bool) {
|
||||
if secid < 0 {
|
||||
return nil, false
|
||||
}
|
||||
off := int64(c.secSize) * (1 + int64(secid))
|
||||
if off >= int64(len(c.data)) {
|
||||
return nil, false
|
||||
}
|
||||
// The returned sector might be truncated,
|
||||
// but we still return it as best effort.
|
||||
end := min(off+int64(c.secSize), int64(len(c.data)))
|
||||
// If not even one int32 fits, then fail.
|
||||
if end-off < 4 {
|
||||
return nil, false
|
||||
}
|
||||
return c.data[off:end], true
|
||||
}
|
||||
|
||||
func (c *cdf) sectorIDs(secid int32) (int32s, bool) {
|
||||
buf, ok := c.sector(secid)
|
||||
if !ok {
|
||||
return int32s{}, ok
|
||||
}
|
||||
return int32s{b: buf}, true
|
||||
}
|
||||
|
||||
// buildSAT records the list of SAT sector ids from the master-SAT (header)
|
||||
// plus any extension blocks chained via firstMSAT. The SAT itself is not
|
||||
// materialized: satAt computes the requested entry directly from c.data via
|
||||
// satSecs. In the common case (no extension chain) satSecs is a zero-copy
|
||||
// sub-slice of the input header.
|
||||
func (c *cdf) buildSAT(masterSAT int32s, firstMSAT int32, nMSAT uint32) {
|
||||
// Fast path: no extension chain. masterSAT is already a sub-slice of raw
|
||||
// input; reuse it directly.
|
||||
if firstMSAT < 0 || nMSAT == 0 {
|
||||
c.satSecs = masterSAT
|
||||
c.satEntries = c.computeSATLen()
|
||||
return
|
||||
}
|
||||
|
||||
// Slow path: gather sector ids from the header plus the extension chain
|
||||
// into a fresh buffer. Even here we only allocate space for ids (4 bytes
|
||||
// each), not the full SAT contents.
|
||||
maxIDs := len(c.data)/c.secSize + 1
|
||||
buf := make([]byte, 0, 4*masterSATSize)
|
||||
for i := 0; i < masterSAT.len(); i++ {
|
||||
if masterSAT.at(i) < 0 {
|
||||
break
|
||||
}
|
||||
buf = append(buf, masterSAT.b[4*i:4*i+4]...)
|
||||
}
|
||||
perSec := c.secSize/4 - 1
|
||||
mid := firstMSAT
|
||||
chain:
|
||||
for j := uint32(0); j < nMSAT && mid >= 0; j++ {
|
||||
msa, ok := c.sectorIDs(mid)
|
||||
if !ok {
|
||||
break
|
||||
}
|
||||
for k := 0; k < perSec; k++ {
|
||||
if k >= msa.len() || msa.at(k) < 0 {
|
||||
break chain
|
||||
}
|
||||
buf = append(buf, msa.b[4*k:4*k+4]...)
|
||||
if len(buf)/4 > maxIDs {
|
||||
break chain // cyclic MSAT chain; stop allocating
|
||||
}
|
||||
}
|
||||
if perSec >= msa.len() {
|
||||
break // no next-MSAT pointer available
|
||||
}
|
||||
mid = msa.at(perSec)
|
||||
}
|
||||
c.satSecs = int32s{b: buf}
|
||||
c.satEntries = c.computeSATLen()
|
||||
}
|
||||
|
||||
// computeSATLen walks satSecs and counts how many SAT entries are actually
|
||||
// reachable in c.data, stopping at the first sentinel id or sector that is not
|
||||
// fully present in the file.
|
||||
func (c *cdf) computeSATLen() int {
|
||||
perSec := c.secSize / 4
|
||||
total := 0
|
||||
for i := 0; i < c.satSecs.len(); i++ {
|
||||
sec := c.satSecs.at(i)
|
||||
if sec < 0 {
|
||||
break
|
||||
}
|
||||
off := int64(c.secSize) * (1 + int64(sec))
|
||||
if off >= int64(len(c.data)) {
|
||||
break
|
||||
}
|
||||
avail := int64(len(c.data)) - off
|
||||
if avail >= int64(c.secSize) {
|
||||
total += perSec
|
||||
continue
|
||||
}
|
||||
total += int(avail / 4)
|
||||
break
|
||||
}
|
||||
return total
|
||||
}
|
||||
|
||||
// readLong reads a long-sector chain starting at sid. If length > 0 the
|
||||
// result is truncated to that many bytes. On truncation or any other failure
|
||||
// it returns whatever sectors were readable.
|
||||
func (c *cdf) readLong(sid int32, length uint32) []byte {
|
||||
// Fast path: when the chain is a single physically contiguous run of
|
||||
// sectors (the common case for the directory and summary streams) the data
|
||||
// is already laid out sequentially in the input, so return a sub-slice of
|
||||
// it instead of allocating a buffer and copying every sector.
|
||||
if sid >= 0 {
|
||||
maxSec := len(c.data)/c.secSize + 1
|
||||
n, s := 0, sid
|
||||
contiguous := true
|
||||
for s >= 0 {
|
||||
if int(s) >= c.satLen() {
|
||||
break // SAT truncated; what remains is still contiguous
|
||||
}
|
||||
n++
|
||||
if n > maxSec {
|
||||
contiguous = false // cyclic chain; let the slow path guard it
|
||||
break
|
||||
}
|
||||
next := c.satAt(s)
|
||||
if next >= 0 && int64(next) != int64(s)+1 {
|
||||
contiguous = false
|
||||
break
|
||||
}
|
||||
s = next
|
||||
}
|
||||
if contiguous {
|
||||
off64 := int64(c.secSize) * (1 + int64(sid))
|
||||
if off64 >= int64(len(c.data)) {
|
||||
return nil
|
||||
}
|
||||
end64 := min(off64+int64(n)*int64(c.secSize), int64(len(c.data)))
|
||||
out := c.data[off64:end64]
|
||||
if length > 0 && int64(length) < int64(len(out)) {
|
||||
out = out[:length]
|
||||
}
|
||||
return out
|
||||
}
|
||||
}
|
||||
|
||||
// Slow path: gather a fragmented chain into a fresh buffer. Real-world
|
||||
// writers (MSI builders, edited Office documents) routinely produce
|
||||
// non-contiguous directory and stream chains, so this fallback is required
|
||||
// for correct detection on those files.
|
||||
maxBytes := len(c.data)
|
||||
out := make([]byte, 0, c.secSize)
|
||||
for sid >= 0 {
|
||||
if int(sid) >= c.satLen() {
|
||||
break // SAT truncated; return what we have
|
||||
}
|
||||
buf, ok := c.sector(sid)
|
||||
if !ok {
|
||||
break
|
||||
}
|
||||
out = append(out, buf...)
|
||||
if len(out) >= maxBytes {
|
||||
break // chain longer than the file: cyclic SAT, stop
|
||||
}
|
||||
sid = c.satAt(sid)
|
||||
}
|
||||
if length > 0 && int64(length) < int64(len(out)) {
|
||||
out = out[:length]
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// readShort reads a short-sector chain at sid by indexing into the short-stream
|
||||
// pool. On truncation or if the pool is unavailable it returns whatever was
|
||||
// readable (possibly nil).
|
||||
func (c *cdf) readShort(sid int32, length uint32) []byte {
|
||||
sst := c.shortStream()
|
||||
if sst == nil {
|
||||
return nil
|
||||
}
|
||||
// TODO: anyway to avoid allocating and copying the bytes?
|
||||
out := make([]byte, 0, c.shortSecSize)
|
||||
for sid >= 0 {
|
||||
off64 := int64(sid) * int64(c.shortSecSize)
|
||||
if off64+int64(c.shortSecSize) > int64(len(sst)) {
|
||||
break // short-stream pool truncated or sid out of range
|
||||
}
|
||||
off := int(off64)
|
||||
out = append(out, sst[off:off+c.shortSecSize]...)
|
||||
if len(out) >= len(sst) {
|
||||
break // chain longer than the pool: cyclic SSAT, stop
|
||||
}
|
||||
sid = c.ssatAt(sid)
|
||||
}
|
||||
if length > 0 && int64(length) < int64(len(out)) {
|
||||
out = out[:length]
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// readChain dispatches to the long or short reader depending on stream size.
|
||||
func (c *cdf) readChain(sid int32, length uint32) []byte {
|
||||
if length < c.minStdStream && c.rootStreamFirst >= 0 {
|
||||
return c.readShort(sid, length)
|
||||
}
|
||||
return c.readLong(sid, length)
|
||||
}
|
||||
|
||||
// dirLen returns the number of directory entries in dirRaw.
|
||||
func (c *cdf) dirLen() int { return len(c.dirRaw) / dirEntrySize }
|
||||
|
||||
// dirAt decodes the i-th directory entry into *out. Callers must ensure
|
||||
// i < dirLen(). The UTF-16LE name is decoded into out.name, ASCII-style,
|
||||
// stopping at the first NUL.
|
||||
func (c *cdf) dirAt(i int, out *dirEntry) {
|
||||
raw := c.dirRaw[i*dirEntrySize:]
|
||||
nameLen := min(int(binary.LittleEndian.Uint16(raw[64:])), 64)
|
||||
k := uint8(0)
|
||||
for j := 0; j < nameLen/2; j++ {
|
||||
// Names are ASCII; keep the low byte of each little-endian UTF-16
|
||||
// code unit and stop at the first NUL.
|
||||
lo, hi := raw[2*j], raw[2*j+1]
|
||||
if lo == 0 && hi == 0 {
|
||||
break
|
||||
}
|
||||
out.name[k] = lo
|
||||
k++
|
||||
}
|
||||
out.nameLen = k
|
||||
out.typ = raw[66]
|
||||
out.streamFirst = readSecID(raw[116:120])
|
||||
out.size = binary.LittleEndian.Uint32(raw[120:])
|
||||
copy(out.storageUUID[:], raw[80:96])
|
||||
}
|
||||
|
||||
// userStream finds a user stream by name and returns its bytes.
|
||||
func (c *cdf) userStream(name string) ([]byte, bool) {
|
||||
var d dirEntry
|
||||
for i, n := 0, c.dirLen(); i < n; i++ {
|
||||
c.dirAt(i, &d)
|
||||
if d.typ == dirTypeUserStream && string(d.nameBytes()) == name {
|
||||
buf := c.readChain(d.streamFirst, d.size)
|
||||
if buf == nil {
|
||||
return nil, false
|
||||
}
|
||||
return buf, true
|
||||
}
|
||||
}
|
||||
return nil, false
|
||||
}
|
||||
|
||||
const (
|
||||
propIDNameOfApplication = 0x12
|
||||
|
||||
typeMask = 0x0fff
|
||||
typeVector = 0x1000
|
||||
typeStringASCII = 0x1e
|
||||
typeStringWide = 0x1f
|
||||
|
||||
sectionDeclOffset = 0x1c // section declaration in property-set header
|
||||
)
|
||||
|
||||
// summaryAppName parses a (Doc)SummaryInformation stream and returns the
|
||||
// value of property NameOfApplication (0x12) as printable ASCII, or nil if
|
||||
// not present or the stream is malformed. This is the only summary property
|
||||
// the detection logic ever consults.
|
||||
func summaryAppName(stream []byte) []byte {
|
||||
if len(stream) < sectionDeclOffset+20 {
|
||||
return nil
|
||||
}
|
||||
sdOff := binary.LittleEndian.Uint32(stream[sectionDeclOffset+16:])
|
||||
if uint64(sdOff)+8 > uint64(len(stream)) {
|
||||
return nil
|
||||
}
|
||||
section := stream[sdOff:]
|
||||
shLen := binary.LittleEndian.Uint32(section[0:])
|
||||
nProps := binary.LittleEndian.Uint32(section[4:])
|
||||
if uint64(shLen) > uint64(len(section)) || nProps > 1<<16 || 8+8*nProps > shLen {
|
||||
return nil
|
||||
}
|
||||
for i := uint32(0); i < nProps; i++ {
|
||||
base := 8 + 8*i
|
||||
id := binary.LittleEndian.Uint32(section[base:])
|
||||
if id != propIDNameOfApplication {
|
||||
continue
|
||||
}
|
||||
off := binary.LittleEndian.Uint32(section[base+4:])
|
||||
if uint64(off)+8 > uint64(shLen) {
|
||||
return nil
|
||||
}
|
||||
typ := binary.LittleEndian.Uint32(section[off:])
|
||||
if typ&typeVector != 0 {
|
||||
return nil
|
||||
}
|
||||
step := uint32(0)
|
||||
switch typ & typeMask {
|
||||
case typeStringASCII:
|
||||
step = 1
|
||||
case typeStringWide:
|
||||
step = 2
|
||||
default:
|
||||
return nil
|
||||
}
|
||||
slen := binary.LittleEndian.Uint32(section[off+4:])
|
||||
start := uint64(off) + 8
|
||||
end := start + uint64(slen)*uint64(step)
|
||||
if end > uint64(shLen) {
|
||||
return nil
|
||||
}
|
||||
return printableLowBytes(section[start:end], int(step))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// printableLowBytes copies the printable low byte of each step-byte unit
|
||||
// in b, stopping at the first NUL.
|
||||
func printableLowBytes(b []byte, step int) []byte {
|
||||
out := make([]byte, 0, len(b)/step)
|
||||
for i := 0; i+step <= len(b); i += step {
|
||||
c := b[i]
|
||||
if c == 0 {
|
||||
break
|
||||
}
|
||||
if c >= 0x20 && c < 0x7f {
|
||||
out = append(out, c)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// pattern is a case-insensitive substring → CDFType mapping. Entries are
|
||||
// tested in order; first match wins. needle is stored upper-cased so it can be
|
||||
// matched case-insensitively by scan.Bytes.Search with scan.IgnoreCase.
|
||||
type pattern struct {
|
||||
needle []byte
|
||||
typ CDFType
|
||||
}
|
||||
|
||||
// app2type maps NameOfApplication values to CDFTypes.
|
||||
// Mirrors app2mime[] in libmagic. Needles are upper-cased for case-insensitive
|
||||
// matching via scan.IgnoreCase.
|
||||
var app2type = []pattern{
|
||||
{[]byte("WORD"), CDFTypeDoc},
|
||||
{[]byte("EXCEL"), CDFTypeXls},
|
||||
{[]byte("POWERPOINT"), CDFTypePpt},
|
||||
{[]byte("ADVANCED INSTALLER"), CDFTypeInstaller},
|
||||
{[]byte("INSTALLSHIELD"), CDFTypeInstaller},
|
||||
{[]byte("MICROSOFT PATCH COMPILER"), CDFTypeInstaller},
|
||||
{[]byte("NANT"), CDFTypeInstaller},
|
||||
{[]byte("WINDOWS INSTALLER"), CDFTypeInstaller},
|
||||
}
|
||||
|
||||
// name2type maps directory entry names to CDFTypes.
|
||||
// Mirrors name2mime[] in libmagic. Needles are upper-cased for case-insensitive
|
||||
// matching via scan.IgnoreCase.
|
||||
var name2type = []pattern{
|
||||
{[]byte("BOOK"), CDFTypeXls},
|
||||
{[]byte("WORKBOOK"), CDFTypeXls},
|
||||
{[]byte("WORDDOCUMENT"), CDFTypeDoc},
|
||||
{[]byte("POWERPOINT"), CDFTypePpt},
|
||||
{[]byte("DIGITALSIGNATURE"), CDFTypeInstaller},
|
||||
}
|
||||
|
||||
// lookupSubstring returns the CDFType for the first entry in t whose needle
|
||||
// is a case-insensitive substring of v. Mirrors C's strcasestr semantics
|
||||
// under the C locale. It allocates nothing: scan.IgnoreCase matches the
|
||||
// upper-cased needle against input of either case.
|
||||
func lookupSubstring(v []byte, t []pattern) (CDFType, bool) {
|
||||
s := scan.Bytes(v)
|
||||
for _, p := range t {
|
||||
if i, _ := s.Search(p.needle, scan.IgnoreCase); i != -1 {
|
||||
return p.typ, true
|
||||
}
|
||||
}
|
||||
return CDFTypeGeneric, false
|
||||
}
|
||||
|
||||
// msiCLSID is the Microsoft Installer root-storage CLSID, in on-disk byte
|
||||
// order (cdf_directory_t.d_storage_uuid stores two little-endian uint64s).
|
||||
var msiCLSID = []byte{
|
||||
0x84, 0x10, 0x0c, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0xc0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x46,
|
||||
}
|
||||
|
||||
// section is a (directory entry name, type) → CDFType mapping.
|
||||
type section struct {
|
||||
name string
|
||||
typ uint8
|
||||
cdf CDFType
|
||||
}
|
||||
|
||||
// sectionTypes maps distinctive directory entries to CDFTypes — a flattened
|
||||
// equivalent of sectioninfo[] in libmagic. Used as a fallback when no
|
||||
// SummaryInformation stream is present. A slice (rather than a map) lets
|
||||
// lookupSection compare entry names without allocating a string key.
|
||||
var sectionTypes = []section{
|
||||
// libmagic uses application/encrypted, but that is not a registered media type.
|
||||
// For now, we skip identifying that and fall-back on CDFTypeGeneric
|
||||
// {"EncryptedPackage", dirTypeUserStream, CDFTypeEncrypted},
|
||||
// {"EncryptedSummary", dirTypeUserStream, CDFTypeEncrypted},
|
||||
{"Book", dirTypeUserStream, CDFTypeXls},
|
||||
{"Workbook", dirTypeUserStream, CDFTypeXls},
|
||||
{"WordDocument", dirTypeUserStream, CDFTypeDoc},
|
||||
{"PowerPoint Document", dirTypeUserStream, CDFTypePpt},
|
||||
{"__properties_version1.0", dirTypeUserStream, CDFTypeMsg},
|
||||
{"__recip_version1.0_#00000000", dirTypeUserStorage, CDFTypeMsg},
|
||||
}
|
||||
|
||||
// lookupSection returns the CDFType for a directory entry whose name and type
|
||||
// match a sectionTypes entry exactly. The string(name) == comparison is
|
||||
// optimized by the compiler to avoid allocating.
|
||||
func lookupSection(name []byte, typ uint8) (CDFType, bool) {
|
||||
for _, s := range sectionTypes {
|
||||
if s.typ == typ && string(name) == s.name {
|
||||
return s.cdf, true
|
||||
}
|
||||
}
|
||||
return CDFTypeGeneric, false
|
||||
}
|
||||
+1
-21
@@ -84,19 +84,8 @@ func FromPlain(content []byte) string {
|
||||
break
|
||||
}
|
||||
}
|
||||
hasHighBit := false
|
||||
for _, c := range content {
|
||||
if c >= 0x80 {
|
||||
hasHighBit = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if hasHighBit && utf8.Valid(content) {
|
||||
return "utf-8"
|
||||
}
|
||||
|
||||
// ASCII is a subset of UTF8. Follow W3C recommendation and replace with UTF8.
|
||||
if ascii(origContent) {
|
||||
if utf8.Valid(content) {
|
||||
return "utf-8"
|
||||
}
|
||||
|
||||
@@ -123,15 +112,6 @@ func latin(content []byte) string {
|
||||
return "iso-8859-1"
|
||||
}
|
||||
|
||||
func ascii(content []byte) bool {
|
||||
for _, b := range content {
|
||||
if textChars[b] != T {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// FromXML returns the charset of an XML document. It relies on the XML
|
||||
// header <?xml version="1.0" encoding="UTF-8"?> and falls back on the plain
|
||||
// text content.
|
||||
|
||||
+3
-3
@@ -12,10 +12,10 @@ import (
|
||||
type Parser struct {
|
||||
comma byte
|
||||
comment byte
|
||||
s scan.Bytes
|
||||
s *scan.Bytes
|
||||
}
|
||||
|
||||
func NewParser(comma, comment byte, s scan.Bytes) *Parser {
|
||||
func NewParser(comma, comment byte, s *scan.Bytes) *Parser {
|
||||
return &Parser{
|
||||
comma: comma,
|
||||
comment: comment,
|
||||
@@ -55,7 +55,7 @@ func (r *Parser) CountFields(collectIndexes bool) (fields int, fieldPos []int, h
|
||||
if finished {
|
||||
return 0, nil, false
|
||||
}
|
||||
finished = len(r.s) == 0 && len(line) == 0
|
||||
finished = len(*r.s) == 0 && len(line) == 0
|
||||
if len(line) == lengthNL(line) {
|
||||
line = nil
|
||||
continue // Skip empty lines.
|
||||
|
||||
+5
@@ -10,6 +10,7 @@ const (
|
||||
QueryGeo = "geo"
|
||||
QueryHAR = "har"
|
||||
QueryGLTF = "gltf"
|
||||
QueryCDX = "cdx"
|
||||
maxRecursion = 4096
|
||||
)
|
||||
|
||||
@@ -40,6 +41,10 @@ var queries = map[string][]query{
|
||||
SearchPath: [][]byte{[]byte("asset"), []byte("version")},
|
||||
SearchVals: [][]byte{[]byte(`"1.0"`), []byte(`"2.0"`)},
|
||||
}},
|
||||
QueryCDX: {{
|
||||
SearchPath: [][]byte{[]byte("bomFormat")},
|
||||
SearchVals: [][]byte{[]byte(`"CycloneDX"`)},
|
||||
}},
|
||||
}
|
||||
|
||||
var parserPool = sync.Pool{
|
||||
|
||||
+68
-15
@@ -3,6 +3,8 @@ package magic
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/binary"
|
||||
|
||||
"github.com/gabriel-vasile/mimetype/internal/mp3"
|
||||
)
|
||||
|
||||
// Flac matches a Free Lossless Audio Codec file.
|
||||
@@ -51,32 +53,83 @@ func AAC(raw []byte, _ uint32) bool {
|
||||
return len(raw) > 1 && ((raw[0] == 0xFF && raw[1] == 0xF1) || (raw[0] == 0xFF && raw[1] == 0xF9))
|
||||
}
|
||||
|
||||
// Mp3 matches an mp3 file.
|
||||
func Mp3(raw []byte, limit uint32) bool {
|
||||
// MP3 matches a .mp3 file.
|
||||
func MP3(raw []byte, limit uint32) bool {
|
||||
if len(raw) < 3 {
|
||||
return false
|
||||
}
|
||||
|
||||
if bytes.HasPrefix(raw, []byte("ID3")) {
|
||||
// MP3s with an ID3v2 tag will start with "ID3"
|
||||
// ID3v1 tags, however appear at the end of the file.
|
||||
// Any ID3v2 is reported as MP3. Not entirely correct, but the mimesniff
|
||||
// standard says so. https://mimesniff.spec.whatwg.org/#matching-an-audio-or-video-type-pattern
|
||||
// Despite the standard only checking for "ID3", we do more validations to
|
||||
// avoid false positives.
|
||||
if id3v2(raw) {
|
||||
return true
|
||||
}
|
||||
|
||||
// Match MP3 files without tags
|
||||
// If no ID3v2 tag found, then we will look for MP3 frames, but:
|
||||
// a. Layer III files are a lot more prevalent than Layer I and II.
|
||||
// b. Layer I frame header has looser constraints than the others: many files
|
||||
// with regularly repeating 0xFFFF bytes can be misidentified as MP3.
|
||||
// c. MP3 files are composed of individual frames and those frames can have
|
||||
// leading garbage bytes: if we want to find all valid MP3s, we have to do a
|
||||
// linear search. #775, #310
|
||||
// d. There are file formats that contain MP3s inside: .mo3 and .swa
|
||||
//
|
||||
// Given a, b, c and d, this code:
|
||||
// - initially tries to match by first two bytes in header
|
||||
// - checks for .mo3 and .swa and disqualifies them
|
||||
// - does linear search for Layer III
|
||||
switch binary.BigEndian.Uint16(raw[:2]) & 0xFFFE {
|
||||
case 0xFFFA:
|
||||
// MPEG ADTS, layer III, v1
|
||||
return true
|
||||
case 0xFFF2:
|
||||
// MPEG ADTS, layer III, v2
|
||||
return true
|
||||
case 0xFFE2:
|
||||
// MPEG ADTS, layer III, v2.5
|
||||
case 0xFFFA, 0xFFF2, 0xFFE2, // layer III: v1, v2, v2.5
|
||||
0xFFFC, 0xFFF4, // layer II: v1, v2
|
||||
0xFFF5: // layer I: v2
|
||||
return true
|
||||
}
|
||||
// http://lclevy.free.fr/mo3/
|
||||
if bytes.HasPrefix(raw, []byte("MO3")) {
|
||||
return false
|
||||
}
|
||||
|
||||
return false
|
||||
// From PRONOM:
|
||||
// Macromedia licensed the MP3 technology in 1995 to use in their Shockwave
|
||||
// product. .swa or Shockwave Audio was originally added as a free plugin
|
||||
// (Xtras) to SoundEdit 16 to export AIFF files to .swa.
|
||||
// There is no media type assigned for .swa.
|
||||
if bytes.HasPrefix(raw, []byte{0x00, 0x00, 0x01, 0x40, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00}) {
|
||||
return false
|
||||
}
|
||||
|
||||
_, size := mp3.ExtractFrame(raw)
|
||||
return size > 0
|
||||
}
|
||||
|
||||
// Based on https://id3.org/Developer%20Information.
|
||||
func id3v2(raw []byte) bool {
|
||||
if len(raw) < 10 || !bytes.HasPrefix(raw, []byte("ID3")) {
|
||||
return false
|
||||
}
|
||||
if raw[3] < 2 || raw[3] > 4 { // Version: ID3v2.2 - ID3v2.4.
|
||||
return false
|
||||
}
|
||||
if raw[4] != 0 { // Revision is 0 for all versions.
|
||||
return false
|
||||
}
|
||||
|
||||
// v2.2 uses 2 bits, v2.3 uses 3 bits and v2.4 uses 4.
|
||||
// For all versions least significant 4 bits should be 0
|
||||
if raw[5]&0b1111 != 0 {
|
||||
return false
|
||||
}
|
||||
|
||||
// Size bytes are synchsafe: most significant bit always 0.
|
||||
if raw[6]&0x80 != 0 || raw[7]&0x80 != 0 || raw[8]&0x80 != 0 || raw[9]&0x80 != 0 {
|
||||
return false
|
||||
}
|
||||
|
||||
size := uint32(raw[6])<<21 | uint32(raw[7])<<14 | uint32(raw[8])<<7 | uint32(raw[9])
|
||||
// Disallow too big frames, let's say 10MB.
|
||||
return size > 0 && size < 10*1024*1024
|
||||
}
|
||||
|
||||
// Wav matches a Waveform Audio File Format file.
|
||||
|
||||
+62
-7
@@ -4,6 +4,7 @@ import (
|
||||
"bytes"
|
||||
"debug/macho"
|
||||
"encoding/binary"
|
||||
"slices"
|
||||
)
|
||||
|
||||
// Lnk matches Microsoft lnk binary format.
|
||||
@@ -117,13 +118,7 @@ func Dbf(raw []byte, limit uint32) bool {
|
||||
0x02, 0x03, 0x04, 0x05, 0x30, 0x31, 0x32, 0x42, 0x62, 0x7B, 0x82,
|
||||
0x83, 0x87, 0x8A, 0x8B, 0x8E, 0xB3, 0xCB, 0xE5, 0xF5, 0xF4, 0xFB,
|
||||
}
|
||||
for _, b := range dbfTypes {
|
||||
if raw[0] == b {
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
return false
|
||||
return slices.Contains(dbfTypes, raw[0])
|
||||
}
|
||||
|
||||
// ElfObj matches an object file.
|
||||
@@ -229,3 +224,63 @@ func TzIf(raw []byte, limit uint32) bool {
|
||||
// Version has to be NUL (0x00), '2' (0x32) or '3' (0x33).
|
||||
return raw[4] == 0x00 || raw[4] == 0x32 || raw[4] == 0x33
|
||||
}
|
||||
|
||||
// Pyc matches a Python compiled file.
|
||||
// The signatures are sourced from libmagic v5.47
|
||||
func Pyc(raw []byte, limit uint32) bool {
|
||||
if len(raw) < 8 {
|
||||
return false
|
||||
}
|
||||
|
||||
// python 1.0 through 3.7 signatures, magic/Magdir/python:13:190
|
||||
pycMagic := []uint32{
|
||||
0x02099900, 0x03099900, 0x892e0d0a, 0x04170d0a, 0x994e0d0a, 0xfcc40d0a,
|
||||
0xfdc40d0a, 0x87c60d0a, 0x88c60d0a, 0x2aeb0d0a, 0x2beb0d0a, 0x2ded0d0a,
|
||||
0x2eed0d0a, 0x3bf20d0a, 0x3cf20d0a, 0x45f20d0a, 0x59f20d0a, 0x63f20d0a,
|
||||
0x6df20d0a, 0x6ef20d0a, 0x77f20d0a, 0x81f20d0a, 0x8bf20d0a, 0x8cf20d0a,
|
||||
0x95f20d0a, 0x9ff20d0a, 0xa9f20d0a, 0xb3f20d0a, 0xb4f20d0a, 0xc7f20d0a,
|
||||
0xd1f20d0a, 0xd2f20d0a, 0xdbf20d0a, 0xe5f20d0a, 0xeff20d0a, 0xf9f20d0a,
|
||||
0x03f30d0a, 0x04f30d0a, 0x0af30d0a, 0xb80b0d0a, 0xc20b0d0a, 0xcc0b0d0a,
|
||||
0xd60b0d0a, 0xe00b0d0a, 0xea0b0d0a, 0xf40b0d0a, 0xf50b0d0a, 0xff0b0d0a,
|
||||
0x090c0d0a, 0x130c0d0a, 0x1d0c0d0a, 0x1f0c0d0a, 0x270c0d0a, 0x3b0c0d0a,
|
||||
0x450c0d0a, 0x4f0c0d0a, 0x580c0d0a, 0x620c0d0a, 0x6c0c0d0a, 0x760c0d0a,
|
||||
0x800c0d0a, 0x8a0c0d0a, 0x940c0d0a, 0x9e0c0d0a, 0xb20c0d0a, 0xbc0c0d0a,
|
||||
0xc60c0d0a, 0xd00c0d0a, 0xda0c0d0a, 0xe40c0d0a, 0xee0c0d0a, 0xf80c0d0a,
|
||||
0x020d0d0a, 0x0c0d0d0a, 0x160d0d0a, 0x170d0d0a, 0x200d0d0a, 0x210d0d0a,
|
||||
0x2a0d0d0a, 0x2b0d0d0a, 0x2c0d0d0a, 0x2d0d0d0a, 0x2f0d0d0a, 0x300d0d0a,
|
||||
0x310d0d0a, 0x320d0d0a, 0x330d0d0a, 0x3e0d0d0a, 0x3f0d0d0a,
|
||||
}
|
||||
|
||||
n := binary.BigEndian.Uint32(raw)
|
||||
|
||||
if slices.Contains(pycMagic, n) {
|
||||
return true
|
||||
}
|
||||
|
||||
if raw[2] == 0x0d && raw[3] == 0x0a {
|
||||
// Only two bits of flag field are currently used.
|
||||
if l := binary.LittleEndian.Uint32(raw[4:]); l > 3 {
|
||||
return false
|
||||
}
|
||||
if raw[1] == 0x0d || raw[1] == 0x0e {
|
||||
return true
|
||||
}
|
||||
// PyPy magic numbers, magic/Magdir/python:233
|
||||
n := binary.LittleEndian.Uint16(raw)
|
||||
return n == 240 || n == 256 || n == 336 || n == 384 || n == 416
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
|
||||
// Pcap identifies "libpcap" capture files.
|
||||
// https://www.tcpdump.org/manpages/pcap-savefile.5.html
|
||||
func Pcap(raw []byte, _ uint32) bool {
|
||||
if len(raw) < 4 {
|
||||
return false
|
||||
}
|
||||
be := binary.BigEndian.Uint32(raw)
|
||||
le := binary.LittleEndian.Uint32(raw)
|
||||
return be == 0xa1b2c3d4 || be == 0xa1b23c4d ||
|
||||
le == 0xa1b2c3d4 || le == 0xa1b23c4d
|
||||
}
|
||||
+41
-8
@@ -3,6 +3,7 @@ package magic
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/binary"
|
||||
"slices"
|
||||
)
|
||||
|
||||
// Woff matches a Web Open Font Format file.
|
||||
@@ -29,12 +30,13 @@ func Ttf(raw []byte, limit uint32) bool {
|
||||
if !bytes.HasPrefix(raw, []byte{0x00, 0x01, 0x00, 0x00}) {
|
||||
return false
|
||||
}
|
||||
// We cannot rely on the first 4 bytes because of false-positives.
|
||||
// We have to digg deeper into the SFNT tables.
|
||||
return hasSFNTTable(raw)
|
||||
}
|
||||
|
||||
func hasSFNTTable(raw []byte) bool {
|
||||
// 49 possible tables as explained below
|
||||
if len(raw) < 16 || binary.BigEndian.Uint16(raw[4:]) >= 49 {
|
||||
if len(raw) < 16 {
|
||||
return false
|
||||
}
|
||||
|
||||
@@ -87,14 +89,45 @@ func hasSFNTTable(raw []byte) bool {
|
||||
0x6e616d65, // "name"
|
||||
0x6f706264, // "opbd"
|
||||
0x4f532f32, // "OS/2"
|
||||
// The above tables come from the original Apple TTF specification,
|
||||
// but the later Microsoft specification has additional tables.
|
||||
// Common tables: https://learn.microsoft.com/en-us/typography/opentype/spec/otvarcommonformats
|
||||
// Layout tables: https://learn.microsoft.com/en-us/typography/opentype/spec/chapter2
|
||||
// Even if the Microsoft specification says OpenType, the tables are
|
||||
// valid for TrueType as well.
|
||||
0x47535542, // "GSUB"
|
||||
0x47504f53, // "GPOS"
|
||||
0x42415345, // "BASE"
|
||||
0x4a535446, // "JSTF"
|
||||
0x47444546, // "GDEF"
|
||||
0x4d415448, // "MATH"
|
||||
0x43424454, // "CBDT"
|
||||
0x43424c43, // "CBLC"
|
||||
0x43464620, // "CFF "
|
||||
0x43464632, // "CFF2"
|
||||
0x434f4c52, // "COLR"
|
||||
0x4350414c, // "CPAL"
|
||||
0x44534947, // "DSIG"
|
||||
0x45424454, // "EBDT"
|
||||
0x45424c43, // "EBLC"
|
||||
0x48564152, // "HVAR"
|
||||
0x4c545348, // "LTSH"
|
||||
0x4d455247, // "MERG"
|
||||
0x4d564152, // "MVAR"
|
||||
0x50434c54, // "PCLT"
|
||||
0x706f7374, // "post"
|
||||
0x70726570, // "prep"
|
||||
0x73626978, // "sbix"
|
||||
0x53544154, // "STAT"
|
||||
0x53564720, // "SVG "
|
||||
0x56444d58, // "VDMX"
|
||||
0x76686561, // "vhea"
|
||||
0x766d7478, // "vmtx"
|
||||
0x564f5247, // "VORG"
|
||||
0x56564152, // "VVAR"
|
||||
}
|
||||
ourTable := binary.BigEndian.Uint32(raw[12:16])
|
||||
for _, t := range possibleTables {
|
||||
if ourTable == t {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
return slices.Contains(possibleTables, ourTable)
|
||||
}
|
||||
|
||||
// Eot matches an Embedded OpenType font file.
|
||||
|
||||
+2
-7
@@ -3,6 +3,7 @@ package magic
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/binary"
|
||||
"slices"
|
||||
)
|
||||
|
||||
// Shp matches a shape format file.
|
||||
@@ -39,13 +40,7 @@ func Shp(raw []byte, limit uint32) bool {
|
||||
31, // MultiPatch
|
||||
}
|
||||
|
||||
for _, st := range shapeTypes {
|
||||
if st == int(binary.LittleEndian.Uint32(raw[108:112])) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
return false
|
||||
return slices.Contains(shapeTypes, int(binary.LittleEndian.Uint32(raw[108:112])))
|
||||
}
|
||||
|
||||
// Shx matches a shape index format file.
|
||||
|
||||
+27
-1
@@ -4,6 +4,8 @@ import (
|
||||
"bytes"
|
||||
"encoding/binary"
|
||||
"slices"
|
||||
|
||||
"github.com/gabriel-vasile/mimetype/internal/scan"
|
||||
)
|
||||
|
||||
// Png matches a Portable Network Graphics file.
|
||||
@@ -15,7 +17,31 @@ func Png(raw []byte, _ uint32) bool {
|
||||
// Apng matches an Animated Portable Network Graphics file.
|
||||
// https://wiki.mozilla.org/APNG_Specification
|
||||
func Apng(raw []byte, _ uint32) bool {
|
||||
return offset(raw, []byte("acTL"), 37)
|
||||
b := scan.Bytes(raw)
|
||||
b.Advance(8) // the first 8 bytes matched by regular png
|
||||
|
||||
// PNG chunks are composed of:
|
||||
// 4 bytes: length in big endian
|
||||
// 4 bytes: chunk type
|
||||
// length bytes: chunk data
|
||||
// 4 bytes: CRC
|
||||
//
|
||||
// Limit to 32, so we don't waste time on huge inputs.
|
||||
// acTL chunk must come before any IDAT chunks.
|
||||
// https://www.w3.org/TR/png-3/#structure
|
||||
for i := 0; i < 32 && len(b) > 0; i++ {
|
||||
sz, _ := b.Uint32be()
|
||||
if bytes.HasPrefix(b, []byte("acTL")) {
|
||||
return true
|
||||
}
|
||||
if bytes.HasPrefix(b, []byte("IDAT")) {
|
||||
return false
|
||||
}
|
||||
if !b.Advance(int(sz + 8)) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Jpg matches a Joint Photographic Experts Group file.
|
||||
|
||||
+7
-2
@@ -136,6 +136,11 @@ func ftyp(raw []byte, sigs ...[]byte) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
type shebangSig struct {
|
||||
sig []byte
|
||||
flag scan.Flags
|
||||
}
|
||||
|
||||
// A valid shebang starts with the "#!" characters,
|
||||
// followed by any number of spaces,
|
||||
// followed by the path to the interpreter,
|
||||
@@ -146,7 +151,7 @@ func ftyp(raw []byte, sigs ...[]byte) bool {
|
||||
// #! /usr/bin/env php
|
||||
//
|
||||
// /usr/bin/env is the interpreter, php is the first and only argument.
|
||||
func shebang(b scan.Bytes, matchFlags scan.Flags, sigs ...[]byte) bool {
|
||||
func shebang(b scan.Bytes, sigs ...shebangSig) bool {
|
||||
line := b.Line()
|
||||
if len(line) < 2 || line[0] != '#' || line[1] != '!' {
|
||||
return false
|
||||
@@ -154,7 +159,7 @@ func shebang(b scan.Bytes, matchFlags scan.Flags, sigs ...[]byte) bool {
|
||||
line = line[2:]
|
||||
line.TrimLWS()
|
||||
for _, s := range sigs {
|
||||
if line.Match(s, matchFlags) != -1 {
|
||||
if line.Match(s.sig, s.flag) != -1 {
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
+39
-13
@@ -3,6 +3,8 @@ package magic
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/binary"
|
||||
|
||||
"github.com/gabriel-vasile/mimetype/internal/cdf"
|
||||
)
|
||||
|
||||
// Xlsx matches a Microsoft Excel 2007 file.
|
||||
@@ -47,6 +49,15 @@ func Ole(raw []byte, limit uint32) bool {
|
||||
// Doc matches a Microsoft Word 97-2003 file.
|
||||
// See: https://github.com/decalage2/oletools/blob/412ee36ae45e70f42123e835871bac956d958461/oletools/common/clsid.py
|
||||
func Doc(raw []byte, _ uint32) bool {
|
||||
fromParsing := cdf.Detect(raw)
|
||||
if fromParsing == cdf.CDFTypeDoc {
|
||||
return true
|
||||
}
|
||||
if fromParsing != cdf.CDFTypeGeneric {
|
||||
return false
|
||||
}
|
||||
// Fallback for inputs where the CDF directory is past the read limit: match
|
||||
// the root storage CLSID, which often lies within the first sectors.
|
||||
clsids := [][]byte{
|
||||
// Microsoft Word 97-2003 Document (Word.Document.8)
|
||||
{0x06, 0x09, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x46},
|
||||
@@ -55,19 +66,25 @@ func Doc(raw []byte, _ uint32) bool {
|
||||
// Microsoft Word Picture (Word.Picture.8)
|
||||
{0x07, 0x09, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x46},
|
||||
}
|
||||
|
||||
for _, clsid := range clsids {
|
||||
if matchOleClsid(raw, clsid) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
|
||||
// Ppt matches a Microsoft PowerPoint 97-2003 file or a PowerPoint 95 presentation.
|
||||
func Ppt(raw []byte, limit uint32) bool {
|
||||
// Root CLSID test is the safest way to detect identify OLE, however, the format
|
||||
fromParsing := cdf.Detect(raw)
|
||||
if fromParsing == cdf.CDFTypePpt {
|
||||
return true
|
||||
}
|
||||
if fromParsing != cdf.CDFTypeGeneric {
|
||||
return false
|
||||
}
|
||||
// Fallback for inputs where the CDF directory is past the read limit.
|
||||
// Root CLSID test is the safest way to identify the OLE, however, the format
|
||||
// often places the root CLSID at the end of the file.
|
||||
if matchOleClsid(raw, []byte{
|
||||
0x10, 0x8d, 0x81, 0x64, 0x9b, 0x4f, 0xcf, 0x11,
|
||||
@@ -94,18 +111,21 @@ func Ppt(raw []byte, limit uint32) bool {
|
||||
}
|
||||
}
|
||||
|
||||
if bytes.HasPrefix(raw[512:], []byte{0xFD, 0xFF, 0xFF, 0xFF}) &&
|
||||
raw[518] == 0x00 && raw[519] == 0x00 {
|
||||
return true
|
||||
}
|
||||
|
||||
return lin > 1152 && bytes.Contains(raw[1152:min(4096, lin)],
|
||||
[]byte("P\x00o\x00w\x00e\x00r\x00P\x00o\x00i\x00n\x00t\x00 D\x00o\x00c\x00u\x00m\x00e\x00n\x00t"))
|
||||
}
|
||||
|
||||
// Xls matches a Microsoft Excel 97-2003 file.
|
||||
func Xls(raw []byte, limit uint32) bool {
|
||||
// Root CLSID test is the safest way to detect identify OLE, however, the format
|
||||
fromParsing := cdf.Detect(raw)
|
||||
if fromParsing == cdf.CDFTypeXls {
|
||||
return true
|
||||
}
|
||||
if fromParsing != cdf.CDFTypeGeneric {
|
||||
return false
|
||||
}
|
||||
// Fallback for inputs where the CDF directory is past the read limit.
|
||||
// Root CLSID test is the safest way to identify the OLE, however, the format
|
||||
// often places the root CLSID at the end of the file.
|
||||
if matchOleClsid(raw, []byte{
|
||||
0x10, 0x08, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
@@ -148,6 +168,15 @@ func Pub(raw []byte, limit uint32) bool {
|
||||
|
||||
// Msg matches a Microsoft Outlook email file.
|
||||
func Msg(raw []byte, limit uint32) bool {
|
||||
fromParsing := cdf.Detect(raw)
|
||||
if fromParsing == cdf.CDFTypeMsg {
|
||||
return true
|
||||
}
|
||||
if fromParsing != cdf.CDFTypeGeneric {
|
||||
return false
|
||||
}
|
||||
// Fallback for inputs where the CDF directory does not carry the streams the
|
||||
// parser keys on: match the root storage CLSID instead.
|
||||
return matchOleClsid(raw, []byte{
|
||||
0x0B, 0x0D, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0xC0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x46,
|
||||
@@ -157,10 +186,7 @@ func Msg(raw []byte, limit uint32) bool {
|
||||
// Msi matches a Microsoft Windows Installer file.
|
||||
// http://fileformats.archiveteam.org/wiki/Microsoft_Compound_File
|
||||
func Msi(raw []byte, limit uint32) bool {
|
||||
return matchOleClsid(raw, []byte{
|
||||
0x84, 0x10, 0x0C, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0xC0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x46,
|
||||
})
|
||||
return cdf.Detect(raw) == cdf.CDFTypeInstaller
|
||||
}
|
||||
|
||||
// One matches a Microsoft OneNote file.
|
||||
|
||||
+147
-82
@@ -130,6 +130,14 @@ func Xfdf(raw []byte, _ uint32) bool {
|
||||
return xml(raw, xmlSig{[]byte("<xfdf"), []byte(`xmlns="http://ns.adobe.com/xfdf/"`)})
|
||||
}
|
||||
|
||||
// CDXXML matches a CycloneDX XML BOM file.
|
||||
// https://cyclonedx.org/docs/1.7/xml/
|
||||
func CDXXML(raw []byte, _ uint32) bool {
|
||||
// xmlns is missing the version suffix because there are too many past versions
|
||||
// and probably future versions to come.
|
||||
return xml(raw, xmlSig{[]byte("<bom"), []byte(`xmlns="http://cyclonedx.org/schema/bom/`)})
|
||||
}
|
||||
|
||||
// VCard matches a Virtual Contact File.
|
||||
func VCard(raw []byte, _ uint32) bool {
|
||||
return ciPrefix(raw, []byte("BEGIN:VCARD\n"), []byte("BEGIN:VCARD\r\n"))
|
||||
@@ -139,6 +147,14 @@ func VCard(raw []byte, _ uint32) bool {
|
||||
func ICalendar(raw []byte, _ uint32) bool {
|
||||
return ciPrefix(raw, []byte("BEGIN:VCALENDAR\n"), []byte("BEGIN:VCALENDAR\r\n"))
|
||||
}
|
||||
|
||||
const (
|
||||
snone = 0
|
||||
scws = scan.CompactWS
|
||||
sfw = scan.FullWord
|
||||
scwsfw = scan.CompactWS | scan.FullWord
|
||||
)
|
||||
|
||||
func phpPageF(raw []byte, _ uint32) bool {
|
||||
return ciPrefix(raw,
|
||||
[]byte("<?PHP"),
|
||||
@@ -149,66 +165,61 @@ func phpPageF(raw []byte, _ uint32) bool {
|
||||
}
|
||||
func phpScriptF(raw []byte, _ uint32) bool {
|
||||
return shebang(raw,
|
||||
scan.CompactWS,
|
||||
[]byte("/usr/local/bin/php"),
|
||||
[]byte("/usr/bin/php"),
|
||||
[]byte("/usr/bin/env php"),
|
||||
[]byte("/usr/bin/env -S php"),
|
||||
shebangSig{[]byte("/usr/local/bin/php"), snone},
|
||||
shebangSig{[]byte("/usr/bin/php"), snone},
|
||||
shebangSig{[]byte("/usr/bin/env php"), scws},
|
||||
shebangSig{[]byte("/usr/bin/env -S php"), scws},
|
||||
)
|
||||
}
|
||||
|
||||
// Js matches a Javascript file.
|
||||
func Js(raw []byte, _ uint32) bool {
|
||||
return shebang(raw,
|
||||
scan.CompactWS,
|
||||
[]byte("/bin/node"),
|
||||
[]byte("/usr/bin/node"),
|
||||
[]byte("/bin/nodejs"),
|
||||
[]byte("/usr/bin/nodejs"),
|
||||
[]byte("/usr/bin/env node"),
|
||||
[]byte("/usr/bin/env -S node"),
|
||||
[]byte("/usr/bin/env nodejs"),
|
||||
[]byte("/usr/bin/env -S nodejs"),
|
||||
shebangSig{[]byte("/bin/node"), snone},
|
||||
shebangSig{[]byte("/usr/bin/node"), snone},
|
||||
shebangSig{[]byte("/bin/nodejs"), snone},
|
||||
shebangSig{[]byte("/usr/bin/nodejs"), snone},
|
||||
shebangSig{[]byte("/usr/bin/env node"), scws},
|
||||
shebangSig{[]byte("/usr/bin/env -S node"), scws},
|
||||
shebangSig{[]byte("/usr/bin/env nodejs"), scws},
|
||||
shebangSig{[]byte("/usr/bin/env -S nodejs"), scws},
|
||||
)
|
||||
}
|
||||
|
||||
// Lua matches a Lua programming language file.
|
||||
func Lua(raw []byte, _ uint32) bool {
|
||||
return shebang(raw,
|
||||
scan.CompactWS|scan.FullWord,
|
||||
[]byte("/usr/bin/lua"),
|
||||
[]byte("/usr/local/bin/lua"),
|
||||
[]byte("/usr/bin/env lua"),
|
||||
[]byte("/usr/bin/env -S lua"),
|
||||
shebangSig{[]byte("/usr/bin/lua"), sfw},
|
||||
shebangSig{[]byte("/usr/local/bin/lua"), sfw},
|
||||
shebangSig{[]byte("/usr/bin/env lua"), scwsfw},
|
||||
shebangSig{[]byte("/usr/bin/env -S lua"), scwsfw},
|
||||
)
|
||||
}
|
||||
|
||||
// Perl matches a Perl programming language file.
|
||||
func Perl(raw []byte, _ uint32) bool {
|
||||
return shebang(raw,
|
||||
scan.CompactWS|scan.FullWord,
|
||||
[]byte("/usr/bin/perl"),
|
||||
[]byte("/usr/bin/env perl"),
|
||||
[]byte("/usr/bin/env -S perl"),
|
||||
shebangSig{[]byte("/usr/bin/perl"), sfw},
|
||||
shebangSig{[]byte("/usr/bin/env perl"), scwsfw},
|
||||
shebangSig{[]byte("/usr/bin/env -S perl"), scwsfw},
|
||||
)
|
||||
}
|
||||
|
||||
// Python matches a Python programming language file.
|
||||
func Python(raw []byte, _ uint32) bool {
|
||||
return shebang(raw,
|
||||
scan.CompactWS,
|
||||
[]byte("/usr/bin/python"),
|
||||
[]byte("/usr/local/bin/python"),
|
||||
[]byte("/usr/bin/env python"),
|
||||
[]byte("/usr/bin/env -S python"),
|
||||
[]byte("/usr/bin/python2"),
|
||||
[]byte("/usr/local/bin/python2"),
|
||||
[]byte("/usr/bin/env python2"),
|
||||
[]byte("/usr/bin/env -S python2"),
|
||||
[]byte("/usr/bin/python3"),
|
||||
[]byte("/usr/local/bin/python3"),
|
||||
[]byte("/usr/bin/env python3"),
|
||||
[]byte("/usr/bin/env -S python3"),
|
||||
shebangSig{[]byte("/usr/bin/python"), snone},
|
||||
shebangSig{[]byte("/usr/local/bin/python"), snone},
|
||||
shebangSig{[]byte("/usr/bin/env python"), scws},
|
||||
shebangSig{[]byte("/usr/bin/env -S python"), scws},
|
||||
shebangSig{[]byte("/usr/bin/python2"), snone},
|
||||
shebangSig{[]byte("/usr/local/bin/python2"), snone},
|
||||
shebangSig{[]byte("/usr/bin/env python2"), scws},
|
||||
shebangSig{[]byte("/usr/bin/env -S python2"), scws},
|
||||
shebangSig{[]byte("/usr/bin/python3"), snone},
|
||||
shebangSig{[]byte("/usr/local/bin/python3"), snone},
|
||||
shebangSig{[]byte("/usr/bin/env python3"), scws},
|
||||
shebangSig{[]byte("/usr/bin/env -S python3"), scws},
|
||||
)
|
||||
|
||||
}
|
||||
@@ -216,30 +227,28 @@ func Python(raw []byte, _ uint32) bool {
|
||||
// Ruby matches a Ruby programming language file.
|
||||
func Ruby(raw []byte, _ uint32) bool {
|
||||
return shebang(raw,
|
||||
scan.CompactWS,
|
||||
[]byte("/usr/bin/ruby"),
|
||||
[]byte("/usr/local/bin/ruby"),
|
||||
[]byte("/usr/bin/env ruby"),
|
||||
[]byte("/usr/bin/env -S ruby"),
|
||||
shebangSig{[]byte("/usr/bin/ruby"), snone},
|
||||
shebangSig{[]byte("/usr/local/bin/ruby"), snone},
|
||||
shebangSig{[]byte("/usr/bin/env ruby"), scws},
|
||||
shebangSig{[]byte("/usr/bin/env -S ruby"), scws},
|
||||
)
|
||||
}
|
||||
|
||||
// Tcl matches a Tcl programming language file.
|
||||
func Tcl(raw []byte, _ uint32) bool {
|
||||
return shebang(raw,
|
||||
scan.CompactWS,
|
||||
[]byte("/usr/bin/tcl"),
|
||||
[]byte("/usr/local/bin/tcl"),
|
||||
[]byte("/usr/bin/env tcl"),
|
||||
[]byte("/usr/bin/env -S tcl"),
|
||||
[]byte("/usr/bin/tclsh"),
|
||||
[]byte("/usr/local/bin/tclsh"),
|
||||
[]byte("/usr/bin/env tclsh"),
|
||||
[]byte("/usr/bin/env -S tclsh"),
|
||||
[]byte("/usr/bin/wish"),
|
||||
[]byte("/usr/local/bin/wish"),
|
||||
[]byte("/usr/bin/env wish"),
|
||||
[]byte("/usr/bin/env -S wish"),
|
||||
shebangSig{[]byte("/usr/bin/tcl"), snone},
|
||||
shebangSig{[]byte("/usr/local/bin/tcl"), snone},
|
||||
shebangSig{[]byte("/usr/bin/env tcl"), scws},
|
||||
shebangSig{[]byte("/usr/bin/env -S tcl"), scws},
|
||||
shebangSig{[]byte("/usr/bin/tclsh"), snone},
|
||||
shebangSig{[]byte("/usr/local/bin/tclsh"), snone},
|
||||
shebangSig{[]byte("/usr/bin/env tclsh"), scws},
|
||||
shebangSig{[]byte("/usr/bin/env -S tclsh"), scws},
|
||||
shebangSig{[]byte("/usr/bin/wish"), snone},
|
||||
shebangSig{[]byte("/usr/local/bin/wish"), snone},
|
||||
shebangSig{[]byte("/usr/bin/env wish"), scws},
|
||||
shebangSig{[]byte("/usr/bin/env -S wish"), scws},
|
||||
)
|
||||
}
|
||||
|
||||
@@ -251,32 +260,31 @@ func Rtf(raw []byte, _ uint32) bool {
|
||||
// Shell matches a shell script file.
|
||||
func Shell(raw []byte, _ uint32) bool {
|
||||
return shebang(raw,
|
||||
scan.CompactWS|scan.FullWord,
|
||||
[]byte("/bin/sh"),
|
||||
[]byte("/bin/bash"),
|
||||
[]byte("/usr/local/bin/bash"),
|
||||
[]byte("/usr/bin/env bash"),
|
||||
[]byte("/usr/bin/env -S bash"),
|
||||
[]byte("/bin/csh"),
|
||||
[]byte("/usr/local/bin/csh"),
|
||||
[]byte("/usr/bin/env csh"),
|
||||
[]byte("/usr/bin/env -S csh"),
|
||||
[]byte("/bin/dash"),
|
||||
[]byte("/usr/local/bin/dash"),
|
||||
[]byte("/usr/bin/env dash"),
|
||||
[]byte("/usr/bin/env -S dash"),
|
||||
[]byte("/bin/ksh"),
|
||||
[]byte("/usr/local/bin/ksh"),
|
||||
[]byte("/usr/bin/env ksh"),
|
||||
[]byte("/usr/bin/env -S ksh"),
|
||||
[]byte("/bin/tcsh"),
|
||||
[]byte("/usr/local/bin/tcsh"),
|
||||
[]byte("/usr/bin/env tcsh"),
|
||||
[]byte("/usr/bin/env -S tcsh"),
|
||||
[]byte("/bin/zsh"),
|
||||
[]byte("/usr/local/bin/zsh"),
|
||||
[]byte("/usr/bin/env zsh"),
|
||||
[]byte("/usr/bin/env -S zsh"),
|
||||
shebangSig{[]byte("/bin/sh"), sfw},
|
||||
shebangSig{[]byte("/bin/bash"), sfw},
|
||||
shebangSig{[]byte("/usr/local/bin/bash"), sfw},
|
||||
shebangSig{[]byte("/usr/bin/env bash"), scwsfw},
|
||||
shebangSig{[]byte("/usr/bin/env -S bash"), scwsfw},
|
||||
shebangSig{[]byte("/bin/csh"), sfw},
|
||||
shebangSig{[]byte("/usr/local/bin/csh"), sfw},
|
||||
shebangSig{[]byte("/usr/bin/env csh"), scwsfw},
|
||||
shebangSig{[]byte("/usr/bin/env -S csh"), scwsfw},
|
||||
shebangSig{[]byte("/bin/dash"), sfw},
|
||||
shebangSig{[]byte("/usr/local/bin/dash"), sfw},
|
||||
shebangSig{[]byte("/usr/bin/env dash"), scwsfw},
|
||||
shebangSig{[]byte("/usr/bin/env -S dash"), scwsfw},
|
||||
shebangSig{[]byte("/bin/ksh"), sfw},
|
||||
shebangSig{[]byte("/usr/local/bin/ksh"), sfw},
|
||||
shebangSig{[]byte("/usr/bin/env ksh"), scwsfw},
|
||||
shebangSig{[]byte("/usr/bin/env -S ksh"), scwsfw},
|
||||
shebangSig{[]byte("/bin/tcsh"), sfw},
|
||||
shebangSig{[]byte("/usr/local/bin/tcsh"), sfw},
|
||||
shebangSig{[]byte("/usr/bin/env tcsh"), scwsfw},
|
||||
shebangSig{[]byte("/usr/bin/env -S tcsh"), scwsfw},
|
||||
shebangSig{[]byte("/bin/zsh"), sfw},
|
||||
shebangSig{[]byte("/usr/local/bin/zsh"), sfw},
|
||||
shebangSig{[]byte("/usr/bin/env zsh"), scwsfw},
|
||||
shebangSig{[]byte("/usr/bin/env -S zsh"), scwsfw},
|
||||
)
|
||||
}
|
||||
|
||||
@@ -352,6 +360,12 @@ func GLTF(raw []byte, limit uint32) bool {
|
||||
return jsonHelper(raw, limit, json.QueryGLTF, json.TokObject)
|
||||
}
|
||||
|
||||
// CDXJSON matches a CycloneDX JSON BOM file.
|
||||
// https://cyclonedx.org/docs/1.7/json/
|
||||
func CDXJSON(raw []byte, limit uint32) bool {
|
||||
return jsonHelper(raw, limit, json.QueryCDX, json.TokObject)
|
||||
}
|
||||
|
||||
// jsonHelper parses raw and tries to match the q query against it. wantToks
|
||||
// ensures we're not wasting time parsing an input that would not pass anyway,
|
||||
// ex: the input is a valid JSON array, but we're looking for a JSON object.
|
||||
@@ -393,8 +407,16 @@ func NdJSON(raw []byte, limit uint32) bool {
|
||||
var l scan.Bytes
|
||||
for len(s) != 0 {
|
||||
l = s.Line()
|
||||
_, inspected, firstToken, _ := json.Parse(json.QueryNone, l)
|
||||
if len(l) != inspected {
|
||||
parsed, inspected, firstToken, _ := json.Parse(json.QueryNone, l)
|
||||
// Only the last line may be truncated by the read limit; for it, it is
|
||||
// enough that the parser inspected all of it. Every other line must be a
|
||||
// complete, valid JSON document, otherwise a single JSON document spread
|
||||
// over multiple lines would be mistaken for NDJSON. #803
|
||||
if len(s) == 0 {
|
||||
if inspected != len(l) {
|
||||
return false
|
||||
}
|
||||
} else if parsed != len(l) {
|
||||
return false
|
||||
}
|
||||
if firstToken == json.TokArray || firstToken == json.TokObject {
|
||||
@@ -563,6 +585,8 @@ func RFC822(raw []byte, limit uint32) bool {
|
||||
// Some of the hints are IgnoreCase, some not. I selected based on what libmagic
|
||||
// does and based on personal observations from sample files.
|
||||
hints := []rfc822Hint{
|
||||
// Enron dataset has Message-ID, Message-Id and Message-id.
|
||||
{[]byte("Message-ID: "), scan.IgnoreCase},
|
||||
{[]byte("From: "), 0},
|
||||
{[]byte("To: "), 0},
|
||||
{[]byte("CC: "), scan.IgnoreCase},
|
||||
@@ -599,3 +623,44 @@ func lineHasRFC822Hint(b scan.Bytes, hints []rfc822Hint) bool {
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func GEDCOM(raw []byte, limit uint32) bool {
|
||||
// Skip if empty
|
||||
if len(raw) == 0 {
|
||||
return false
|
||||
}
|
||||
|
||||
// GEDCOM header fits within first 4KB
|
||||
searchLimit := min(len(raw), 4096)
|
||||
raw = raw[:searchLimit]
|
||||
|
||||
b := scan.Bytes(raw)
|
||||
|
||||
// Skip BOM if present: UTF-8, UTF-16BE, UTF-16LE
|
||||
for _, bom := range [][]byte{
|
||||
{0xEF, 0xBB, 0xBF}, // UTF-8
|
||||
{0xFE, 0xFF}, // UTF-16BE
|
||||
{0xFF, 0xFE}, // UTF-16LE
|
||||
} {
|
||||
if bytes.HasPrefix(b, bom) {
|
||||
b.Advance(len(bom))
|
||||
break // Only one BOM can exist at the start
|
||||
}
|
||||
}
|
||||
|
||||
b.TrimLWS()
|
||||
|
||||
firstLine := b.Line()
|
||||
if !bytes.Equal(firstLine, []byte("0 HEAD")) {
|
||||
return false
|
||||
}
|
||||
|
||||
// "1 GEDC" is mandatory in the header
|
||||
for i := 0; i < 10; i++ {
|
||||
line := b.Line()
|
||||
if bytes.Equal(line, []byte("1 GEDC")) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
+17
-4
@@ -17,8 +17,7 @@ func TSV(raw []byte, limit uint32) bool {
|
||||
|
||||
func sv(in []byte, comma byte, limit uint32) bool {
|
||||
s := scan.Bytes(in)
|
||||
s.DropLastLine(limit)
|
||||
r := csv.NewParser(comma, '#', s)
|
||||
r := csv.NewParser(comma, '#', &s)
|
||||
|
||||
headerFields, _, hasMore := r.CountFields(false)
|
||||
if headerFields < 2 || !hasMore {
|
||||
@@ -30,8 +29,22 @@ func sv(in []byte, comma byte, limit uint32) bool {
|
||||
if !hasMore && fields == 0 {
|
||||
break
|
||||
}
|
||||
csvLines++
|
||||
if fields != headerFields {
|
||||
if fields == headerFields {
|
||||
csvLines++
|
||||
} else {
|
||||
// maybeTruncated signals the input was cut at the read limit,
|
||||
// meaning the last line may be an incomplete CSV record.
|
||||
maybeTruncated := limit > 0 && uint64(len(in)) >= uint64(limit)
|
||||
if maybeTruncated && fields < headerFields {
|
||||
// Allow the last row to have any number of fields
|
||||
// if the input is maybeTruncated.
|
||||
// BUG: if len(input) == limit, then the input is not truncated
|
||||
// but it is still allowed to have the wrong number of fields
|
||||
// and it will be reported as valid CSV.
|
||||
if len(s) == 0 {
|
||||
break
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
if csvLines >= 10 {
|
||||
|
||||
+2
-4
@@ -49,10 +49,8 @@ func isMatroskaFileTypeMatched(in []byte, flType string) bool {
|
||||
// The logic of search is: find first instance of \x42\x82 and then
|
||||
// search for given string after n bytes of above instance.
|
||||
func isFileTypeNamePresent(in []byte, flType string) bool {
|
||||
ind, maxInd, lenIn := 0, 4096, len(in)
|
||||
if lenIn < maxInd { // restricting length to 4096
|
||||
maxInd = lenIn
|
||||
}
|
||||
ind, lenIn := 0, len(in)
|
||||
maxInd := min(4096, lenIn)
|
||||
ind = bytes.Index(in[:maxInd], []byte("\x42\x82"))
|
||||
if ind > 0 && lenIn > ind+2 {
|
||||
ind += 2
|
||||
|
||||
+79
-4
@@ -187,11 +187,11 @@ func msoxml(raw scan.Bytes, searchFor zipEntries, stopAfter int) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
var zipLocalFileHeader = []byte("PK\003\004")
|
||||
|
||||
// next extracts the name of the next zip entry.
|
||||
func (i *zipIterator) next() []byte {
|
||||
pk := []byte("PK\003\004")
|
||||
|
||||
n := bytes.Index(i.b, pk)
|
||||
n := bytes.Index(i.b, zipLocalFileHeader)
|
||||
if n == -1 {
|
||||
return nil
|
||||
}
|
||||
@@ -212,10 +212,85 @@ func (i *zipIterator) next() []byte {
|
||||
return i.b[:l]
|
||||
}
|
||||
|
||||
// skipZipflingerEntry tries to detect a Zipflinger virtual entry and skips it.
|
||||
// The detection is based on the following properties:
|
||||
// - compression method is 0
|
||||
// - CRC32 is 0
|
||||
// - compressed size is 0
|
||||
// - uncompressed size is 0
|
||||
// - file name is empty
|
||||
// Returns true if it was found and skipped.
|
||||
func (i *zipIterator) skipZipflingerEntry() (skipped bool) {
|
||||
// Make a backup of the data so the inspection does not loses it.
|
||||
b := i.b
|
||||
defer func() {
|
||||
// If no zipflinger was found, restore the original data.
|
||||
if !skipped {
|
||||
i.b = b
|
||||
}
|
||||
}()
|
||||
|
||||
n := bytes.Index(i.b, zipLocalFileHeader)
|
||||
if n == -1 {
|
||||
return false
|
||||
}
|
||||
if !i.b.Advance(0x08) {
|
||||
return false
|
||||
}
|
||||
|
||||
// Check compression method
|
||||
if cm, ok := i.b.Uint16(); !ok || cm != 0 {
|
||||
return false
|
||||
}
|
||||
|
||||
// Advance up to the CRC32 field
|
||||
if !i.b.Advance(0x04) {
|
||||
return false
|
||||
}
|
||||
|
||||
// Check CRC32
|
||||
if crc32, ok := i.b.Uint32(); !ok || crc32 != 0 {
|
||||
return false
|
||||
}
|
||||
|
||||
// Check compressed size
|
||||
if compressedSize, ok := i.b.Uint32(); !ok || compressedSize != 0 {
|
||||
return false
|
||||
}
|
||||
|
||||
// Check uncompressed size
|
||||
if uncompressedSize, ok := i.b.Uint32(); !ok || uncompressedSize != 0 {
|
||||
return false
|
||||
}
|
||||
|
||||
// Check for empty file name
|
||||
if l, ok := i.b.Uint16(); !ok || l != 0 {
|
||||
return false
|
||||
}
|
||||
|
||||
// Reached a zipflinger virtual entry: skip extra data
|
||||
l, ok := i.b.Uint16()
|
||||
if !ok {
|
||||
return false
|
||||
}
|
||||
|
||||
if !i.b.Advance(int(l)) {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// APK matches an Android Package Archive.
|
||||
// The source of signatures is https://github.com/file/file/blob/1778642b8ba3d947a779a36fcd81f8e807220a19/magic/Magdir/archive#L1820-L1887
|
||||
func APK(raw []byte, _ uint32) bool {
|
||||
return zipHas(raw, zipEntries{{
|
||||
iter := zipIterator{raw}
|
||||
|
||||
// If a Zipflinger Virtual Entry is detected, then the data is considered APK
|
||||
if iter.skipZipflingerEntry() {
|
||||
return true
|
||||
}
|
||||
|
||||
return zipHas(iter.b, zipEntries{{
|
||||
name: []byte("AndroidManifest.xml"),
|
||||
}, {
|
||||
name: []byte("META-INF/com/android/build/gradle/app-metadata.properties"),
|
||||
|
||||
+140
@@ -0,0 +1,140 @@
|
||||
package mp3
|
||||
|
||||
import "bytes"
|
||||
|
||||
// minTruncatedSyncMatches is the minimum number of confirmed successive
|
||||
// header matches required to accept a candidate frame when the buffer ends
|
||||
// before maxFrameSyncMatches confirmations can be performed.
|
||||
const minTruncatedSyncMatches = 2
|
||||
|
||||
func ExtractFrame(b []byte) (start, size int) {
|
||||
limit := min(len(b), 2048+headerSize)
|
||||
for i := 0; i < limit-headerSize; i++ {
|
||||
j := bytes.IndexByte(b[i:limit-headerSize], 0xFF)
|
||||
if j < 0 {
|
||||
break
|
||||
}
|
||||
i += j
|
||||
hdr := header{b[i], b[i+1], b[i+2], b[i+3]}
|
||||
if !hdr.valid() {
|
||||
continue
|
||||
}
|
||||
frameBytes := hdr.frameBytes()
|
||||
frameAndPad := frameBytes + hdr.padding()
|
||||
|
||||
validHere := frameBytes > 0 && i+frameAndPad <= len(b) && matchFrame(b[i:])
|
||||
// When the buffer is exactly one frame, matchFrame cannot look ahead for
|
||||
// a subsequent header to confirm the stream. Trust the validated header.
|
||||
exact := i == 0 && frameAndPad == len(b)
|
||||
if validHere || exact {
|
||||
return i, frameAndPad
|
||||
}
|
||||
}
|
||||
return 0, 0
|
||||
}
|
||||
|
||||
// matchFrame confirms a candidate header by stepping forward and checking that
|
||||
// subsequent headers are consistent.
|
||||
func matchFrame(buf []byte) bool {
|
||||
// maxFrameSyncMatches limits how many valid frames we look at.
|
||||
const maxFrameSyncMatches = 10
|
||||
hdr := header{buf[0], buf[1], buf[2], buf[3]}
|
||||
i := hdr.frameBytes() + hdr.padding()
|
||||
for nmatch := 0; nmatch < maxFrameSyncMatches; nmatch++ {
|
||||
if i+headerSize > len(buf) {
|
||||
return nmatch >= minTruncatedSyncMatches
|
||||
}
|
||||
cmp := header{buf[i], buf[i+1], buf[i+2], buf[i+3]}
|
||||
if !hdr.compatibleWith(cmp) {
|
||||
return false
|
||||
}
|
||||
i += cmp.frameBytes() + cmp.padding()
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
const headerSize = 4
|
||||
|
||||
type header [headerSize]byte
|
||||
|
||||
func (h header) isFreeFormat() bool { return h[2]&0xF0 == 0 }
|
||||
func (h header) isMPEG1() bool { return h[1]&0x8 != 0 }
|
||||
func (h header) isMPEG25() bool { return h[1]&0x10 == 0 }
|
||||
func (h header) rawLayer() byte { return h[1] >> 1 & 3 }
|
||||
func (h header) rawBitrate() byte { return h[2] >> 4 }
|
||||
func (h header) rawSampleRate() byte { return h[2] >> 2 & 3 }
|
||||
func (h header) rawEmphasis() byte { return h[3] & 0b11 }
|
||||
func (h header) isFrame576() bool { return h[1]&14 == 2 }
|
||||
func (h header) padding() int {
|
||||
if h[2]&0x2 != 0 {
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// valid reports whether the four bytes form a syntactically valid MP3 header.
|
||||
func (h header) valid() bool {
|
||||
return h[0] == 0xff &&
|
||||
((h[1]&0xF0) == 0xf0 || (h[1]&0xFE) == 0xe2) &&
|
||||
h.rawLayer() == 1 && // Layer III
|
||||
h.rawBitrate() != 15 && // Not allowed by spec.
|
||||
h.rawSampleRate() != 3 &&
|
||||
h.rawEmphasis() != 2 &&
|
||||
// The code for extracting frame size for free-format is tedious and
|
||||
// free-format MP3s are extinct.
|
||||
!h.isFreeFormat()
|
||||
}
|
||||
|
||||
// compatibleWith reports whether two headers describe frames belonging to the
|
||||
// same MP3 stream — same MPEG version, layer, sample-rate index.
|
||||
func (h header) compatibleWith(o header) bool {
|
||||
return o.valid() &&
|
||||
(h[1]^o[1])&0xFE == 0 &&
|
||||
(h[2]^o[2])&0x0C == 0
|
||||
}
|
||||
|
||||
// bitrateKbps returns the bitrate of the frame in kilobits per second.
|
||||
func (h header) bitrateKbps() int {
|
||||
// halfrate[mpeg1?][bitrate_idx] holds bitrate/2 in kbps.
|
||||
halfrate := [2][15]uint8{
|
||||
{0, 4, 8, 12, 16, 20, 24, 28, 32, 40, 48, 56, 64, 72, 80},
|
||||
{0, 16, 20, 24, 28, 32, 40, 48, 56, 64, 80, 96, 112, 128, 160},
|
||||
}
|
||||
mpeg1 := 0
|
||||
if h.isMPEG1() {
|
||||
mpeg1 = 1
|
||||
}
|
||||
return 2 * int(halfrate[mpeg1][h.rawBitrate()])
|
||||
}
|
||||
|
||||
// sampleRateHz returns the sampling rate of the frame in Hz.
|
||||
func (h header) sampleRateHz() int {
|
||||
base := [3]int{44100, 48000, 32000}[h.rawSampleRate()]
|
||||
if !h.isMPEG1() {
|
||||
base >>= 1
|
||||
}
|
||||
if h.isMPEG25() {
|
||||
base >>= 1
|
||||
}
|
||||
return base
|
||||
}
|
||||
|
||||
// frameSamples returns the number of audio samples per channel encoded in
|
||||
// the frame.
|
||||
func (h header) frameSamples() int {
|
||||
if h.isFrame576() {
|
||||
return 576
|
||||
}
|
||||
return 1152
|
||||
}
|
||||
|
||||
// frameBytes returns the size of the frame body (header + side info + audio
|
||||
// data, excluding padding) in bytes.
|
||||
func (h header) frameBytes() int {
|
||||
br := h.bitrateKbps()
|
||||
sr := h.sampleRateHz()
|
||||
if br == 0 || sr == 0 {
|
||||
return 0
|
||||
}
|
||||
return h.frameSamples() * br * 125 / sr
|
||||
}
|
||||
+21
-25
@@ -122,26 +122,6 @@ func (b *Bytes) Line() Bytes {
|
||||
return line
|
||||
}
|
||||
|
||||
// DropLastLine drops the last incomplete line from b.
|
||||
//
|
||||
// mimetype limits itself to ReadLimit bytes when performing a detection.
|
||||
// This means, for file formats like CSV for NDJSON, the last line of the input
|
||||
// can be an incomplete line.
|
||||
// If b length is less than readLimit, it means we received an incomplete file
|
||||
// and proceed with dropping the last line.
|
||||
func (b *Bytes) DropLastLine(readLimit uint32) {
|
||||
if readLimit == 0 || uint64(len(*b)) < uint64(readLimit) {
|
||||
return
|
||||
}
|
||||
|
||||
for i := len(*b) - 1; i > 0; i-- {
|
||||
if (*b)[i] == '\n' {
|
||||
*b = (*b)[:i]
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (b *Bytes) Uint16() (uint16, bool) {
|
||||
if len(*b) < 2 {
|
||||
return 0, false
|
||||
@@ -151,6 +131,24 @@ func (b *Bytes) Uint16() (uint16, bool) {
|
||||
return v, true
|
||||
}
|
||||
|
||||
func (b *Bytes) Uint32() (uint32, bool) {
|
||||
if len(*b) < 4 {
|
||||
return 0, false
|
||||
}
|
||||
v := binary.LittleEndian.Uint32(*b)
|
||||
*b = (*b)[4:]
|
||||
return v, true
|
||||
}
|
||||
|
||||
func (b *Bytes) Uint32be() (uint32, bool) {
|
||||
if len(*b) < 4 {
|
||||
return 0, false
|
||||
}
|
||||
v := binary.BigEndian.Uint32(*b)
|
||||
*b = (*b)[4:]
|
||||
return v, true
|
||||
}
|
||||
|
||||
type Flags int
|
||||
|
||||
const (
|
||||
@@ -205,10 +203,8 @@ func (b Bytes) Match(p []byte, flags Flags) int {
|
||||
if l == 0 {
|
||||
return -1
|
||||
}
|
||||
// If no flags, or scanning for full word at the end of pattern then
|
||||
// do a fast HasPrefix check.
|
||||
// For other flags it's not possible to use HasPrefix.
|
||||
if flags == 0 || flags&FullWord > 0 {
|
||||
// Some cases we can handle with a simple bytes.HasPrefix.
|
||||
if flags == 0 || flags == FullWord {
|
||||
if bytes.HasPrefix(b, p) {
|
||||
b = b[len(p):]
|
||||
p = p[len(p):]
|
||||
@@ -232,7 +228,7 @@ func (b Bytes) Match(p []byte, flags Flags) int {
|
||||
return -1
|
||||
}
|
||||
b = b[1:]
|
||||
if !ByteIsWS(p[0]) {
|
||||
if len(p) > 0 && !ByteIsWS(p[0]) {
|
||||
b.TrimLWS()
|
||||
}
|
||||
} else {
|
||||
|
||||
+15
-5
@@ -23,6 +23,14 @@ type MIME struct {
|
||||
}
|
||||
|
||||
// String returns the string representation of the MIME type, e.g., "application/zip".
|
||||
// String return values can change between releases, for example, when [IANA]
|
||||
// assigns a new media type. Use [MIME.Is] to avoid breaking changes.
|
||||
//
|
||||
// mtype := mimetype.Detect(zipFile)
|
||||
// if mtype.String() == "application/zip" { /* Plain string comparison is brittle. */ }
|
||||
// if mtype.Is("application/zip") { /* Will continue to work between releases */ }
|
||||
//
|
||||
// [IANA]: https://www.iana.org/assignments/media-types/media-types.xhtml
|
||||
func (m *MIME) String() string {
|
||||
return m.mime
|
||||
}
|
||||
@@ -38,17 +46,19 @@ func (m *MIME) Extension() string {
|
||||
// Each MIME type has a non-nil parent, except for the root MIME type.
|
||||
//
|
||||
// For example, the application/json and text/html MIME types have text/plain as
|
||||
// their parent because they are text files who happen to contain JSON or HTML.
|
||||
// their parent because they are text files that happen to contain JSON or HTML.
|
||||
// Another example is the ZIP format, which is used as container
|
||||
// for Microsoft Office files, EPUB files, JAR files, and others.
|
||||
func (m *MIME) Parent() *MIME {
|
||||
return m.parent
|
||||
}
|
||||
|
||||
// Is checks whether this MIME type, or any of its aliases, is equal to the
|
||||
// Is checks whether this MIME type, or any of its [aliases], is equal to the
|
||||
// expected MIME type. MIME type equality test is done on the "type/subtype"
|
||||
// section, ignores any optional MIME parameters, ignores any leading and
|
||||
// trailing whitespace, and is case insensitive.
|
||||
//
|
||||
// [aliases]: https://github.com/gabriel-vasile/mimetype/blob/master/supported_mimes.md
|
||||
func (m *MIME) Is(expectedMIME string) bool {
|
||||
// Parsing is needed because some detected MIME types contain parameters
|
||||
// that need to be stripped for the comparison.
|
||||
@@ -129,7 +139,7 @@ func (m *MIME) flatten() []*MIME {
|
||||
// hierarchy returns an easy to read list of ancestors for m.
|
||||
// For example, application/json would return json>txt>root.
|
||||
func (m *MIME) hierarchy() string {
|
||||
h := ""
|
||||
var h strings.Builder
|
||||
for m := m; m != nil; m = m.Parent() {
|
||||
e := strings.TrimPrefix(m.Extension(), ".")
|
||||
if e == "" {
|
||||
@@ -142,9 +152,9 @@ func (m *MIME) hierarchy() string {
|
||||
e = "root"
|
||||
}
|
||||
}
|
||||
h += ">" + e
|
||||
h.WriteString(">" + e)
|
||||
}
|
||||
return strings.TrimPrefix(h, ">")
|
||||
return strings.TrimPrefix(h.String(), ">")
|
||||
}
|
||||
|
||||
// clone creates a new MIME with the provided optional MIME parameters.
|
||||
|
||||
+6
-6
@@ -1,6 +1,6 @@
|
||||
// Package mimetype uses magic number signatures to detect the MIME type of a file.
|
||||
//
|
||||
// File formats are stored in a hierarchy with application/octet-stream at its root.
|
||||
// File formats are stored in a hierarchy with "application/octet-stream" at its root.
|
||||
// For example, the hierarchy for HTML format is application/octet-stream ->
|
||||
// text/plain -> text/html.
|
||||
package mimetype
|
||||
@@ -12,14 +12,14 @@ import (
|
||||
"sync/atomic"
|
||||
)
|
||||
|
||||
const defaultLimit uint32 = 3072
|
||||
const defaultLimit uint32 = 4096
|
||||
|
||||
// readLimit is the maximum number of bytes from the input used when detecting.
|
||||
var readLimit uint32 = defaultLimit
|
||||
|
||||
// Detect returns the MIME type found from the provided byte slice.
|
||||
//
|
||||
// The result is always a valid MIME type, with application/octet-stream
|
||||
// The result is always a valid MIME type, with "application/octet-stream"
|
||||
// returned when identification failed.
|
||||
func Detect(in []byte) *MIME {
|
||||
// Using atomic because readLimit can be written at the same time in other goroutine.
|
||||
@@ -34,7 +34,7 @@ func Detect(in []byte) *MIME {
|
||||
|
||||
// DetectReader returns the MIME type of the provided reader.
|
||||
//
|
||||
// The result is always a valid MIME type, with application/octet-stream
|
||||
// The result is always a valid MIME type, with "application/octet-stream"
|
||||
// returned when identification failed with or without an error.
|
||||
// Any error returned is related to the reading from the input reader.
|
||||
//
|
||||
@@ -72,7 +72,7 @@ func DetectReader(r io.Reader) (*MIME, error) {
|
||||
|
||||
// DetectFile returns the MIME type of the provided file.
|
||||
//
|
||||
// The result is always a valid MIME type, with application/octet-stream
|
||||
// The result is always a valid MIME type, with "application/octet-stream"
|
||||
// returned when identification failed with or without an error.
|
||||
// Any error returned is related to the opening and reading from the input file.
|
||||
func DetectFile(path string) (*MIME, error) {
|
||||
@@ -112,7 +112,7 @@ func SetLimit(limit uint32) {
|
||||
}
|
||||
|
||||
// Extend adds detection for other file formats.
|
||||
// It is equivalent to calling Extend() on the root MIME type "application/octet-stream".
|
||||
// It is equivalent to calling [MIME.Extend] on the root MIME type "application/octet-stream".
|
||||
func Extend(detector func(raw []byte, limit uint32) bool, mime, extension string, aliases ...string) {
|
||||
root.Extend(detector, mime, extension, aliases...)
|
||||
}
|
||||
|
||||
+10
-5
@@ -1,4 +1,4 @@
|
||||
## 199 Supported MIME types
|
||||
## 204 Supported MIME types
|
||||
This file is automatically generated when running tests. Do not edit manually.
|
||||
|
||||
Extension | MIME type <br> Aliases | Hierarchy
|
||||
@@ -42,7 +42,7 @@ Extension | MIME type <br> Aliases | Hierarchy
|
||||
**.oga** | **audio/ogg** | oga>ogg>root
|
||||
**.ogv** | **video/ogg** | ogv>ogg>root
|
||||
**.png** | **image/png** | png>root
|
||||
**.png** | **image/vnd.mozilla.apng** | png>png>root
|
||||
**.apng** | **image/apng** <br> image/vnd.mozilla.apng | apng>png>root
|
||||
**.jpg** | **image/jpeg** | jpg>root
|
||||
**.jxl** | **image/jxl** | jxl>root
|
||||
**.jp2** | **image/jp2** | jp2>root
|
||||
@@ -67,7 +67,6 @@ Extension | MIME type <br> Aliases | Hierarchy
|
||||
**.bmp** | **image/bmp** <br> image/x-bmp, image/x-ms-bmp | bmp>root
|
||||
**.123** | **application/vnd.lotus-1-2-3** | 123>root
|
||||
**.ico** | **image/x-icon** | ico>root
|
||||
**.mp3** | **audio/mpeg** <br> audio/x-mpeg, audio/mp3 | mp3>root
|
||||
**.flac** | **audio/flac** | flac>root
|
||||
**.midi** | **audio/midi** <br> audio/mid, audio/sp-midi, audio/x-mid, audio/x-midi | midi>root
|
||||
**.ape** | **audio/ape** | ape>root
|
||||
@@ -95,7 +94,7 @@ Extension | MIME type <br> Aliases | Hierarchy
|
||||
**.webm** | **video/webm** <br> audio/webm | webm>root
|
||||
**.avi** | **video/x-msvideo** <br> video/avi, video/msvideo | avi>root
|
||||
**.flv** | **video/x-flv** | flv>root
|
||||
**.mkv** | **video/x-matroska** | mkv>root
|
||||
**.mkv** | **video/matroska** <br> video/x-matroska | mkv>root
|
||||
**.asf** | **video/x-ms-asf** <br> video/asf, video/x-ms-wmv | asf>root
|
||||
**.aac** | **audio/aac** | aac>root
|
||||
**.voc** | **audio/x-unknown** | voc>root
|
||||
@@ -116,7 +115,7 @@ Extension | MIME type <br> Aliases | Hierarchy
|
||||
**.shp** | **application/vnd.shp** | shp>shx>root
|
||||
**.dbf** | **application/x-dbf** | dbf>root
|
||||
**.dcm** | **application/dicom** | dcm>root
|
||||
**.rar** | **application/x-rar-compressed** <br> application/x-rar | rar>root
|
||||
**.rar** | **application/vnd.rar** <br> application/x-rar-compressed, application/x-rar | rar>root
|
||||
**.djvu** | **image/vnd.djvu** | djvu>root
|
||||
**.mobi** | **application/x-mobipocket-ebook** | mobi>root
|
||||
**.lit** | **application/x-ms-reader** | lit>root
|
||||
@@ -158,6 +157,9 @@ Extension | MIME type <br> Aliases | Hierarchy
|
||||
**.hlp** | **application/x-os2-hlp** | hlp>root
|
||||
**.fm** | **application/vnd.framemaker** | fm>root
|
||||
**.bufr** | **application/bufr** | bufr>root
|
||||
**.pyc** | **application/x-bytecode.python** | pyc>root
|
||||
**.pcap** | **application/vnd.tcpdump.pcap** | pcap>root
|
||||
**.mp3** | **audio/mpeg** <br> audio/x-mpeg, audio/mp3 | mp3>root
|
||||
**.txt** | **text/plain** | txt>root
|
||||
**.svg** | **image/svg+xml** | svg>txt>root
|
||||
**.html** | **text/html** | html>txt>root
|
||||
@@ -176,6 +178,7 @@ Extension | MIME type <br> Aliases | Hierarchy
|
||||
**.xfdf** | **application/vnd.adobe.xfdf** | xfdf>xml>txt>root
|
||||
**.owl** | **application/owl+xml** | owl>xml>txt>root
|
||||
**.html** | **application/xhtml+xml** | html>xml>txt>root
|
||||
**.xml** | **application/vnd.cyclonedx+xml** | xml>xml>txt>root
|
||||
**.php** | **text/x-php** | php>txt>root
|
||||
**.js** | **text/javascript** <br> application/x-javascript, application/javascript | js>txt>root
|
||||
**.lua** | **text/x-lua** | lua>txt>root
|
||||
@@ -186,6 +189,7 @@ Extension | MIME type <br> Aliases | Hierarchy
|
||||
**.geojson** | **application/geo+json** | geojson>json>txt>root
|
||||
**.har** | **application/json** | har>json>txt>root
|
||||
**.gltf** | **model/gltf+json** | gltf>json>txt>root
|
||||
**.json** | **application/vnd.cyclonedx+json** | json>json>txt>root
|
||||
**.ndjson** | **application/x-ndjson** | ndjson>txt>root
|
||||
**.rtf** | **text/rtf** <br> application/rtf | rtf>txt>root
|
||||
**.srt** | **application/x-subrip** <br> application/x-srt, text/x-srt | srt>txt>root
|
||||
@@ -202,3 +206,4 @@ Extension | MIME type <br> Aliases | Hierarchy
|
||||
**.ppm** | **image/x-portable-pixmap** | ppm>txt>root
|
||||
**.pam** | **image/x-portable-arbitrarymap** | pam>txt>root
|
||||
**.eml** | **message/rfc822** | eml>txt>root
|
||||
**.ged** | **text/vnd.familysearch.gedcom** | ged>txt>root
|
||||
+26
-15
@@ -19,12 +19,16 @@ var root = newMIME("application/octet-stream", "",
|
||||
func([]byte, uint32) bool { return true },
|
||||
xpm, sevenZ, zip, pdf, fdf, ole, ps, psd, p7s, ogg, png, jpg, jxl, jp2, jpx,
|
||||
jpm, jxs, gif, webp, exe, elf, ar, tar, xar, bz2, fits, tiff, bmp, lotus, ico,
|
||||
mp3, flac, midi, ape, musePack, amr, wav, aiff, au, mpeg, quickTime, mp4, webM,
|
||||
flac, midi, ape, musePack, amr, wav, aiff, au, mpeg, quickTime, mp4, webM,
|
||||
avi, flv, mkv, asf, aac, voc, m3u, rmvb, gzip, class, swf, crx, ttf, woff,
|
||||
woff2, otf, ttc, eot, wasm, shx, dbf, dcm, rar, djvu, mobi, lit, bpg, cbor,
|
||||
sqlite3, dwg, nes, lnk, macho, qcp, icns, hdr, mrc, mdb, accdb, zstd, cab,
|
||||
rpm, xz, lzip, torrent, cpio, tzif, xcf, pat, gbr, glb, cabIS, jxr, parquet,
|
||||
oneNote, chm, wpd, dxf, grib, zlib, inf, hlp, fm, bufr,
|
||||
oneNote, chm, wpd, dxf, grib, zlib, inf, hlp, fm, bufr, pyc, pcap,
|
||||
// MP3 is late because it does a linear search in the input. That means
|
||||
// containers that embed an MP3, for example: an mp4 file, or a zip without
|
||||
// compression, would pass as MP3s.
|
||||
mp3,
|
||||
// Keep text last because it is the slowest check.
|
||||
text,
|
||||
)
|
||||
@@ -82,16 +86,17 @@ var (
|
||||
alias("application/x-ogg")
|
||||
oggAudio = newMIME("audio/ogg", ".oga", magic.OggAudio)
|
||||
oggVideo = newMIME("video/ogg", ".ogv", magic.OggVideo)
|
||||
text = newMIME("text/plain", ".txt", magic.Text, svg, html, xml, php, js, lua, perl, python, ruby, json, ndJSON, rtf, srt, tcl, csv, tsv, vCard, iCalendar, warc, vtt, shell, netpbm, netpgm, netppm, netpam, rfc822)
|
||||
xml = newMIME("text/xml", ".xml", magic.XML, rss, atom, x3d, kml, xliff, collada, gml, gpx, tcx, amf, threemf, xfdf, owl2, xhtml).
|
||||
text = newMIME("text/plain", ".txt", magic.Text, svg, html, xml, php, js, lua, perl, python, ruby, json, ndJSON, rtf, srt, tcl, csv, tsv, vCard, iCalendar, warc, vtt, shell, netpbm, netpgm, netppm, netpam, rfc822, gedcom)
|
||||
xml = newMIME("text/xml", ".xml", magic.XML, rss, atom, x3d, kml, xliff, collada, gml, gpx, tcx, amf, threemf, xfdf, owl2, xhtml, cdxxml).
|
||||
alias("application/xml")
|
||||
xhtml = newMIME("application/xhtml+xml", ".html", magic.XHTML)
|
||||
json = newMIME("application/json", ".json", magic.JSON, geoJSON, har, gltf)
|
||||
json = newMIME("application/json", ".json", magic.JSON, geoJSON, har, gltf, cdxJSON)
|
||||
har = newMIME("application/json", ".har", magic.HAR)
|
||||
csv = newMIME("text/csv", ".csv", magic.CSV)
|
||||
tsv = newMIME("text/tab-separated-values", ".tsv", magic.TSV)
|
||||
geoJSON = newMIME("application/geo+json", ".geojson", magic.GeoJSON)
|
||||
ndJSON = newMIME("application/x-ndjson", ".ndjson", magic.NdJSON)
|
||||
cdxJSON = newMIME("application/vnd.cyclonedx+json", ".json", magic.CDXJSON)
|
||||
html = newMIME("text/html", ".html", magic.HTML)
|
||||
php = newMIME("text/x-php", ".php", magic.Php)
|
||||
rtf = newMIME("text/rtf", ".rtf", magic.Rtf).alias("application/rtf")
|
||||
@@ -104,6 +109,7 @@ var (
|
||||
perl = newMIME("text/x-perl", ".pl", magic.Perl)
|
||||
python = newMIME("text/x-python", ".py", magic.Python).
|
||||
alias("text/x-script.python", "application/x-python")
|
||||
pyc = newMIME("application/x-bytecode.python", ".pyc", magic.Pyc)
|
||||
ruby = newMIME("text/x-ruby", ".rb", magic.Ruby).
|
||||
alias("application/x-ruby")
|
||||
shell = newMIME("text/x-shellscript", ".sh", magic.Shell).
|
||||
@@ -127,13 +133,15 @@ var (
|
||||
tcx = newMIME("application/vnd.garmin.tcx+xml", ".tcx", magic.Tcx)
|
||||
amf = newMIME("application/x-amf", ".amf", magic.Amf)
|
||||
threemf = newMIME("application/vnd.ms-package.3dmanufacturing-3dmodel+xml", ".3mf", magic.Threemf)
|
||||
cdxxml = newMIME("application/vnd.cyclonedx+xml", ".xml", magic.CDXXML)
|
||||
png = newMIME("image/png", ".png", magic.Png, apng)
|
||||
apng = newMIME("image/vnd.mozilla.apng", ".png", magic.Apng)
|
||||
jpg = newMIME("image/jpeg", ".jpg", magic.Jpg)
|
||||
jxl = newMIME("image/jxl", ".jxl", magic.Jxl)
|
||||
jp2 = newMIME("image/jp2", ".jp2", magic.Jp2)
|
||||
jpx = newMIME("image/jpx", ".jpf", magic.Jpx)
|
||||
jpm = newMIME("image/jpm", ".jpm", magic.Jpm).
|
||||
apng = newMIME("image/apng", ".apng", magic.Apng).
|
||||
alias("image/vnd.mozilla.apng")
|
||||
jpg = newMIME("image/jpeg", ".jpg", magic.Jpg)
|
||||
jxl = newMIME("image/jxl", ".jxl", magic.Jxl)
|
||||
jp2 = newMIME("image/jp2", ".jp2", magic.Jp2)
|
||||
jpx = newMIME("image/jpx", ".jpf", magic.Jpx)
|
||||
jpm = newMIME("image/jpm", ".jpm", magic.Jpm).
|
||||
alias("video/jpm")
|
||||
jxs = newMIME("image/jxs", ".jxs", magic.Jxs)
|
||||
xpm = newMIME("image/x-xpixmap", ".xpm", magic.Xpm)
|
||||
@@ -156,7 +164,7 @@ var (
|
||||
heifSeq = newMIME("image/heif-sequence", ".heif", magic.HeifSequence)
|
||||
hdr = newMIME("image/vnd.radiance", ".hdr", magic.Hdr)
|
||||
avif = newMIME("image/avif", ".avif", magic.AVIF)
|
||||
mp3 = newMIME("audio/mpeg", ".mp3", magic.Mp3).
|
||||
mp3 = newMIME("audio/mpeg", ".mp3", magic.MP3).
|
||||
alias("audio/x-mpeg", "audio/mp3")
|
||||
flac = newMIME("audio/flac", ".flac", magic.Flac)
|
||||
midi = newMIME("audio/midi", ".midi", magic.Midi).
|
||||
@@ -192,7 +200,8 @@ var (
|
||||
avi = newMIME("video/x-msvideo", ".avi", magic.Avi).
|
||||
alias("video/avi", "video/msvideo")
|
||||
flv = newMIME("video/x-flv", ".flv", magic.Flv)
|
||||
mkv = newMIME("video/x-matroska", ".mkv", magic.Mkv)
|
||||
mkv = newMIME("video/matroska", ".mkv", magic.Mkv).
|
||||
alias("video/x-matroska")
|
||||
asf = newMIME("video/x-ms-asf", ".asf", magic.Asf).
|
||||
alias("video/asf", "video/x-ms-wmv")
|
||||
rmvb = newMIME("application/vnd.rn-realmedia-vbr", ".rmvb", magic.Rmvb)
|
||||
@@ -242,8 +251,8 @@ var (
|
||||
odc = newMIME("application/vnd.oasis.opendocument.chart", ".odc", magic.Odc).
|
||||
alias("application/x-vnd.oasis.opendocument.chart")
|
||||
sxc = newMIME("application/vnd.sun.xml.calc", ".sxc", magic.Sxc)
|
||||
rar = newMIME("application/x-rar-compressed", ".rar", magic.RAR).
|
||||
alias("application/x-rar")
|
||||
rar = newMIME("application/vnd.rar", ".rar", magic.RAR).
|
||||
alias("application/x-rar-compressed", "application/x-rar")
|
||||
djvu = newMIME("image/vnd.djvu", ".djvu", magic.DjVu)
|
||||
mobi = newMIME("application/x-mobipocket-ebook", ".mobi", magic.Mobi)
|
||||
lit = newMIME("application/x-ms-reader", ".lit", magic.Lit)
|
||||
@@ -294,4 +303,6 @@ var (
|
||||
hlp = newMIME("application/x-os2-hlp", ".hlp", magic.Hlp)
|
||||
fm = newMIME("application/vnd.framemaker", ".fm", magic.FrameMaker)
|
||||
bufr = newMIME("application/bufr", ".bufr", magic.BUFR)
|
||||
gedcom = newMIME("text/vnd.familysearch.gedcom", ".ged", magic.GEDCOM)
|
||||
pcap = newMIME("application/vnd.tcpdump.pcap", ".pcap", magic.Pcap)
|
||||
)
|
||||
+23
-10
@@ -19,6 +19,10 @@ import (
|
||||
// no limit.
|
||||
var MaxPacketLengthBytes int64 = math.MaxInt32
|
||||
|
||||
// MaxNestingDepth specifies the maximum allowed nesting depth when calling ReadPacket, DecodePacket, or
|
||||
// DecodePacketErr. Set to 0 for no limit.
|
||||
var MaxNestingDepth int = 1000
|
||||
|
||||
type Packet struct {
|
||||
Identifier
|
||||
Value interface{}
|
||||
@@ -218,7 +222,7 @@ func printPacket(out io.Writer, p *Packet, indent int, printBytes bool) {
|
||||
|
||||
// ReadPacket reads a single Packet from the reader.
|
||||
func ReadPacket(reader io.Reader) (*Packet, error) {
|
||||
p, _, err := readPacket(reader)
|
||||
p, _, err := readPacket(reader, 0)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -278,7 +282,7 @@ func int64Length(i int64) (numBytes int) {
|
||||
// DecodePacket decodes the given bytes into a single Packet
|
||||
// If a decode error is encountered, nil is returned.
|
||||
func DecodePacket(data []byte) *Packet {
|
||||
p, _, _ := readPacket(bytes.NewBuffer(data))
|
||||
p, _, _ := readPacket(bytes.NewBuffer(data), 0)
|
||||
|
||||
return p
|
||||
}
|
||||
@@ -286,7 +290,7 @@ func DecodePacket(data []byte) *Packet {
|
||||
// DecodePacketErr decodes the given bytes into a single Packet
|
||||
// If a decode error is encountered, nil is returned.
|
||||
func DecodePacketErr(data []byte) (*Packet, error) {
|
||||
p, _, err := readPacket(bytes.NewBuffer(data))
|
||||
p, _, err := readPacket(bytes.NewBuffer(data), 0)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -294,12 +298,20 @@ func DecodePacketErr(data []byte) (*Packet, error) {
|
||||
}
|
||||
|
||||
// readPacket reads a single Packet from the reader, returning the number of bytes read.
|
||||
func readPacket(reader io.Reader) (*Packet, int, error) {
|
||||
func readPacket(reader io.Reader, depth int) (*Packet, int, error) {
|
||||
if MaxNestingDepth > 0 && depth >= MaxNestingDepth {
|
||||
return nil, 0, fmt.Errorf("nesting depth %d exceeds maximum %d", depth, MaxNestingDepth)
|
||||
}
|
||||
|
||||
identifier, length, read, err := readHeader(reader)
|
||||
if err != nil {
|
||||
return nil, read, err
|
||||
}
|
||||
|
||||
if length != LengthIndefinite && MaxPacketLengthBytes > 0 && int64(length) > MaxPacketLengthBytes {
|
||||
return nil, read, fmt.Errorf("length %d greater than maximum %d", length, MaxPacketLengthBytes)
|
||||
}
|
||||
|
||||
p := &Packet{
|
||||
Identifier: identifier,
|
||||
}
|
||||
@@ -326,13 +338,19 @@ func readPacket(reader io.Reader) (*Packet, int, error) {
|
||||
}
|
||||
|
||||
// Read the next packet
|
||||
child, r, err := readPacket(reader)
|
||||
child, r, err := readPacket(reader, depth+1)
|
||||
if err != nil {
|
||||
return nil, read, unexpectedEOF(err)
|
||||
}
|
||||
contentRead += r
|
||||
read += r
|
||||
|
||||
// Enforce the aggregate size limit for constructed packets. Indefinite length declares
|
||||
// no bound up front, so the content bytes are only known as they are read.
|
||||
if MaxPacketLengthBytes > 0 && int64(contentRead) > MaxPacketLengthBytes {
|
||||
return nil, read, fmt.Errorf("length %d greater than maximum %d", contentRead, MaxPacketLengthBytes)
|
||||
}
|
||||
|
||||
// Test is this is the EOC marker for our packet
|
||||
if isEOCPacket(child) {
|
||||
if length == LengthIndefinite {
|
||||
@@ -351,11 +369,6 @@ func readPacket(reader io.Reader) (*Packet, int, error) {
|
||||
return nil, read, errors.New("indefinite length used with primitive type")
|
||||
}
|
||||
|
||||
// Read definite-length content
|
||||
if MaxPacketLengthBytes > 0 && int64(length) > MaxPacketLengthBytes {
|
||||
return nil, read, fmt.Errorf("length %d greater than maximum %d", length, MaxPacketLengthBytes)
|
||||
}
|
||||
|
||||
var content []byte
|
||||
if length > 0 {
|
||||
// Read the content and limit it to the parsed length.
|
||||
|
||||
+5
-4
@@ -40,7 +40,7 @@ func readLength(reader io.Reader) (length int, read int, err error) {
|
||||
}
|
||||
|
||||
// Accumulate into a 64-bit variable
|
||||
var length64 int64
|
||||
var length64 uint64
|
||||
for i := 0; i < lengthBytes; i++ {
|
||||
b, err = readByte(reader)
|
||||
if err != nil {
|
||||
@@ -53,13 +53,14 @@ func readLength(reader io.Reader) (length int, read int, err error) {
|
||||
|
||||
// x.600, 8.1.3.5
|
||||
length64 <<= 8
|
||||
length64 |= int64(b)
|
||||
length64 |= uint64(b)
|
||||
}
|
||||
|
||||
// Cast to a platform-specific integer
|
||||
length = int(length64)
|
||||
// Ensure we didn't overflow
|
||||
if int64(length) != length64 {
|
||||
// Ensure we didn't overflow or wrap negative. Length octets are unsigned
|
||||
// (x.600, 8.1.3.5), so a negative result is unrepresentable, not indefinite.
|
||||
if length < 0 || uint64(length) != length64 {
|
||||
return 0, read, errors.New("long-form length overflow")
|
||||
}
|
||||
|
||||
|
||||
+3
@@ -83,6 +83,9 @@ func (l *Conn) Add(addRequest *AddRequest) error {
|
||||
return err
|
||||
}
|
||||
|
||||
if len(packet.Children) < 2 {
|
||||
return fmt.Errorf("ldap: malformed response: expected at least 2 children, got %d", len(packet.Children))
|
||||
}
|
||||
if packet.Children[1].Tag == ApplicationAddResponse {
|
||||
err := GetLDAPError(packet)
|
||||
if err != nil {
|
||||
|
||||
+61
-16
@@ -3,13 +3,13 @@ package ldap
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/md5"
|
||||
"crypto/rand"
|
||||
"encoding/binary"
|
||||
"encoding/hex"
|
||||
enchex "encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"math/rand"
|
||||
"strings"
|
||||
"unicode/utf16"
|
||||
|
||||
@@ -221,12 +221,15 @@ func (l *Conn) DigestMD5Bind(digestMD5BindRequest *DigestMD5BindRequest) (*Diges
|
||||
}
|
||||
|
||||
if len(params) > 0 {
|
||||
resp := computeResponse(
|
||||
resp, err := computeResponse(
|
||||
params,
|
||||
"ldap/"+strings.ToLower(digestMD5BindRequest.Host),
|
||||
digestMD5BindRequest.Username,
|
||||
digestMD5BindRequest.Password,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("compute digest-md5 response: %s", err)
|
||||
}
|
||||
packet = ber.Encode(ber.ClassUniversal, ber.TypeConstructed, ber.TagSequence, nil, "LDAP Request")
|
||||
packet.AppendChild(ber.NewInteger(ber.ClassUniversal, ber.TypePrimitive, ber.TagInteger, l.nextMessageID(), "MessageID"))
|
||||
|
||||
@@ -291,12 +294,22 @@ func parseParams(str string) (map[string]string, error) {
|
||||
m := make(map[string]string)
|
||||
var key, value string
|
||||
var state int
|
||||
var escaped bool
|
||||
for i := 0; i <= len(str); i++ {
|
||||
switch state {
|
||||
case 0: // reading key
|
||||
if i == len(str) {
|
||||
return nil, fmt.Errorf("syntax error on %d", i)
|
||||
}
|
||||
// The digest-challenge is an RFC 2068 #rule (RFC 2831 section 2.1.1),
|
||||
// which permits optional linear whitespace around the comma directive
|
||||
// separators. Directive names are tokens that never contain
|
||||
// whitespace, so skip it here; otherwise a directive following
|
||||
// "..., name" is keyed with a leading space and the lookups in
|
||||
// computeResponse (realm, nonce, authzid) miss it.
|
||||
if str[i] == ' ' || str[i] == '\t' {
|
||||
continue
|
||||
}
|
||||
if str[i] != '=' {
|
||||
key += string(str[i])
|
||||
continue
|
||||
@@ -307,6 +320,14 @@ func parseParams(str string) (map[string]string, error) {
|
||||
m[key] = value
|
||||
break
|
||||
}
|
||||
// Linear whitespace outside a quoted string is not part of the
|
||||
// value: an unquoted value is a token and a quoted value's content
|
||||
// is read in the quoted state below. Skipping it lets a challenge
|
||||
// using the whitespace the #rule allows (e.g. `nonce="n" , qop=auth`)
|
||||
// parse the same as the unspaced form.
|
||||
if str[i] == ' ' || str[i] == '\t' {
|
||||
continue
|
||||
}
|
||||
switch str[i] {
|
||||
case ',':
|
||||
m[key] = value
|
||||
@@ -325,20 +346,34 @@ func parseParams(str string) (map[string]string, error) {
|
||||
if i == len(str) {
|
||||
return nil, fmt.Errorf("syntax error on %d", i)
|
||||
}
|
||||
if str[i] != '"' {
|
||||
switch {
|
||||
case escaped:
|
||||
// RFC 2831 section 7.1 quoted-pair: a backslash escapes the
|
||||
// following character, so the next byte is taken literally
|
||||
// (this is how a server sends a literal " or \ in a realm or
|
||||
// nonce).
|
||||
value += string(str[i])
|
||||
} else {
|
||||
escaped = false
|
||||
case str[i] == '\\':
|
||||
escaped = true
|
||||
case str[i] == '"':
|
||||
state = 1
|
||||
default:
|
||||
value += string(str[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
func computeResponse(params map[string]string, uri, username, password string) string {
|
||||
func computeResponse(params map[string]string, uri, username, password string) (string, error) {
|
||||
nc := "00000001"
|
||||
qop := "auth"
|
||||
cnonce := enchex.EncodeToString(randomBytes(16))
|
||||
rb, err := randomBytes(16)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
cnonce := enchex.EncodeToString(rb)
|
||||
x := username + ":" + params["realm"] + ":" + password
|
||||
y := md5Hash([]byte(x))
|
||||
|
||||
@@ -361,14 +396,24 @@ func computeResponse(params map[string]string, uri, username, password string) s
|
||||
resp := enchex.EncodeToString(md5Hash([]byte(kd)))
|
||||
return fmt.Sprintf(
|
||||
`username="%s",realm="%s",nonce="%s",cnonce="%s",nc=00000001,qop=%s,digest-uri="%s",response=%s`,
|
||||
username,
|
||||
params["realm"],
|
||||
params["nonce"],
|
||||
quotedStringEscape(username),
|
||||
quotedStringEscape(params["realm"]),
|
||||
quotedStringEscape(params["nonce"]),
|
||||
cnonce,
|
||||
qop,
|
||||
uri,
|
||||
quotedStringEscape(uri),
|
||||
resp,
|
||||
)
|
||||
), nil
|
||||
}
|
||||
|
||||
// quotedStringEscape escapes the two characters that may not appear unescaped
|
||||
// inside a DIGEST-MD5 quoted string per RFC 2831 section 7.1: the backslash
|
||||
// and the double quote. The backslash is replaced first so the quotes escaped
|
||||
// afterwards are not doubled.
|
||||
func quotedStringEscape(s string) string {
|
||||
s = strings.ReplaceAll(s, `\`, `\\`)
|
||||
s = strings.ReplaceAll(s, `"`, `\"`)
|
||||
return s
|
||||
}
|
||||
|
||||
func md5Hash(b []byte) []byte {
|
||||
@@ -377,12 +422,12 @@ func md5Hash(b []byte) []byte {
|
||||
return hasher.Sum(nil)
|
||||
}
|
||||
|
||||
func randomBytes(len int) []byte {
|
||||
b := make([]byte, len)
|
||||
for i := 0; i < len; i++ {
|
||||
b[i] = byte(rand.Intn(256))
|
||||
func randomBytes(length int) ([]byte, error) {
|
||||
b := make([]byte, length)
|
||||
if _, err := rand.Read(b); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return b
|
||||
return b, nil
|
||||
}
|
||||
|
||||
var externalBindRequest = requestFunc(func(envelope *ber.Packet) error {
|
||||
|
||||
+3
@@ -46,6 +46,9 @@ func (l *Conn) Compare(dn, attribute, value string) (bool, error) {
|
||||
return false, err
|
||||
}
|
||||
|
||||
if len(packet.Children) < 2 {
|
||||
return false, fmt.Errorf("ldap: malformed response: expected at least 2 children, got %d", len(packet.Children))
|
||||
}
|
||||
if packet.Children[1].Tag == ApplicationCompareResponse {
|
||||
err := GetLDAPError(packet)
|
||||
|
||||
|
||||
+55
-11
@@ -8,6 +8,7 @@ import (
|
||||
"fmt"
|
||||
"net"
|
||||
"net/url"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
@@ -112,7 +113,11 @@ type Conn struct {
|
||||
outstandingRequests uint
|
||||
messageMutex sync.Mutex
|
||||
|
||||
err error
|
||||
// errMutex guards err only. It is a leaf lock: processMessages and reader
|
||||
// record errors while another goroutine may hold messageMutex, so err must
|
||||
// not share messageMutex or those writers could deadlock.
|
||||
errMutex sync.Mutex
|
||||
err error
|
||||
}
|
||||
|
||||
var _ Client = &Conn{}
|
||||
@@ -160,10 +165,18 @@ type DialContext struct {
|
||||
|
||||
func (dc *DialContext) dial(u *url.URL) (net.Conn, error) {
|
||||
if u.Scheme == "ldapi" {
|
||||
if u.Path == "" || u.Path == "/" {
|
||||
u.Path = "/var/run/slapd/ldapi"
|
||||
// RFC 4516 (and draft-chu-ldap-ldapi) put the socket path in the
|
||||
// host component, percent-encoded; the path is an optional DN.
|
||||
// parseLDAPURL has already decoded the host. Accept the older
|
||||
// ldapi:///path form too so existing callers keep working.
|
||||
path := u.Host
|
||||
if path == "" {
|
||||
path = u.Path
|
||||
}
|
||||
return dc.dialer.Dial("unix", u.Path)
|
||||
if path == "" || path == "/" {
|
||||
path = "/var/run/slapd/ldapi"
|
||||
}
|
||||
return dc.dialer.Dial("unix", path)
|
||||
}
|
||||
|
||||
host, port, err := net.SplitHostPort(u.Host)
|
||||
@@ -222,12 +235,33 @@ func DialTLS(network, addr string, config *tls.Config) (*Conn, error) {
|
||||
return conn, nil
|
||||
}
|
||||
|
||||
// parseLDAPURL parses an LDAP URL. It defers to net/url for the common
|
||||
// ldap/ldaps/cldap schemes, but handles ldapi specially: the spec puts the
|
||||
// unix socket path in the host, percent-encoded with %2F, and net/url rejects
|
||||
// that as invalid. Pull the host out manually and decode it.
|
||||
func parseLDAPURL(addr string) (*url.URL, error) {
|
||||
const ldapi = "ldapi://"
|
||||
if !strings.HasPrefix(addr, ldapi) {
|
||||
return url.Parse(addr)
|
||||
}
|
||||
rest := addr[len(ldapi):]
|
||||
host, path := rest, ""
|
||||
if i := strings.IndexByte(rest, '/'); i >= 0 {
|
||||
host, path = rest[:i], rest[i:]
|
||||
}
|
||||
decodedHost, err := url.PathUnescape(host)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("ldapi: invalid host %q: %w", host, err)
|
||||
}
|
||||
return &url.URL{Scheme: "ldapi", Host: decodedHost, Path: path}, nil
|
||||
}
|
||||
|
||||
// DialURL connects to the given ldap URL.
|
||||
// The following schemas are supported: ldap://, ldaps://, ldapi://,
|
||||
// and cldap:// (RFC1798, deprecated but used by Active Directory).
|
||||
// On success a new Conn for the connection is returned.
|
||||
func DialURL(addr string, opts ...DialOpt) (*Conn, error) {
|
||||
u, err := url.Parse(addr)
|
||||
u, err := parseLDAPURL(addr)
|
||||
if err != nil {
|
||||
return nil, NewError(ErrorNetwork, err)
|
||||
}
|
||||
@@ -338,11 +372,21 @@ func (l *Conn) nextMessageID() int64 {
|
||||
// GetLastError returns the last recorded error from goroutines like processMessages and reader.
|
||||
// Only the last recorded error will be returned.
|
||||
func (l *Conn) GetLastError() error {
|
||||
l.messageMutex.Lock()
|
||||
defer l.messageMutex.Unlock()
|
||||
l.errMutex.Lock()
|
||||
defer l.errMutex.Unlock()
|
||||
return l.err
|
||||
}
|
||||
|
||||
// setError records the connection's last error. The background goroutines that
|
||||
// call it (processMessages, reader, the per-request timeout helper and the
|
||||
// SearchAsync worker) run concurrently with callers of GetLastError, so the
|
||||
// write must take the mutex the getter reads under.
|
||||
func (l *Conn) setError(err error) {
|
||||
l.errMutex.Lock()
|
||||
defer l.errMutex.Unlock()
|
||||
l.err = err
|
||||
}
|
||||
|
||||
// StartTLS sends the command to start a TLS session and then creates a new TLS Client
|
||||
func (l *Conn) StartTLS(config *tls.Config) error {
|
||||
if l.isTLS {
|
||||
@@ -491,7 +535,7 @@ func (l *Conn) sendProcessMessage(message *messagePacket) bool {
|
||||
func (l *Conn) processMessages() {
|
||||
defer func() {
|
||||
if err := recover(); err != nil {
|
||||
l.err = fmt.Errorf("ldap: recovered panic in processMessages: %v", err)
|
||||
l.setError(fmt.Errorf("ldap: recovered panic in processMessages: %v", err))
|
||||
}
|
||||
for messageID, msgCtx := range l.messageContexts {
|
||||
// If we are closing due to an error, inform anyone who
|
||||
@@ -541,7 +585,7 @@ func (l *Conn) processMessages() {
|
||||
timer := time.NewTimer(time.Duration(requestTimeout))
|
||||
defer func() {
|
||||
if err := recover(); err != nil {
|
||||
l.err = fmt.Errorf("ldap: recovered panic in RequestTimeout: %v", err)
|
||||
l.setError(fmt.Errorf("ldap: recovered panic in RequestTimeout: %v", err))
|
||||
}
|
||||
|
||||
timer.Stop()
|
||||
@@ -563,7 +607,7 @@ func (l *Conn) processMessages() {
|
||||
if msgCtx, ok := l.messageContexts[message.MessageID]; ok {
|
||||
msgCtx.sendResponse(&PacketResponse{message.Packet, nil}, time.Duration(l.getTimeout()))
|
||||
} else {
|
||||
l.err = fmt.Errorf("ldap: received unexpected message %d, %v", message.MessageID, l.IsClosing())
|
||||
l.setError(fmt.Errorf("ldap: received unexpected message %d, %v", message.MessageID, l.IsClosing()))
|
||||
l.Debug.PrintPacket(message.Packet)
|
||||
}
|
||||
case MessageTimeout:
|
||||
@@ -590,7 +634,7 @@ func (l *Conn) reader() {
|
||||
cleanstop := false
|
||||
defer func() {
|
||||
if err := recover(); err != nil {
|
||||
l.err = fmt.Errorf("ldap: recovered panic in reader: %v", err)
|
||||
l.setError(fmt.Errorf("ldap: recovered panic in reader: %v", err))
|
||||
}
|
||||
if !cleanstop {
|
||||
l.Close()
|
||||
|
||||
+84
-27
@@ -565,12 +565,20 @@ func DecodeControl(packet *ber.Packet) (Control, error) {
|
||||
case 1:
|
||||
// just type, no criticality or value
|
||||
packet.Children[0].Description = "Control Type (" + ControlTypeMap[ControlType] + ")"
|
||||
ControlType = packet.Children[0].Value.(string)
|
||||
ct, ok := packet.Children[0].Value.(string)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("control type is not a string: %T", packet.Children[0].Value)
|
||||
}
|
||||
ControlType = ct
|
||||
|
||||
case 2:
|
||||
packet.Children[0].Description = "Control Type (" + ControlTypeMap[ControlType] + ")"
|
||||
if packet.Children[0].Value != nil {
|
||||
ControlType = packet.Children[0].Value.(string)
|
||||
ct, ok := packet.Children[0].Value.(string)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("control type is not a string: %T", packet.Children[0].Value)
|
||||
}
|
||||
ControlType = ct
|
||||
} else if packet.Children[0].Data != nil {
|
||||
ControlType = packet.Children[0].Data.String()
|
||||
} else {
|
||||
@@ -579,9 +587,9 @@ func DecodeControl(packet *ber.Packet) (Control, error) {
|
||||
|
||||
// Children[1] could be criticality or value (both are optional)
|
||||
// duck-type on whether this is a boolean
|
||||
if _, ok := packet.Children[1].Value.(bool); ok {
|
||||
if crit, ok := packet.Children[1].Value.(bool); ok {
|
||||
packet.Children[1].Description = "Criticality"
|
||||
Criticality = packet.Children[1].Value.(bool)
|
||||
Criticality = crit
|
||||
} else {
|
||||
packet.Children[1].Description = "Control Value"
|
||||
value = packet.Children[1]
|
||||
@@ -589,10 +597,18 @@ func DecodeControl(packet *ber.Packet) (Control, error) {
|
||||
|
||||
case 3:
|
||||
packet.Children[0].Description = "Control Type (" + ControlTypeMap[ControlType] + ")"
|
||||
ControlType = packet.Children[0].Value.(string)
|
||||
ct, ok := packet.Children[0].Value.(string)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("control type is not a string: %T", packet.Children[0].Value)
|
||||
}
|
||||
ControlType = ct
|
||||
|
||||
packet.Children[1].Description = "Criticality"
|
||||
Criticality = packet.Children[1].Value.(bool)
|
||||
crit, ok := packet.Children[1].Value.(bool)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("criticality is not a bool: %T", packet.Children[1].Value)
|
||||
}
|
||||
Criticality = crit
|
||||
|
||||
packet.Children[2].Description = "Control Value"
|
||||
value = packet.Children[2]
|
||||
@@ -606,6 +622,9 @@ func DecodeControl(packet *ber.Packet) (Control, error) {
|
||||
case ControlTypeManageDsaIT:
|
||||
return NewControlManageDsaIT(Criticality), nil
|
||||
case ControlTypePaging:
|
||||
if value == nil {
|
||||
return nil, fmt.Errorf("paging control value is missing")
|
||||
}
|
||||
value.Description += " (Paging)"
|
||||
c := new(ControlPaging)
|
||||
if value.Value != nil {
|
||||
@@ -617,11 +636,21 @@ func DecodeControl(packet *ber.Packet) (Control, error) {
|
||||
value.Value = nil
|
||||
value.AppendChild(valueChildren)
|
||||
}
|
||||
if len(value.Children) == 0 {
|
||||
return nil, fmt.Errorf("paging control value is empty")
|
||||
}
|
||||
value = value.Children[0]
|
||||
value.Description = "Search Control Value"
|
||||
if len(value.Children) < 2 {
|
||||
return nil, fmt.Errorf("paging control value has %d children, expected 2", len(value.Children))
|
||||
}
|
||||
value.Children[0].Description = "Paging Size"
|
||||
value.Children[1].Description = "Cookie"
|
||||
c.PagingSize = uint32(value.Children[0].Value.(int64))
|
||||
pagingSize, ok := value.Children[0].Value.(int64)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("paging size is not an integer: %T", value.Children[0].Value)
|
||||
}
|
||||
c.PagingSize = uint32(pagingSize)
|
||||
c.Cookie = value.Children[1].Data.Bytes()
|
||||
value.Children[1].Value = c.Cookie
|
||||
return c, nil
|
||||
@@ -729,7 +758,16 @@ func DecodeControl(packet *ber.Packet) (Control, error) {
|
||||
c.ControlType = ControlType
|
||||
c.Criticality = Criticality
|
||||
if value != nil {
|
||||
c.ControlValue = value.Value.(string)
|
||||
// A non-conforming or malicious server can send a non-string
|
||||
// (or nil) value here; the previous unchecked cast panicked
|
||||
// the calling goroutine, see #561. Fall back to the raw bytes
|
||||
// when the value isn't a string so we surface an error
|
||||
// instead of crashing.
|
||||
if s, ok := value.Value.(string); ok {
|
||||
c.ControlValue = s
|
||||
} else if value.Data != nil {
|
||||
c.ControlValue = value.Data.String()
|
||||
}
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
@@ -921,28 +959,44 @@ func (c *ControlServerSideSorting) GetControlType() string {
|
||||
}
|
||||
|
||||
func NewControlServerSideSorting(value *ber.Packet) (*ControlServerSideSorting, error) {
|
||||
sortKeys := []*SortKey{}
|
||||
val, err := ber.DecodePacketErr(value.Data.Bytes())
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("decode packet err: %s", err)
|
||||
}
|
||||
|
||||
val := value.Children[1].Children
|
||||
|
||||
if len(val) != 1 {
|
||||
if len(val.Children) == 0 {
|
||||
return nil, fmt.Errorf("no sequence value in packet")
|
||||
}
|
||||
|
||||
sequences := val[0].Children
|
||||
var sortKeys []*SortKey
|
||||
|
||||
for i, sequence := range sequences {
|
||||
sortKey := new(SortKey)
|
||||
|
||||
if len(sequence.Children) < 2 {
|
||||
return nil, fmt.Errorf("attributeType or matchingRule is missing from sequence %d", i)
|
||||
for i, sequence := range val.Children {
|
||||
if len(sequence.Children) < 1 || len(sequence.Children) > 3 {
|
||||
return nil, fmt.Errorf("attributeType is missing from sequence %d", i)
|
||||
}
|
||||
|
||||
sortKey.AttributeType = sequence.Children[0].Value.(string)
|
||||
sortKey.MatchingRule = sequence.Children[1].Value.(string)
|
||||
sortKey := new(SortKey)
|
||||
|
||||
if len(sequence.Children) == 3 {
|
||||
sortKey.Reverse = sequence.Children[2].Value.(bool)
|
||||
for _, child := range sequence.Children {
|
||||
switch {
|
||||
case child.ClassType == ber.ClassUniversal && child.Tag == ber.TagOctetString:
|
||||
// A constructed-form OCTET STRING matches this case but leaves
|
||||
// Value nil; guard the assertion so a malformed attributeType is
|
||||
// rejected below rather than panicking.
|
||||
if attrType, ok := child.Value.(string); ok {
|
||||
sortKey.AttributeType = attrType
|
||||
}
|
||||
|
||||
case child.ClassType == ber.ClassContext && child.Tag == 0:
|
||||
sortKey.MatchingRule = child.Data.String()
|
||||
|
||||
case child.ClassType == ber.ClassContext && child.Tag == 1:
|
||||
b := child.Data.Bytes()
|
||||
sortKey.Reverse = len(b) > 0 && b[0] != 0
|
||||
}
|
||||
}
|
||||
if sortKey.AttributeType == "" {
|
||||
return nil, fmt.Errorf("attributeType is missing from sequence %d", i)
|
||||
}
|
||||
|
||||
sortKeys = append(sortKeys, sortKey)
|
||||
@@ -959,7 +1013,6 @@ func (c *ControlServerSideSorting) Encode() *ber.Packet {
|
||||
packet := ber.Encode(ber.ClassUniversal, ber.TypeConstructed, ber.TagSequence, nil, "Control")
|
||||
control := ber.NewString(ber.ClassUniversal, ber.TypePrimitive, ber.TagOctetString, c.GetControlType(), "Control Type")
|
||||
|
||||
value := ber.Encode(ber.ClassUniversal, ber.TypePrimitive, ber.TagOctetString, nil, "Control Value")
|
||||
seqs := ber.Encode(ber.ClassUniversal, ber.TypeConstructed, ber.TagSequence, nil, "SortKeyList")
|
||||
|
||||
for _, f := range c.SortKeys {
|
||||
@@ -968,9 +1021,11 @@ func (c *ControlServerSideSorting) Encode() *ber.Packet {
|
||||
seq.AppendChild(
|
||||
ber.NewString(ber.ClassUniversal, ber.TypePrimitive, ber.TagOctetString, f.AttributeType, "attributeType"),
|
||||
)
|
||||
seq.AppendChild(
|
||||
ber.NewString(ber.ClassContext, ber.TypePrimitive, 0, f.MatchingRule, "orderingRule"),
|
||||
)
|
||||
if f.MatchingRule != "" {
|
||||
seq.AppendChild(
|
||||
ber.NewString(ber.ClassContext, ber.TypePrimitive, 0, f.MatchingRule, "orderingRule"),
|
||||
)
|
||||
}
|
||||
if f.Reverse {
|
||||
seq.AppendChild(
|
||||
ber.NewBoolean(ber.ClassContext, ber.TypePrimitive, 1, f.Reverse, "reverseOrder"),
|
||||
@@ -980,7 +1035,7 @@ func (c *ControlServerSideSorting) Encode() *ber.Packet {
|
||||
seqs.AppendChild(seq)
|
||||
}
|
||||
|
||||
value.AppendChild(seqs)
|
||||
value := ber.Encode(ber.ClassUniversal, ber.TypePrimitive, ber.TagOctetString, string(seqs.Bytes()), "Control Value")
|
||||
|
||||
packet.AppendChild(control)
|
||||
packet.AppendChild(value)
|
||||
@@ -1060,6 +1115,8 @@ func NewControlServerSideSortingResult(pkt *ber.Packet) (*ControlServerSideSorti
|
||||
return nil, err
|
||||
}
|
||||
|
||||
control.Result = ControlServerSideSortingCode(codeInt)
|
||||
|
||||
return control, nil
|
||||
}
|
||||
|
||||
|
||||
+3
@@ -52,6 +52,9 @@ func (l *Conn) Del(delRequest *DelRequest) error {
|
||||
return err
|
||||
}
|
||||
|
||||
if len(packet.Children) < 2 {
|
||||
return fmt.Errorf("ldap: malformed response: expected at least 2 children, got %d", len(packet.Children))
|
||||
}
|
||||
if packet.Children[1].Tag == ApplicationDelResponse {
|
||||
err := GetLDAPError(packet)
|
||||
if err != nil {
|
||||
|
||||
Loaded 100 of 362 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user