Commit Graph

60 Commits

Author SHA1 Message Date
Brendan Allan
23fbb1e408 Move location online status out of database (#560)
* move location online status out of database

* remove useLibrarySubscription

* remove old code
2023-02-07 05:58:47 -08:00
jake
e5b45d8f49 [ENG-355] Keychain integration (and some typesafety) (#558)
* update crypto MSRV

* rename `keychain` to `keyring`

* make a start on the keymanager unlock refactor/keychain integration

* update routes

* update bindings

* add const identifiers

* add UI/front-end support for unlocking KM with OS keychains

* remove SK from lib creation dialog

* update query name

* add keyring functions

* attempt to update `change_master_password()` to use the keychain

* cleanup, fix master password change ui, better secret key in keyring detection

* cleanup TS a little

* add route for getting secret key from keyring

* update bindings

* update var names + show secret key in keys settings

* add `react-qr-code` and option to view the secret key (if it's in the OS keyring)

* allow copying SK to clipboard

* add `key_type` so we're not reliant on specific UUIDs for root/verification key handling

* clippy

* fix mobile typecheck

* fix typecheck, fix typo and tweak balloon hash parameters

* minor cleanup + typo fix

* use newtype structs

* WIP type refactoring (major readability boost!)

* update `use`

* add tokio `sync` feature

* too many structs? idk

* more cleanup

* add `generate` and `Nonce`

* `Nonce` and `Key` typesafety (beautiful)

* clippy + cleanup

* update code & examples

* fix bug & remove `ProtectedVec` as it looked out of place

* use `Key`

* add a query invalidation to make the UI extremely responsive

* ci pls work

* remove `keyringHasSk` route
2023-02-07 12:03:12 +00:00
Brendan Allan
1ca84a2d33 remove temp_cas_id 2023-02-01 11:23:59 +08:00
Brendan Allan
224ed772ac store cas ids on file paths 2023-01-28 11:42:24 +08:00
Brendan Allan
80f85d6f7c Eng 332 auto generate sync ids (#523)
* generate sync ids and use model name from ModelActions

* cleanup AddLocationDialog imports

* rustfmt
2023-01-15 07:25:11 -08:00
jake
3c0729e7aa [ENG-331] StoredKey overhaul (#513)
* add wip storedkey versioning

* storedkey versioning! (not pretty, but it never will be)

* add version to `StoredKey` and re-gen migrations to handle serde

* use `serde` for interacting with the DB + handle errors
2023-01-11 05:15:08 -08:00
Brendan Allan
7daa0b0aa8 update prisma (#505) 2023-01-05 08:05:33 -08:00
Brendan Allan
2153e1dd56 Beginning of sync stuff (#502) 2023-01-04 23:11:55 -08:00
jake
49fa8f6576 [ENG-318] Implement a KDF to derive new keys (#487)
* add blake3, `derive_key()` and clean up code

* fix a couple of things from the previous commit

* add context strings for root/file key derivation

* add salt to schema

* update refs of `salt` to `content_salt` within the keyslot

* cleanup code and add kdf salt to the keyslot

* rename salt to content salt in examples

* cleanup header code + remove dead code

* implement key derivation for keyslots

* gen new migrations that contain a salt column

* keymanager refactor (code is very idiomatic now) - needs thorough testing

* further cleanup

* clippy

* add a master password context string

* use key derivation for deriving keys from the root key

* update to use new code and remove `match` from `en/decrypt_bytes()`

* clippy

* use less unwraps in library manager code

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-12-30 20:09:44 +00:00
Ericson "Fogo" Soares
3b142c6180 ENG 223 Location Awareness (#468)
* Introducing location online/offline checks and location relink

* Some initial drafts for location watchers

* Location metadata relink and add library

* Many improvements at job system
Now using prisma batching at identifier job
Removing blocking I/O from extension subcrate
Implementing lazy generation of thumbnails
New current directory identifier job to be used on light rescans

* Some optimizations on identifier and object validator jobs

* merge jamie's identifier PR

* fully repaired identifier job

* properly hooked up object kind

* inspector fix

* fix video badge

* small improvements to libraries settings

* identifier and inspector improvements

* fix feature flags and hook up context menu location utilities

* BETTER CONTEXT MENU x100

* test-files

* style tweaks

* new icon designs

* manifest

* fix thumbnails on web

* media data

* New Location Watcher and some minor fixes

* disable broken media_data extractor, wip

* wip

* function name fix

* Fixing pnpm prep and some warnings

* Solving a race condition beetween indexer job and FS event handlerSome other minor warnings

* Generating thumbnails on watcher

* Remove event handler on watcher

* Some initial works on modify events and other small fixes

* File update event

* Trying to be more generic with used events and some tests to validate our assumptions

* Turning on location metadata file

* Introducing core unit tests on CI pipeline

* Submiting new unit test assumptions to validate on windows CI

* Fixing unit tests

* Fixing unit tests again

* Fixing unit tests

* Fixing unit tests for macos

* Fixing unit tests for macos again

* New structure for platform dependent event handling
Implementing event handlers for Linux and MacOS

* minor fixes + rustfmt + clippy

* Windows event handling

* Introducing a feature gate to only use location watching on desktop app for now

* Putting more stuff behind feature gates to avoid warnings

* Adding feature to cargo test on CI

* Changing some debug logs to trace logs and removing Jamie specific stuff

* Make location removal from manager less async

* fix build when "location-watcher" feature disabled

* fix types + clippy

* make location manager non-static

* remove uses of `to_string_lossy`

* more invalidate_query calls

* Expose `library_ctx` directly to avoid needless clones

* New materialized_path handling for directories

* Removing cascade delete between file_path and object
- Some other minor stuff

* remove unused `CurrentDirFileIdentifierJob`

Co-authored-by: Jamie Pine <ijamespine@me.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-12-31 00:53:24 +08:00
jake
55e9d02136 [ENG-307] Key manager features (#467)
* working key management in settings page

* sync with library button

* fix `rspc` feature and add passphrase generation

* untested key manager rework

* trying to return values from mutations

* update library manager and remove settiong master PW

* update bindings

* set static secret key/master password

* prompt user for master password if correct one hasn't been provided yet

* add `hasMasterPassword` route

* add `clearMasterPassword` route + remove dead code

* tweak `set_master_password()` and add dedicated error

* tweak UI, fix `few hooks than expected`, add unmount+lock button

* remove old comment

* fmt

* clippy

* move static key/password setting so it doesn't fail sometimes

* add dedicated `get_key()` and remove keys from memory

* add `getKey` route

* update bindings

* use `const` instead of `let`

* comment updates

* update schema to remove salt

* add string parse error

* generate passphrase within key manager

* generate new migrations

* feature gate serde support in `crypto` crate

* fmt

* more specific error types

* foramatting

* add locking mechanism to keysettings page (not working?)

* fix react hook issues

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* remove empty onclick

* add keymanager dropdown menu

* working key backup functionality

* add experimental master password changing support

* update bindings

* semi-working change master password dialog

* use gear/lock icons to clean up key manager UI

* make settings button functional

* make buttons uniform and format code

* fix double base64 encode

* add change master password dialog and secret key dialog

* code cleanup

* restore backup dialog

* change UI wording

* make a start on restoring from a backup

* potentially working keystore restore

* don't overwrite verification key if one is set

* working backup restore + fix master password changing

* fix typo in static password/verification key check logic

* change wording to make UI clearer

* disable mount button if key is empty

* handle errors+remove type annotations

* show total imported keys on backup restoration

* add zxcvbn package

* change input border colour based on zxcvbn score

* clippy and formatting

* password strength meter

* remove nbsp

* add button type to stop early form submission

* use `react-hook-form` for backup restoration dialog

* more `react-hook-form` stuff

* attempt to fix password meter

* small cleanup

* Fix password meter

* update colours

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2022-12-02 10:18:21 +00:00
jake
84d0954294 [ENG-262] Key Manager Integration (#450)
* add keys router

* make progress on keymanager integration

* make name non-negotiable

* hyphenate encryption algorithm names

* Revert "make name non-negotiable"

This reverts commit 9c0f51329e.

* add some more keymanager queries

* add master password and default key routes

* add newly registered key to db + fmt

* clippy, formatting and `updateKeyName` route

* add automount to schema+automount keys where `true`

* update bindings

* working key add/mount

* working keylist

* mounted keys show first

* cleanup code

* add comments, code cleanup, more functions

* unmount all keys button

* comment and keymanager `clear_master_password()`

* add no keys available message

* fix unmount button

* use dashmap for concurrency

* fix missing keylist issue and add invalidate query macro

* set correct RSPC types

* statically set master password (TEMPORARILY)

* add remove key function within the keymanager

* key dropdown menu and impl

* formatting

* allow `option_if_let_else`

* add comment about key stats

* add additional comment

* rpsc error handling for the keys route

* fix rspc errors with an impl

* crypto crate errors to `sd-crypto::Error`

* remove `map_err`

* use custom result type

* cargo fmt

* clippy

* fix builds

* remove `Error::MutexLock`

* fix unnecessary unwrap

* mutex error handling (buggy for some reason)

* clean default key logic

* fix default key clearing

* allow a key to be removed without bugs

* implement requested changes

* use a single `useMemo`

* update schema with defaults

* re-generate migrations

* use rust enums in TS

* remove dead code

* remove mutate expansion

* read key list from keymanager, not prisma

* add "Default" key marker and cleanup TS

* rustfmt

* remove dead code
2022-11-05 04:18:01 -07:00
Oscar Beaumont
ef94923bf1 fix database migrations 2022-10-25 06:53:24 +10:00
jake
702e3a7613 [ENG-258] Key Manager and Crypto Crate Improvements (#423)
* add base keymanager structs/functions

* change md/pvm `new()` to prevent useless re-hashing

* update lockfile

* update keymanager

* cleanup code, make things easier to understand

* move md and pvm construction to the header

* update pvm ser/de

* update metadata ser/de

* additional API changes, update example

* formatting and clippy

* update examples

* move `impl` to associated files

* formatting+clippy

* add more keymanager functionality

* add docs

* wrap `master_password` as an option

* add `sd-crypto` crate as a dependency to `core`

* add key manager to `LibraryContext`

* rename `id` -> `uuid`

* add more keymanager functions

* add `set_master_password()` function

* function to see if keymanager has master password

* update schema

* add default bool to schema

* populate keystore on SD startup

* clippy+formatting

* implement requested changes

* remove unwrap

* fmt, clippy, remove ser/de derives

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2022-10-23 03:47:41 -07:00
jake
06b9720988 CAS ID Improvements (#413)
* remove `ring` dependency and use `sha2` instead

* use BLAKE3 and include full file checksum

* update schema comments
2022-10-13 15:31:47 -07:00
Jamie Pine
b9ce8e9f57 Rename Files to Objects (#396)
* backend rename complete

* finalize Object refactor frontend
+ remove some exhaustive dep suppression
+ FIX INVALIDATE QUERY BUG
2022-10-04 09:16:49 -07:00
Brendan Allan
72069411bf move prisma schema and migrations to /core (#392)
* move prisma schema and migrations to /core

* fix prisma updates
2022-10-04 02:32:46 -07:00
Jamie Pine
6796262619 Added crates folder (#389)
* restructure core for crates folder

- improve naming consistency
- added placeholder files for near future code
- added docs

* - removed all typescript from core
- moved rspc bindings to client

* remove useless git ignore & rustfmt
2022-09-29 21:02:29 -07:00
Brendan Allan
a7c3816419 Use feature gated database branch of PCR (#387)
use feature gating branch of pcr
2022-09-22 23:40:15 -07:00
Brendan Allan
18079ba378 Use PCR for migrations (#385)
* use pcr migrations branch

* use 0.6.2 branch with migrations

* use latest prisma stuff

* allow force reset of db in dev

* remove .spacedrive file

* update rspc in apps/server

* use rspc 0.0.5 in all crates

* add os to prisma client cache key

* add runner os to clippy prisma cache
2022-09-22 00:50:21 -07:00
Oscar Beaumont
1fca6fb60a onboarding UI flow 2022-09-12 23:43:12 +08:00
Oscar Beaumont
bd778a4f6b Merge remote-tracking branch 'origin/main' into eng-237-explorer-performance-2 2022-09-09 15:30:17 +08:00
Oscar Beaumont
efb451ee36 Better db migrations (#381)
* use prisma migration engine for migrations

* squash migrations + remove excessive logging
2022-09-08 15:57:07 -07:00
Ericson Fogo Soares
3a8d0c91e9 Implementing metadata for jobs 2022-09-07 23:08:11 -03:00
Brendan Allan
e9336a1417 utilise include instead of with for type safety 2022-09-07 04:09:57 +08:00
Oscar Beaumont
873dc7b89c Upgraded rspc with libraries + upgrade PCR 0.6.0 2022-09-06 22:08:19 +08:00
Jamie Pine
d621145f49 Assignable Tags (#370)
* Tag queries + Identifier bug fix

- added empty pages for docs
- added non-functional "New Library" button to sidebar
- to accomplish above, moved dialog logic to component folder
- fixed Identifier bug where location id was not considered in the orphan files query, meaning it would attempt to identify many non-existent paths and fail to create the legitimate ones

* (fix) maintain file item aspect ratio in grid view
(fix) get tag query + get all tags

* codegen

* resurrected context menu

* fix window flash

* remove location based thumb sorting + update react on landing

* add is_archived to location

* improved context menu

* assign tag + refactor explorer store

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* assign/unassign tags from context menu

* fix lint

* keep context menu open on tag change

* fix brendan feedback + merge main

* fix type

* codegen

* style changes

* revert windows size and xcode proj

* remove outdated doc

* coming sooooooon

* updated release notes

* release notes

* dashes > dots

* clean up docs

* more docs!

* restore entitlements

* remove unused util

* regenerated migrations post merge
- some additional error handling added to migration runner, needs more work!

* refactor explorer to support tags

* fix error and revert explorer bg color

* put it in a box they said

* revert location id store removal

* upgrade hero icons + style tweaks

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
2022-09-04 13:00:24 -07:00
Ericson "Fogo" Soares
ddd638723a Introducing Indexer rules (#363)
* Introducing indexer rules for the indexer job
* Solving a minor race condition at `Worker::spawn`
* Many refactors at database accesses with prisma
* Documenting some features of indexer rules

* Updating to new prisma rev

* Updating photo

* Some documentation tweaks

* fix note editing

* Implementing some requested changes
* Fixing unit tests on walk function
* Using rsps::Error::with_cause instead of new

* Introducing a compoung unique key between `node_id` and `local_path` at `locations` table

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-09-01 16:38:26 -07:00
Brendan Allan
dc81c83825 use latest prisma-client-rust 2022-08-19 07:41:08 +08:00
Oscar Beaumont
3804f034f1 Move Rust backend to rspc (#345)
* move Rust backend to rspc

* move server to Axum + remove parts of old bridge

* move frontend over to @rspc/client

* move core to rspc + update deps

* fix Typescript errors + upgrade deps

* document invalidate_query! macro

* general cleanup + upgrade to PCR 0.6.0

* prisma error handling

* upgrade to rspc 0.0.4

* update vite-plugin-ssr

* fix typescript

* fix builds

* put landing page app name back

* hardcode ffmpeg version on Windows

* rename 'command' to 'mutation' to line up with react-query terminology

* upgrade rspc to v0.0.5 + fix types

* use shared ffmpeg binaries

* general Typescript cleanup

* fix clippy workflow failing due to tauri proc-macro

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2022-08-03 08:36:03 -07:00
Ericson "Fogo" Soares
1d30850c21 Resumable Jobs + Lefthook Integration (#344)
* Introducing Lefthook for git hooks automation
* TypeScript typechecking and linting
* Rust formatting and linting
* Spellchecking (also corrected spell errors in some files)
* Check links in md files

* Introducing resumable jobs
* Abstractions to pause and resume jobs automatically when application exits and is started
* Changing database to use Bytes for UUID fields
* Changing uuid fields on core to use uuid::Uuid instead of String
* Updating some dependencies and introducing msg_pack serialization to save job state on database

* Fixing some clippy warnings

* Fixing a regression introduced on identifier job, doing too much db accesses concurrently
2022-07-27 00:06:34 -07:00
Jamie Pine
47ae02b743 Bunch 'O Features (#336)
* CRUD for tags.

* Implement tags query in open.rs and make some changes to CRUD.

* Tag update.

* Hopefully working get tags.

* added node config + spaces schema

* add missing routes

* begin tag ui

* renaming query names to better fit convention

* tags progress

* tag edit

* fix delete tag description

Co-authored-by: xPolar <polar@polar.blue>
2022-07-17 20:45:04 -07:00
Oscar Beaumont
017ef8b005 Library manager (#258) 2022-07-10 19:05:24 -07:00
Jamie Pine
efd322c6d7 Location Settings (#302)
* Delete locations

+ Show online status

* style tweaks

* tweaks

* location rescan button

* fix location delete db locking bug

* opting to remove self referencial relation on file_path

* correct query

* consolodate migration

* consolodate migrations
2022-06-24 06:26:45 -07:00
xPolar
d0d31e42c6 Notes on files. (#253)
* Temp fix for build.rs

* Note implementation

* Notes work now, need to iron out some things.

* Finish file notes.

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* Remove lodash debounce

* Fix pnpm locl

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
2022-06-20 21:14:05 -07:00
maxichrome
d6d00f878c Merge branch 'main' into jamie-unsorted-changes 2022-05-29 02:39:29 -05:00
Polar
7babc6619d various grammar and typo fixes. 2022-05-27 14:54:56 -07:00
Brendan Allan
2e6ff18347 cleanup + update to prisma-client-rust 0.5.0
still waiting to rebase for changes to identifier
2022-05-26 16:17:46 +08:00
Jamie Pine
e587480bc4 database fixes 2022-05-22 22:28:33 -07:00
Jamie Pine
d2f7a3af94 remove migration history (#173), fix incorect name 2022-05-22 22:02:18 -07:00
maxichrome
85c1404542 Format existing code with new conventions 2022-05-22 23:24:43 -05:00
maxichrome
2f14ff0b5f format using tabs 2022-05-22 23:07:35 -05:00
Jamie Pine
20fcc9ac66 improved sync schema 2022-05-22 20:42:41 -07:00
Jamie Pine
d03cde5a66 removed redundant schema column + some demo data (unfinished) 2022-05-21 18:48:51 -07:00
Jamie Pine
694b7e0bdd add name to jobs 2022-05-11 20:09:25 -07:00
Jamie Pine
81fa5f8a2f much 2022-05-10 09:03:20 -07:00
Jamie Pine
6922378291 better statistics 2022-04-27 20:42:28 -07:00
Jamie Pine
264293a93b added semi functional statistics, working macos traffic lights 2022-04-27 19:15:55 -07:00
Jamie Pine
ce1d932111 fix library loader, add library statistics, tag flicker fix 2022-04-27 16:25:47 -07:00
Oscar Beaumont
3ae90351a6 feature gated p2p + update deps + pushed bindings 2022-04-27 17:22:26 +08:00