Commit Graph
115 Commits
Author SHA1 Message Date
Brendan Allan 00dba54082 [ENG-678] Hidden object filtering (#895)
object hidden filter
2023-05-31 10:56:36 +00:00
Oscar BeaumontandBrendan Allan ba9ca3dd16 [ENG-674] fix invalidation (#891)
* reenable the invalidate requests system

* a tester for the invalidate system

* watch locations in Library::load

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-05-31 08:55:46 +00:00
Brendan Allan 1de37c29b3 [ENG-662] Don't use job system for shallow operations (#886)
* make it work

* use functions instead of jobs for shallow stuff

* use generalised AbortOnDrop

* simplify naming

* error stuff + alphabetical rspc types

* log errors

* don't run CI on push

* remove ephemeral

* private Worker::new

* remove comments
2023-05-30 17:07:41 +00:00
e9a22fbba0 [ENG-546] Improve error handling (#802)
* log to disk

* remove some unwraps

* panicless

* some p2p error handling

* clippy moment

* Fix `<ErrorBoundary />`

* open logs button

* 39 to 0

* fix types

* update deps and comment out broken tests

* clippy

* more clippy

* upgrade rimraf - https://github.com/isaacs/rimraf/issues/259

* regen broken lockfile

* update `notify` and update `commands.ts`

* more clippy (pls work)

* allow deprecated for p2p (hopefully temporary)

* a

* upgrade deps for p2p

* do betterer

* do it correctly

* remove unused import

* improve core startup error + bc keypair

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
2023-05-30 09:24:05 +00:00
Brendan Allan 5d2fe028bd [ENG-588] Fix total capacity on macOS (#882)
* correct macos root volume handling

* DiskType enum
2023-05-30 06:25:35 +00:00
Ericson "Fogo" SoaresandVítor Vasconcellos 8906cc631f [ENG-613] Indexer rule whitelist/blacklist per rule (#847)
* Introducing many rules on a single indexer rule

* Rust fmt

* Some doc fixes and small stuff

* Small warnings

* Making front typecheks

* Migration fix

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-05-25 19:27:31 +00:00
Brendan Allan 9995db2a32 Use search.objects for recents query (#850)
* separate search + filter + ordering

* typesafe categories

* make ordering great again

* fix ts

* eslint
2023-05-23 12:58:45 +00:00
Brendan AllanandJamie Pine ce1cf7f495 Replace getRecent with search.paths (#832)
replace getRecent with search.paths

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-05-20 22:11:43 +00:00
Jamie Pine 6d6e9c2dd7 [ENG-624] Explorer order by (#831)
added order by + fixed explorer padding situation
2023-05-19 15:51:17 +00:00
4a879594fb [ENG-581, ENG-597, ENG-600] Overview categories (#779)
* added base UI for categories on overview

* update core

* cleanup ui

* Inspector default view if nothing is selected, explorer takes child components, hidden menu tweak if no items are sm:flex

* wip

* somewhat functional

* scroll

* category fixes

* clean category bar

* added config store + made toolbar available on all explorer screens

* clean up overview.tsx

* added counts

* fix inspector bug

* add support for favorites
+ add book extension support

* refactor into smaller components

* Some small rust nitpicks

* fix camel case location_type

* Rust fmt

* fix typescript CI

---------

Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: nikec <nikec.job@gmail.com>
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-05-17 20:34:05 +00:00
Ericson "Fogo" SoaresandBrendan Allan 5da703af2c [ENG-536] Improve indexer for huge locations (#781)
* WIP materialized_path abstraction revamp

* Optimizing indexer rules loading

* Using a better serialize impl for indexer rules

* New interruptable and faster Walker

* WIP new indexer

* WIP first success compiling after breaking the world

* Fixing some warnings

* Handling some lifetime issues in the walker

* New job completed with errors feature

* Introducing completed with errors to indexer
Removing IOError variant from JobError and using FileIOError instead

* Rust fmt

* Adding missing job status

* Better ergonomics to IsolatedFilePathData
Conversions from db's file_path kinds
Keeping original's relative path data to better conversion to OS's path
First unit tests

* Testing and fixing parent method

* Some error handling

* Rust fmt

* Some small fixes

* Fixing indexer rules decoding

* Bunch of small fixes

* Rust fmt

* Fixing indexer rules

* Updating frontend to new materialized_path format

* Trying to fix windows CI

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-05-16 14:15:51 +00:00
Brendan AllanandJamie Pine 936e016000 Replace locations.getExplorerData with search.paths (#812)
* retire locations.getExplorerData from interface

* regen core.ts

* replace tags.getExplorerData with search.objects

* types out the wazooo

* better optional halding

* ternary-ify

---------

Co-authored-by: Jamie Pine <ijamespine@me.com>
2023-05-16 08:43:59 +00:00
jake 2086a6e958 [ENG-609] Remove file from recent files (#811)
allow removing a file from recents
2023-05-12 19:14:31 +00:00
jake 5e63256b27 source has_thumbnail correctly for the recent file query (#797)
* source `has_thumbnail` correctly

* correctly check if thumbnail exists in watcher

* regen bindings

* add helper fn for getting thumbnail directory and cleanup old thumbnails if the watcher creates a new one
2023-05-05 22:01:34 +00:00
nikec 0849e9d605 [ENG-575] Infinite scroll (#785)
Infinite scroll
2023-05-04 19:08:38 +00:00
Oscar BeaumontandUtku Bakir a972e64956 Upgrade rspc TS (#775)
* finally

* should be working?

* fix types

* fix types

* Wouldn't it be nice if Metro would just work

* idk

* try harder Metro

* potentially fix bundling issues

* idk, maybe fix it?

* fix metro

* update podfile.lock

* bruh

* bruhz

* tailwind is drunk again

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-05-04 06:21:42 +00:00
ameer2468andbrxken128 7d56bda3e5 ENG-287: Overview recent files (#780)
* experimental recent file querying

* working recent files

* fix querying and make amount returned customisable

* experimental recent file querying

* working recent files

* fix querying and make amount returned customisable

* return `FilePath`s from `files.getRecent`

* return `ExplorerItem` from `files.getRecent`

* Overview recents

* commands.ts

* cleanup

---------

Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
2023-05-03 17:06:09 +00:00
Brendan Allan 00a1dd300d paginated location explorer (#774)
* paginated location explorer

* remove props from GridView

* formatting
2023-05-01 09:45:25 +00:00
Brendan Allan d47b5925d9 update prisma (#773)
* update prisma

* idk
2023-04-30 23:03:23 -07:00
Vítor VasconcellosandBrendan Allan d0f8e01839 [ENG-471, ENG-472] Add dry_run parameter to some API routes and related UI changes (#750)
* Add dry_run for `location.create` and `location.addLibrary`

* Add dry_run to IndexerRuleCreateArgs
- Add invalidate_query to create and edit api method of the location route
- Adjust some code to use the new dry_run property

* `AddLocationDialog` and `IndexerRuleEditor` now validate with backend without user interaction
 - Create `useCallbackToWatchForm` to make it easier to watch form changes with an async function that also executes during component mount
 - `method` is now part of the `addLocationDialog` form schema, and a hidden input
 - Add an `index.ts` for hooks and components

* Fix mobile

* Remove redundant type definition

* Add `useCallbackToWatchForm` as an additional hook to `react-hooks/exhaustive-deps`
 - Improve `useCallbackToWatchForm` argument handling, to avoid duplicated references to form

* Fix unnecessary form value updates in `addLocationDialog`
 - Move reset `method` on `path` change logic from the superfluous `useEffect` to `useCallbackToWatchForm` in `addLocationDialog`
 - Improve core API debug logs for `dry_run`
 - Fix incorrect file name for `useCallbackToWatchForm` hook
 - Improve the documentation for `useCallbackToWatchForm`
 - Fix `useCallbackToWatchForm` not ignoring callback errors
 - Fix `useCallbackToWatchForm` ignoring returned Promise from `onWatch`
 - Add basic validation for the `form` argument in `useCallbackToWatchForm`

* Remove superflous if checks
 - generate new core.ts

* Remove DryRunError

* Remove unused import

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-04-29 10:49:51 +00:00
Brendan Allan a93103ab17 Remove all field name string literals (#772)
remove all field name string literals
2023-04-28 21:59:11 -07:00
Brendan AllanandEricson Soares ec280e35a8 File Opening (#762)
* file opening

* eslint disable in gererated commands

* bruh?

* Fixing materialized_path joining and some warnings

* no unwraps

* formatting

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-04-28 06:55:00 -07:00
Oscar Beaumont 2623843b43 Spacedrop (#761)
* A hint of file drop

* backport from #671

* accept/reject Spacedrop working

* final cleanup

* Rename spacedrop.tsx to Spacedrop.tsx

* Update index.tsx
2023-04-27 04:41:33 +00:00
Brendan AllanandOscar Beaumont d8ee053d74 Give file paths pub_id and use it primarily (#754)
* give file paths pub_id and use it primarily

* fix object indexing

* file path location + sync

* remove future plan

* use id instead of pub_id for more indexer queries

* use chain_optional_iter

---------

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2023-04-25 16:53:36 +00:00
Oscar Beaumont c5f68bc341 fix type exporting 2023-04-25 23:51:48 +08:00
Oscar Beaumont 3f1a93ffb8 fix desktop 2023-04-25 22:47:41 +08:00
Oscar Beaumont 7724c738ec Rspc v3 syntax (#758)
just fixed a couple of lines
2023-04-25 17:12:42 +08:00
Oscar Beaumont a435acf4a7 rspc 0.1.3 prep (#756)
* Upgrade rspc + Specta + PCR

* fix jsonrpc executor

* fix invalidate excessive logging

* fix `Protected` type

* fix mobile

* clippy

* lol
2023-04-25 15:14:52 +08:00
da58ba916f [ENG-490] Add support for late key manager setup (#740)
* support for late key manager setup and remove password from onboarding

* mobile

* delete master-password.tsx

* remove secure temp keystore

* remove more km_config artifacts

* fix builds

* Fix most of the UI, there still some minor bugs

* Workaround portalled closing Popovers

* Restore search in core/src/api/mod.rs

* Fix Password eye icon closing Popover on click

* Fix keymanager setup submit button spinner not centered
 - Fix keymanager setup missing logic for disabling form
 - Adjust keymanager tab roundness to match app style

* Restore .gitignore

* Restore .prettierignore

* fixerino tests

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-04-22 09:10:21 +00:00
Vítor Vasconcellos 31a87f6794 [ENG-384] Indexer rules editor (#723)
* WIP Create Indexer Rule UI + Backend api fix

* Complete IndexerEditor UI for creating new rules
- WIP testing to ensure all rules are parsed correctly
- Add utilities to satisfy the TypeScript typechecker
- Introduce a utility function to facilitate extracting information from an RSPCError
- Modify AddLocationDialog to utilize the aforementioned utility function

* Validation and submit logic is now functional
- Reorganize UI for improved user flow
- Implement validation and error messaging, replace some bare inputs with form inputs, and fix styling issues
- Resolve issues with post-processing of rules during submission
- Wrap editable portion of component in a `FormProvider` due to the `Form` being in a `Portal`
- Add specialized `ErrorMessage` component to `@sd/ui` library
- Update `AddLocationDialog` and `FormField` to utilize the new `ErrorMessage` component

* Fix submit not waiting for confirmation from backend
 - Reset form after submit
 - Disable form while submiting
 - Update form related dependencies

* Implement deleting an indexer rule
 - Modify indexer rule api route to disallow deleting default rules

* Fix form reset on error
 - Minor style adjustments
2023-04-22 04:30:11 +00:00
jakeandEricson Soares b26ac18dbe [ENG-516] Prevent copies/cuts to the same source/dest (#748)
* add basic UI protection to stop cut/copies to the same path

* add rust src/dest protections

* fix pasting logic

* fix path canonicalization

* skip paths instead of overwriting them

* Using non-blocking ops and fixing some warnings

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-04-22 02:11:59 +00:00
ameer2468andjake ca33a9d44e [ENG-473] - Job Manager UI Improvements (#745)
* add support for deleting a specific job

* wip

* Update JobManager.tsx

* Job manager design tweaks + popover open state ignore prop

* tweaks

---------

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
2023-04-21 17:08:32 -07:00
Brendan Allan 356519848a Explorer search (#738)
* basic file path search

* cleanup

* render explorer in search page

* update materialized path docs
2023-04-21 14:49:46 +00:00
nikecandEricson Soares d4eb84e7e2 [ENG-295] Explorer media view (#730)
* remove unused onScroll

* Add open media file with double click

* Hide file thumb in inspector

* Enable media layout

* Add media layout options

* Add media view

* Fix vertical images

* Fix vertical square aspect

* query

* Update Location.tsx

* Some optimizations on explorer data query

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-04-21 02:15:22 +00:00
Oscar BeaumontandEricson Soares 5f5afccaba Eng 494 adding a location does not update the UI (#728)
* `sd_init.json` support

* bruh

* Add `sd_init.json` to developer docs

* Ran cargo clippy --fix to fix warnings

* Dafuq, cargo clippy --fix messed up cargo fmt

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-04-20 22:05:57 +00:00
Oscar BeaumontandEricson Soares 54f757bf1f Job system improvements v2 (#707)
* Add `JobInitData` trait + change `StatefulJob::name` to a constant

* `StatefulJob::new`

* Move `invalidate_query` into `StatefulJob::finalize`

* Cleanup `spawn_job` signature

* Remove `queue_job` and run jobs from the `StatefulJob::finalize` method

* `StatefulJob::queue_jobs`

* `JobManager::ingest` return `Result`

* Remove `jobs.isRunning`

* Invalidation system direct push + batching invalidate

* Look ma, only a single clippy warning!!!!

* Error handling for JobManager

* Rust fmt

* Introducing Job hierarchy to enable job enqueuing

* Rust fmt again 🙄

* core.ts

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-04-15 14:43:04 +08:00
096741a8fd [Desktop] Renamable files (#675)
* Add rename file option

* make Ericson's changes

* Prevent arrow selecting when renaming, start renaming with Enter option

* fix rename

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Jamie Pine <ijamespine@me.com>
2023-04-11 20:43:19 -07:00
Brendan Allan 3792cf1034 fix ts (#661)
* fix ts

* eslint
2023-03-31 00:22:54 -07:00
Ericson "Fogo" Soares 2609f69282 Eng 372 location awareness improvements (#612)
* Complying with a pedantic Clippy

* Solving duplicated directories events
When creating a directory through MacOS's Finder, for some reason
FSEvents receives 2 Create Folder events that we have to handle

* Handling moving to trash bin and restoring on Mac
Still missing the feature to restore a directory and its children

* Now handling creation of empty files on MacOS

* Enabling restore of directories and its children

* Now working: moving a directory to another
inside the same location

* Now Indexer also remove file_paths not on fs

* Enabling multiple file moves on location watcher

* Fix Windows conditional compilation issues

* Fixing cas_id generation and bumping some deps

* Many Windows specific improvs and some refactors

* Rust fmt

* Using conditional compilation on extract inode function

* Linux fixes and some MaterializedPath improvements

* Rust fmt again

* Introducing tick behavior on location watchers

* Making LastFilePathIdManager more atomic

* Some vscode launch configs for lldb debugger

* Simplifying some lifetimes

* Making all watchers more consistent
2023-03-31 04:59:33 +00:00
Oscar BeaumontandBrendan Allan 0cafbf8aea Sync over p2p (#606)
* sync no networkey

* sync technically wired up

* fix p2p endianness for sync payload

* some sync debug stuff

* feature gate message production

* bump lockfile

* execute queries when sync messages disabled

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-03-21 06:03:08 -07:00
Jamie Pine 0c65f48d5e [ENG-441] Explorer size toggle (#611)
explorer size toggle + light theme icons start + radix checkbox
2023-03-16 18:59:06 -07:00
jake f487db254b [ENG-429] Crypto organization/refactor (#607)
* update comment

* key manager feature gating

* update deps & more feature gating

* fix keyring feature gating

* add an `exhaustive_read` function for crypto/stream

* restructure STREAM module

* move tests+shared utils

* clean up stream decryption

* further cleanup

* impl to `GenericArray` for `Nonce`

* update examples

* update refs & `use`s

* fix `Nonce` -> `GenericArray` conversions

* better `Protected` conversions + remove `Password` type

* a work of art

* finishing touches

* some API changes

* rename `StreamX` to `X`

* fix everything else

* separate `primitives` from `types`

* update imports & fix build
2023-03-15 16:09:36 +00:00
Ericson "Fogo" Soares 12a437ffb6 [ENG-371] Subpath shallow re indexing (#599)
* Introducing sub path indexing for IndexerJob

* Introducing shallow indexer job

* Some feature flags to avoid warnings at pnpm prep

* Many fixes at shallow indexer job

* Changing how we implement the bookkeeping for file_path ids
Now we're account for independent locations, and also integrating
better with the LibraryContext, instead of using a static global

* Making indexer job reentrant

* Introducing shallow file identifier job
And a bunch of minor refactors

* Rust fmt

* Removing light scan from getExplorerData query
Light scan is a mutation, so we can call it on useEffect function from
the Explorer component in the frontend, when a location_id or the explorer
path changes

* Handling job early finish on init for identifier

* Only invalidate query if we have orphan paths

* Introducing ShalowThumbnailerJob

* Clippy warnings about `into_iter()`

* Naming scheme for Prisma's selects and includes

* Invalidating getExplorerData at thumbnailer

* Small mistakes lol

* Some nitpicks with pnpm prep

* Rust fmt

* Changing indexer's walk log to `trace!`

* Not deleting all file_paths on location fullRescan

* TS typecheck

* Removing `file_path` selection just with id
2023-03-10 16:25:58 +00:00
Oscar Beaumont 7588442d5d Basic Spacedrop working with rspc! (#601) 2023-03-09 06:23:17 -08:00
Brendan Allan 25b4b72cf1 rename LibraryContext to Library (#595) 2023-03-07 01:15:20 -08:00
Brendan Allan c2d8c2f072 More sync impls (#586)
tags and stuff
2023-03-02 21:23:10 -08:00
Ericson "Fogo" Soares ae41b1a0f4 Fix unique constraint on file path table (#585)
* Complying with a pedantic Clippy

* Some task runners for vscode

* Solving duplicated directories events
When creating a directory through MacOS's Finder, for some reason
FSEvents receives 2 Create Folder events that we have to handle

* Damn rustfmt
2023-03-02 02:36:49 +00:00
Jamie PineandBrendan Allan 963584f065 [ENG-379] Explorer item resizer (#580)
* added item resizer
fixed explorer store bug
refactored file image component

* better sizing for videos

* fixed inspector width issue

* remove console.log

* added column titles to list view + extra details

* moved util

* remove imports

* Update packages/interface/src/components/explorer/FileColumns.tsx

Co-authored-by: Brendan Allan <brendonovich@outlook.com>

* address issues

* fix extension in file list name

* Update packages/interface/src/components/explorer/FileColumns.tsx

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-02-25 06:16:57 +00:00
Jamie PineandBrendan Allan fab84b7f7a [ENG-363] Spacedrop UI + Misc Improvements (#568)
* begin spacedrop ui + misc ui improvements

* better 404 xox

* Update extensions.rs

I think I prefer Container

* added DragRegion component,  ot tested cuz im on my fone

* Update DragRegion.tsx

fix import

* added dummy drop items

* better dummy data

* added clouds & search bar

* added action buttons to spacedrop items

* customize subtle button

* added support for apng, thanks luka big pants

* use relative path in sidebar

* use BYTES const

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-02-24 08:12:21 +00:00
Oscar BeaumontandEricson Soares 60d5bc4afe improvements to spacedrive:// custom URI protocol (#550)
* fix `spacedrive://` custom protocol on Windows
(hopefully)

* custom protocol using `http::Response` + fix broken web

* import patches before App on web

* use `http::Request` for input to `handle_custom_uri`

* break  into dedicated file + error handling

* serving files via custom protocol

* cargo fmt because vscode did cringe

* lru cache to reduce video chunk request time

* add  helper to JS

* clippy be like

* remove duplicate Open buttons in context menu

* fix Linux 🙏

* no shot

* fix Windows custom URI passing (hopefully)

* better fix for custom uri on Linux

* upgrade Tauri for  feature

* switch url replacement order

* prevent React dev tools script being added in prod to desktop

* remove React devtools from html

* upgrade Tauri; required upgrading rspc, Axum, PCR

* pass typecheck + less cringe bigint

* clippy is love, clippy is life

* Typecheck plz

* fix bigint to number conversion

* use httpz + localhost server for Linux

* clippy be right

* Remove console.log

* [wip] proper auth

* fix Linux sidebar padding

* Secure Axum server with random

* Extracting app setup specific to linux to a different file

* remove outdated comment

* Some tweaks on cursom_uri.rs

* file_path_with_location doesn't need to be a named include

* fix typo

* factually wrong comment

* Change `unwrap` to `expect`

* bruh

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-02-14 13:27:11 +08:00